{
  "id": 5150930,
  "title": "Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases",
  "url": "https://urgent.news/2026/09/02/understanding-apache-airflow-3-3-key-improvements-features-every-data",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-02T19:14:57.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/adev3loper/understanding-apache-airflow-33-key-improvements-features-every-data-engineer-should-know-and-32b2"
  },
  "original_language": "en",
  "account": "On 6 July 2026, Apache Airflow 3.3.0 was released, followed by 3.3.1 about five weeks later. While the user interface and the way DAGs are written remained unchanged, there were several improvements that addressed issues data engineers have been dealing with for years. This article will discuss the key features of Airflow 3.3, the problems each feature solves, and how they can be applied in real-world use cases.\n\nAirflow is an open-source orchestrator that allows users to describe workflows as Directed Acyclic Graphs (DAGs) in Python. The scheduler determines when each task runs, in what order, and how to handle failures. It doesn't move data itself, but rather integrates with various systems like warehouses, Spark, dbt, object storage, and other APIs. The main benefits of Airflow are its ability to handle dependency resolution, retries, backfills, and maintain a record of what ran.\n\nOne of the most significant additions in Airflow 3.3 is the Task and Asset State Store, which allows tasks to persist arbitrary key-value state that survives retries and reruns. Previously, states were managed using XComs, Variables, or side tables in a separate database, all of which had their shortcomings. Now, task state and asset state are stored in the metadata database by default, with per-key retention and periodic garbage collection. There are options for clearing task state after a successful run and limiting REST API write sizes.\n\nAn example of the new state store is the ResumableJobMixin, which is now used by the SparkSubmitOperator. This allows workers to reconnect to an existing Spark job upon retry instead of submitting a second one, saving time and resources. Another improvement is the expanded asset partitioning, which was introduced in 3.2. With 3.3, there are better controls for mapping upstream events to downstream runs, including FanOutMapper, FixedKeyMapper paired with SegmentWindow, wait policies, windows that enumerate forward or backward from an anchor, and a runtime PartitionedAtRuntime timetable. This allows for more precise scheduling and avoids issues like firing a daily summary on the first event of the day or manually offsetting cron jobs.\n\nThese improvements in Airflow 3.3 greatly enhance the capabilities of data engineering pipelines, making them more reliable, efficient, and easier to manage.",
  "summary": "Airflow 3.3.0 shipped on 6 July 2026, with 3.3.1 following about five weeks later. It's a much smaller release than 3.0 was, and that's fine. Nothing about how you write Dags has changed. What did change is a set of things most of us have been working around for years. Tasks that need to remember a cursor across a retry. Daily jobs that fire before all their hourly inputs land. Retry loops that…",
  "key_points": [
    "Apache Airflow 3.3 released on 6 July 2026",
    "Task and Asset State Store introduced for persistent key-value state",
    "ResumableJobMixin and expanded asset partitioning enhance reliability and efficiency"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}