Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases
Airflow 3.3.0 shipped on 6 July 2026, with 3.3.1 following about five weeks later. It's a much smaller release than 3.0 was, and that's fine. Nothing about how you write Dags has changed. What did change is a set of things most of us have been working around for years. Tasks that need to remember a cursor across a retry. Daily jobs that fire before all their hourly inputs land. Retry loops that…
On 6 July 2026, Apache Airflow 3.3.0 was released, followed by 3.3.1 about five weeks later. While the user interface and the way DAGs are written remained unchanged, there were several improvements that addressed issues data engineers have been dealing with for years. This article will discuss the key features of Airflow 3.3, the problems each feature solves, and how they can be applied in real-world use cases.
Airflow is an open-source orchestrator that allows users to describe workflows as Directed Acyclic Graphs (DAGs) in Python. The scheduler determines when each task runs, in what order, and how to handle failures. It doesn't move data itself, but rather integrates with various systems like warehouses, Spark, dbt, object storage, and other APIs. The main benefits of Airflow are its ability to handle dependency resolution, retries, backfills, and maintain a record of what ran.
One of the most significant additions in Airflow 3.3 is the Task and Asset State Store, which allows tasks to persist arbitrary key-value state that survives retries and reruns. Previously, states were managed using XComs, Variables, or side tables in a separate database, all of which had their shortcomings. Now, task state and asset state are stored in the metadata database by default, with per-key retention and periodic garbage collection. There are options for clearing task state after a successful run and limiting REST API write sizes.
An example of the new state store is the ResumableJobMixin, which is now used by the SparkSubmitOperator. This allows workers to reconnect to an existing Spark job upon retry instead of submitting a second one, saving time and resources. Another improvement is the expanded asset partitioning, which was introduced in 3.2. With 3.3, there are better controls for mapping upstream events to downstream runs, including FanOutMapper, FixedKeyMapper paired with SegmentWindow, wait policies, windows that enumerate forward or backward from an anchor, and a runtime PartitionedAtRuntime timetable.
This allows for more precise scheduling and avoids issues like firing a daily summary on the first event of the day or manually offsetting cron jobs.
These improvements in Airflow 3.3 greatly enhance the capabilities of data engineering pipelines, making them more reliable, efficient, and easier to manage.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
