fix: mark DAGs as stale when their bundle is removed from config #67271#67431
fix: mark DAGs as stale when their bundle is removed from config #67271#67431bramhanandlingala wants to merge 2 commits into
Conversation
…che#67271) When a DAG bundle is removed from `dag_bundle_config_list`, `sync_bundles_to_db` correctly sets `bundle.active = False` but never updated the associated `DagModel` rows to `is_stale = True`. This caused removed-bundle DAGs to remain visible in the UI as active workflows. Add a bulk UPDATE in the removed-bundle loop so that all DAGs belonging to the deactivated bundle are immediately marked stale, and extend the fixture teardown and add a regression test to cover this path.
|
@jedcunningham @ephraimbuddy thanks in advance |
|
@bramhanandlingala — I've removed the Automated triage note drafted by an AI-assisted tool — may get things wrong; a real Apache Airflow maintainer takes the next look once it's green. (why automated) Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting |
ephraimbuddy
left a comment
There was a problem hiding this comment.
This was fixed in 3.2.2. See PR: #66948
Blocking merge. Can you tell me if you reproduced this in 3.2.2 or latest main?
Problem
When a DAG bundle is removed from dag_bundle_config_list, sync_bundles_to_db logs "DAG bundle X is no longer found in config and has been disabled" and correctly sets bundle.active = False, but it never updates the associated DagModel rows to is_stale = True. As a result, the removed bundle's DAGs remain visible in the UI as functional workflows.
Root cause
In DagBundlesManager.sync_bundles_to_db (airflow-core/src/airflow/dag_processing/bundles/manager.py), the loop that handles bundles absent from config only sets bundle.active = False and cleans up import errors. There is no statement to flip DagModel.is_stale.
Changes
airflow-core/src/airflow/dag_processing/bundles/manager.py
Added update to the sqlalchemy import.
Added a local import of DagModel (alongside the existing ParseImportError local import, to avoid circular imports).
After marking a bundle inactive, execute a bulk UPDATE dag SET is_stale = true WHERE bundle_name = .
airflow-core/tests/unit/dag_processing/bundles/test_dag_bundle_manager.py
Updated the clear_db fixture to also clear DagModel rows before/after tests.
Added test_sync_bundles_to_db_marks_dags_stale_on_bundle_removal, creates a bundle, adds a non-stale DAG for it, removes the bundle from config, and asserts dag.is_stale is True.