Skip to content

Add airflow dags stability check CLI command#70446

Open
202201142 wants to merge 4 commits into
apache:mainfrom
202201142:add-dag-stability-command
Open

Add airflow dags stability check CLI command#70446
202201142 wants to merge 4 commits into
apache:mainfrom
202201142:add-dag-stability-command

Conversation

@202201142

@202201142 202201142 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Add a new CLI command to detect non-deterministic DAG serialization by parsing and serializing DAGs twice and comparing the resulting hashes.

Non-deterministic DAG definitions can cause unnecessary dag_version changes, making deployments and code reviews harder. This command helps identify such DAGs and can be used in CI/CD to catch serialization instability before merging.

If the serialized hash differs between parses for the DAG, the command reports the differences between the serialized representations and exits with status code 1.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request introduces a new airflow dags stability CLI subcommand intended to detect unstable Dag serialization across repeated parses (a potential contributor to dag_version inflation). It wires the command into the CLI configuration, adds unit tests to validate pass/fail behavior, and updates the FAQ to mention the new workflow option.

Changes:

  • Add dag_stability_check implementation that parses Dags twice and reports differing serialization via unified diffs.
  • Register a new airflow dags stability [dag_folder] command and --fail-fast flag in the CLI.
  • Add unit tests covering stable/unstable Dags and missing --dag-id handling, plus a small FAQ update.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File Description
airflow-core/src/airflow/cli/commands/dag_command.py Implements Dag stability check helpers and the new CLI action.
airflow-core/src/airflow/cli/cli_config.py Adds CLI args and registers the new dags stability command.
airflow-core/tests/unit/cli/commands/test_dag_command.py Adds unit tests validating stability check success/failure behavior.
airflow-core/docs/faq.rst Mentions the new CLI command as an additional way to catch Dag version inflation issues.

Comment thread airflow-core/src/airflow/cli/commands/dag_command.py
Comment thread airflow-core/src/airflow/cli/commands/dag_command.py Outdated
Comment thread airflow-core/src/airflow/cli/commands/dag_command.py Outdated
Comment thread airflow-core/src/airflow/cli/commands/dag_command.py Outdated
Comment thread airflow-core/src/airflow/cli/cli_config.py
Comment thread airflow-core/docs/faq.rst Outdated
202201142 and others added 3 commits July 25, 2026 22:48
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants