Skip to content

Fail deferred Cloud Composer tasks when the GCP operation errors#70430

Open
SEPURI-SAI-KRISHNA wants to merge 1 commit into
apache:mainfrom
SEPURI-SAI-KRISHNA:fix-composer-trigger-error-detection
Open

Fail deferred Cloud Composer tasks when the GCP operation errors#70430
SEPURI-SAI-KRISHNA wants to merge 1 commit into
apache:mainfrom
SEPURI-SAI-KRISHNA:fix-composer-trigger-error-detection

Conversation

@SEPURI-SAI-KRISHNA

Copy link
Copy Markdown

CloudComposerExecutionTrigger.run() checked operation.done before operation.error, in an if/elif. Per the long-running operation contract, "if done == false, neither error nor response is set", and a finished operation has done=True with exactly one of error or response populated.

So the two branches were inverted relative to the API:

  • the elif operation.error.message branch was only ever reached while done was false — the one state where error is guaranteed unset, making it dead code;
  • a genuinely failed operation (done=True with error set) broke out of the loop and yielded operation_done: True, the success shape.

The consequence differs per operator, all three of which defer with this trigger:

Operator execute_complete Result when the operation failed
CloudComposerUpdateEnvironmentOperator returns on operation_done, no state check task marked success
CloudComposerDeleteEnvironmentOperator pass task marked success
CloudComposerCreateEnvironmentOperator re-fetches the environment and validates its state task fails, but with a misleading NotFound/wrong-state error rather than the GCP failure reason

This also made the deferrable path disagree with the synchronous one: with deferrable=False the operators call hook.wait_for_operation()operation.result(), which raises on a failed operation.

Checking error inside the done branch fixes all three. The existing raise is relocated, not new.

CloudComposerExecutionTrigger had no test coverage, so this adds two cases: a finished-with-error operation must raise, and a finished-without-error operation must still yield operation_done: True. The first fails without this change; the second guards the new nested condition against over-raising on success.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

@boring-cyborg boring-cyborg Bot added area:providers provider:google Google (including GCP) related issues labels Jul 25, 2026
@boring-cyborg

boring-cyborg Bot commented Jul 25, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant