Skip to content

Fix TimeSensor start_from_trigger causing dag_version churn#69610

Open
bramhanandlingala wants to merge 4 commits into
apache:mainfrom
bramhanandlingala:fix/#69543
Open

Fix TimeSensor start_from_trigger causing dag_version churn#69610
bramhanandlingala wants to merge 4 commits into
apache:mainfrom
bramhanandlingala:fix/#69543

Conversation

@bramhanandlingala

Copy link
Copy Markdown
Contributor

closes: #69543

TimeSensor computed its target time using datetime.now() at DAG-parse
time. When start_from_trigger=True, this value got baked into the
serialized DAG, causing the dag_version to change on every parse that
crossed a day boundary.

This can't be fixed by computing the value differently, since the target
moment is inherently different each day but start_trigger_args is only
computed once per DAG version. So start_from_trigger=True now raises a
ValueError instead of silently producing a broken serialization.
deferrable=True still works as before and is unaffected.

Also fixed the docs, which incorrectly said the target time is evaluated
against data_interval_end/run_after — it never was.

Added a test for the new ValueError; all existing tests still pass.

@bramhanandlingala

Copy link
Copy Markdown
Contributor Author

Hi @bugraoz93, @choo121600, @ephraimbuddy, @henry3260, @jason810496, @rawwar @kaxil @Lee-W
Request You All, please review and Merge as need for this fix

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dag_version inflection when using TimeSensor with start_from_trigger = True

3 participants