Add job_runs lifecycle.triggers.on_value_change - #6341
Open
radakam wants to merge 78 commits into
Open
Conversation
Re-fire a run when matched file contents change, using stable hash fingerprints so mtime-only updates (touch) do not recreate.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
The modernize linter rejects the manual m[k]=v copy loop, failing CI lint.
Honor .gitignore and sync.exclude when hashing matches. Cover mysubdir globs, READPLAN, and keep LF on Windows so restored files hash cleanly.
The glob test edits migrations/ignored.txt, but the test .gitignore excluded it from the commit so CI clones would miss the file.
Plan serialization is already covered by on_file_change; the glob cases only assert the hash map.
Cover deploying glob hashes from a saved plan, not only from a live re-plan.
Keep the gitignore fixture as dot_gitignore so it does not affect the repo while developing, drop unit tests that duplicate acceptance coverage, and allocate trigger state once when a trigger is armed.
The previous simplification omitted OnBundleDeploy from the struct literal, which the linter rejects.
Always persist the same lifecycle.triggers shape so trigger fingerprints compare at the same paths. Wrap on_file_change hashes in files so dropping the trigger updates instead of recreating.
Nest lifecycle by value so structdiff descends to the leaf that changed instead of reporting the whole subtree, and let RemapState supply the empty shape. GetRun never returns the fingerprints, so the remote type no longer advertises a lifecycle it cannot fill.
radakam
marked this pull request as ready for review
August 21, 2026 15:59
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: 4f50a39
Top 9 slowest tests (at least 2 minutes):
|
Drop the files wrapper so plan diffs land on lifecycle.triggers.on_file_change, matching the config path without an extra state-only level.
The trigger, PrepareState and OverrideChangeDesc cases are asserted end to end by on_bundle_deploy and on_file_change, so the unit copies only duplicated goldens. Keep the ones acceptance cannot reach without stubbing GetRun.
Flattening on_file_change to a map made an unarmed trigger drop out of the serialized state, so the three plans that print an unarmed job_run now report an empty triggers object.
The unit test staged an asynchronous cancellation by hand; the acceptance test shows the real request order a user's destroy produces. Also drops two unit tests that the on_file_change acceptance tests already cover.
Re-fire a run when a resolved expression changes, including resource refs that become deploy-graph edges. Removing the trigger is a state-only update.
Include the on_value_change input and state paths required by generated-file validation.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-value-change
branch
from
August 24, 2026 09:36
0db1f4b to
e431088
Compare
Re-fire a run when matched file contents change, using stable hash fingerprints so mtime-only updates (touch) do not recreate.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
Satisfy the linter on PrepareState literals after adding OnFileChange, and refresh out.fields.txt so validate-generated stays green.
Reject patterns outside the sync root and directory-only matches so the trigger cannot fingerprint escaped paths or silently disarm.
Drop size/mtime fingerprints and the resolve fast path so content identity alone drives recreate, and touch no longer needs a planner exception.
The modernize linter rejects the manual m[k]=v copy loop, failing CI lint.
Honor .gitignore and sync.exclude when hashing matches. Cover mysubdir globs, READPLAN, and keep LF on Windows so restored files hash cleanly.
The glob test edits migrations/ignored.txt, but the test .gitignore excluded it from the commit so CI clones would miss the file.
Plan serialization is already covered by on_file_change; the glob cases only assert the hash map.
Cover deploying glob hashes from a saved plan, not only from a live re-plan.
Keep the gitignore fixture as dot_gitignore so it does not affect the repo while developing, drop unit tests that duplicate acceptance coverage, and allocate trigger state once when a trigger is armed.
The previous simplification omitted OnBundleDeploy from the struct literal, which the linter rejects.
Always persist the same lifecycle.triggers shape so trigger fingerprints compare at the same paths. Wrap on_file_change hashes in files so dropping the trigger updates instead of recreating.
Nest lifecycle by value so structdiff descends to the leaf that changed instead of reporting the whole subtree, and let RemapState supply the empty shape. GetRun never returns the fingerprints, so the remote type no longer advertises a lifecycle it cannot fill.
Drop the files wrapper so plan diffs land on lifecycle.triggers.on_file_change, matching the config path without an extra state-only level.
Flattening on_file_change to a map made an unarmed trigger drop out of the serialized state, so the three plans that print an unarmed job_run now report an empty triggers object.
Directory-only and sync-excluded matches silently disarmed the trigger. Treat them as errors, warn that ** is not recursive, and cover both in acc.
The goldens were captured against a one-line-taller databricks.yml, so CI failed after the source settled on line 24.
Clearing a trigger is now a skip, so the plan reports the run as unchanged and the deploy makes no API call.
Patterns must still land under the sync root. Reject **, warn when a glob matches directories, and skip only when the trigger itself is cleared.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
from
August 25, 2026 08:12
17857f9 to
45c4ccd
Compare
filepath.Join rebases an absolute glob under the bundle root, so reject those up front. Key missing-file placeholders to the sync root, and skip a cleared trigger from path.String() instead of prefix matching.
…s-lifecycle-on-value-change The base branch was rebased and gained YAML-relative on_file_change paths, absolute-pattern rejection, and a reworked prevent_destroy diagnostic. Adapt on_value_change to the new shape: extend preventDestroyError to name every armed trigger instead of listing two by hand, fold on_value_change into the base's path.String() switch for cleared triggers, and drop the jobRunTriggerLocalPaths helper the base replaced.
radakam
force-pushed
the
deco-job-runs-lifecycle-on-file-change
branch
from
August 25, 2026 10:12
d65e640 to
b0437a3
Compare
…s-lifecycle-on-value-change The base was rebased onto a newer main and softened the on_file_change "nothing to hash" error into a warning. Adapt on_value_change: follow the base in dropping the unreachable "lifecycle.triggers" case from OverrideChangeDesc, keeping on_value_change alongside on_file_change on the leaf paths, and take the zero-value emptyJobRunLifecycleState now that it no longer lists fields by hand. Also restore the initialize.go ApplySeqContext formatting that the previous merge reverted.
preventDestroyError took one bool per trigger, so every new trigger widened the signature and risked a swapped argument. Have JobRun report the armed names instead.
extractReferences validated a config path against the state type with the Terraform rule that treats [0] on a struct as a no-op. That let lifecycle.triggers[0].on_value_change match the flattened state map and register a reference that cannot resolve. ValidatePathStrict drops the tolerance for these native DABs paths.
State kept each watched value verbatim and keyed it by the expression, so a long or sensitive resolved value ended up in the plan and in state. Key the entries by the value itself and hash it once it grows past the digest, normalizing through structvar as soon as the last reference resolves. With the expression gone from the key, dropping an entry can only mean the config stopped watching that value, so it skips instead of re-firing the run. DropJobRunValueChangeConfigRefs goes away with the strict path validation.
Keying state by value treated two watches converging as a removal, so the run did not re-fire. A list in trigger order distinguishes a changed fingerprint from a dropped watch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Add
lifecycle.triggers.on_value_changeforresources.job_runs(direct engine). The run is recreated when the resolved expression changes, and${resources.*}refs in the expression become deploy-graph dependencies. Multiple watched values are separate list entries (one key each).State keys each entry by the resolved value rather than by the expression text, and records a
sha256:digest once the value is longer than the digest itself, so a long value does not land in the plan or in state. Since the key is the value, an entry can only disappear when the config stops watching it: removing a trigger is unchanged and does not start a new run (same ason_file_change/on_bundle_deploy).Supporting change:
extractReferencesvalidated config paths against the state type using Terraform's rule that treats[0]on a struct as a no-op, solifecycle.triggers[0].on_value_changematched the flattened state map and registered a reference that can never resolve.structaccess.ValidatePathStrictdrops that tolerance for paths that are native to DABs.Why
job_runsalready supporton_bundle_deployandon_file_change. Watching a resolved value (a variable or a resource id) is the remaining trigger from the job_runs design, so a run can re-fire when an upstream resource or interpolated value changes.Tests
validate_job_run_triggers(empty/whitespace/two keys/prevent_destroy); value fingerprinting, and normalization once the last reference resolves;OverrideChangeDesc, including that anon_file_changeentry dropping out still recreates;extractReferencesrejects the config list path.on_value_change: three watched values, one long enough to be hashed; changing one variable recreates while the other fingerprints stay; removing one trigger, then the last, is unchanged with no run-now.on_value_change_ref: concatenated${resources.jobs.other.id}-${resources.jobs.extra.id};depends_onincludes both jobs; persisted state is[OTHER_ID]-[EXTRA_ID]; recreatingotherre-fires the run;READPLAN=""andREADPLAN=1.