Skip to content

postgres: fix deploy failing with unknown field path in update_mask - #6374

Draft
denik wants to merge 1 commit into
mainfrom
denik/update-mask-fix
Draft

postgres: fix deploy failing with unknown field path in update_mask#6374
denik wants to merge 1 commit into
mainfrom
denik/update-mask-fix

Conversation

@denik

@denik denik commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Why

Deploying a postgres_project whose custom_tags carry a value that changes each deploy failed with Unknown field path in update_mask: 'spec.custom_tags[0].value'. The mask came from the plan's change list, which addresses a list element by index; the API only accepts the whole repeated field. Reported by a customer.

Changes

Each postgres resource gets a static map from spec field path to the mask path the API accepts, and the mask is derived from the request body so the two always agree. Probing the API showed that neither * nor a static list works: the backend expands the mask to leaves and requires every masked leaf to be populated in the request, which a bundle that omits optional fields never is. Two more paths differ from the field name — members of a oneof are masked under the group name (spec.expiration, spec.suspension), and immutable fields have no path at all. All five postgres resources shared the bug.

Tests

New acceptance test postgres_projects/update_custom_tags reproduces the failure and asserts the tag value the API actually applied. TestPostgresSpecUpdateMasks walks each spec with reflection, so a new SDK field fails the test until it has a mask entry. The fake server now rejects an indexed mask path like the real API does. Ran the whole postgres suite against a real AWS workspace.

Updates to a duration or timestamp field (spec.suspension, spec.expiration) are not covered: structdiff sees duration.Duration and types/time.Time as always equal because each wraps a single unexported protobuf pointer, so the direct engine never plans an update for them. Separate bug, separate fix.

Deploying a postgres_project whose custom_tags carry a value that changes
each deploy failed with

    400 INVALID_PARAMETER_VALUE
    Unknown field path in update_mask: 'spec.custom_tags[0].value'

The mask was built from the plan's change list, which addresses a list
element by index. The API only accepts the whole repeated field.

Probing the API shows a static list or "*" does not work either: the
backend expands the mask to leaves and requires every masked leaf to be
populated in the request body, which a bundle that omits optional fields
never is. Two more paths differ from the field name: members of a oneof
are masked under the group name (spec.expiration, spec.suspension), and
immutable fields have no path at all.

So each resource gets a static map from spec field path to accepted mask
path, and the mask is derived from the request body so the two always
agree. All five postgres resources shared the bug.

Co-authored-by: Isaac
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

16 files changed
Suggested: @pietern
Also eligible: @janniklasrose, @shreyas-goenka, @andrewnester, @anton-107, @lennartkats-db

/bundle/ - needs approval

9 files changed
Suggested: @pietern
Also eligible: @janniklasrose, @shreyas-goenka, @andrewnester, @anton-107, @lennartkats-db

General files (require maintainer)

Files: .nextchanges/bundles/postgres-update-mask.md, libs/testserver/postgres.go
Based on git history:

  • @pietern -- recent work in bundle/direct/dresources/, libs/testserver/, acceptance/bundle/resources/postgres_endpoints/update_autoscaling/

Any maintainer (@andrewnester, @anton-107, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rugpanov, @rclarey) can approve all areas.
See OWNERS for ownership rules.

@denik
denik marked this pull request as draft August 25, 2026 09:24
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: aa81513

Run: 32831243755

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 1 4 288 1176 5:33
💚​ aws windows 1 4 290 1174 7:32
💚​ azure linux 1 4 273 1181 3:52
💚​ azure windows 1 4 275 1179 3:22
🔄​ gcp linux 2 1 4 272 1181 4:32
🔄​ gcp windows 6 1 4 270 1179 4:40
13 interesting tests: 8 flaky, 4 SKIP, 1 RECOVERED
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestFsCpDirToDirWithOverwriteFlag/dbfs_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFsCpFileToDir ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFsCpFileToDir/dbfs_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFsCpFileToNonExistentDir/uc-volumes_to_dbfs ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFsCpFileToNonExistentDir/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestFilerReadDir ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
🔄​ TestFilerReadDir/files ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:44 aws windows TestAccept
3:17 azure windows TestAccept
3:12 gcp windows TestAccept
2:18 aws windows TestAccept/bundle/resources/postgres_endpoints/update_autoscaling/DATABRICKS_BUNDLE_ENGINE=terraform
2:13 aws windows TestAccept/bundle/resources/postgres_databases/update/DATABRICKS_BUNDLE_ENGINE=terraform
2:05 aws windows TestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=terraform

denik added a commit that referenced this pull request Aug 25, 2026
The direct engine now sends the PATCH, and the real API rejects the mask path
it builds: suspend_timeout_duration is a oneof member, so the API only accepts
the group name spec.suspension. Separate bug, separate fix (#6374). The
testserver ignores update_mask, which is why the local run is unaffected.

Co-authored-by: Isaac
denik added a commit that referenced this pull request Aug 25, 2026
The direct engine now sends the PATCH, and the API rejects the mask path it
builds: suspend_timeout_duration is a member of the suspension oneof, so only
the group name spec.suspension is accepted. Separate bug, separate fix (#6374).

Teach the testserver to validate update_mask against the paths the real API
accepts, so the failure reproduces locally rather than only on aws. The engines
diverge from the second deploy on, so those steps go back to per-engine files,
and the deploy requests are recorded per engine: same body, different mask.

Cloud stays enabled, with the golden recording what the real workspace does.

Co-authored-by: Isaac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants