fix(Spanner): prevent unnecessary GCE metadata probing during client …#9327
Merged
Conversation
Collaborator
|
This looks GTM but for some reason the Emulator System Tests are failing, I don't think this is related though 🤨 |
cy-yun
force-pushed
the
fix-gce-metadata-probe-delay
branch
from
July 8, 2026 19:53
b04d25e to
0dcf2c8
Compare
Contributor
Author
You're right. It's not related. It's a flakey test that I'm addressing in #9329 The Spanner Emulator occasionally drops connections and triggers our automatic retry logic after it has already committed a row, causing the retry attempt to crash. I'll be swapping those tests to use idempotent insertOrUpdate() calls to fix the flakiness. |
Changes insert to insertOrUpdate and insertBatch to insertOrUpdateBatch in Transaction and PartitionedDML system tests. This prevents ConflictException (ALREADY_EXISTS) when tests are retried or a transaction closure is retried after a transient emulator error (e.g. ABORTED) where the initial mutation was still applied.
Hectorhammett
approved these changes
Jul 20, 2026
bshaffer
approved these changes
Jul 20, 2026
bshaffer
enabled auto-merge (squash)
July 20, 2026 21:56
cy-yun
disabled auto-merge
July 20, 2026 22:02
cy-yun
enabled auto-merge (squash)
July 20, 2026 22:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: Prevent unnecessary GCE metadata probing during
SpannerClientinitialization**When a
SpannerClientis instantiated, it automatically callsconfigureMetrics(). Previously, this method unconditionally resolved the Google Compute Engine (GCE) location by calling$this->getLocation().In non-GCE environments (such as local development environments, emulators, or external CI pipelines), this triggers an HTTP probe to the GCE metadata server (
169.254.169.254) that ultimately times out. This timeout introduces a completely unnecessary ~1.5 second delay to every single client instantiation.Because the Spanner system tests repeatedly instantiate the
SpannerClient, this delay heavily impacted the overall test suite execution time.Changes Made
$location = $this->getLocation();resolution below the early return for when built-in metrics are disabled (which is the default behavior).testSpannerClientInstantiatesWithoutDelayWhenMetricsDisabled()to assert that theSpannerClientinstantiates in under 1.0 second when metrics are disabled, ensuring no future regressions re-introduce the metadata probe delay.Impact
By completely bypassing the metadata server probe for the default use case, this instantly recovers 1.5 seconds per
SpannerClientinstantiation in non-GCP environments. This dramatically speeds up local system test runs likeBackupTest.phpand improves local developer velocity.Issue
Additional Spanner Emulator System Test Fix
This PR addresses the flaky
ConflictException(ALREADY_EXISTS) failures occurring in the Spanner emulator system tests, specifically intestPdmlandtestRunTransaction.The Problem:
When running against the emulator (or real backend), operations like
runTransactioncan occasionally fail with transient errors (such asABORTED). When this happens, the transaction closure is automatically retried. However, if the initial mutation was actually applied before the abort occurred, a subsequent retry usinginsertorinsertBatchwill fail with anALREADY_EXISTSerror.The Fix:
Changed the mutation methods from
insertandinsertBatchto their idempotent equivalentsinsertOrUpdateandinsertOrUpdateBatchin:PartitionedDmlTest.phpPgPartitionedDmlTest.phpTransactionTest.phpPgTransactionTest.phpThis ensures that retried transactions or reused test databases won't fail due to existing rows.