Skip to content

fix(Spanner): prevent unnecessary GCE metadata probing during client …#9327

Merged
cy-yun merged 3 commits into
mainfrom
fix-gce-metadata-probe-delay
Jul 20, 2026
Merged

fix(Spanner): prevent unnecessary GCE metadata probing during client …#9327
cy-yun merged 3 commits into
mainfrom
fix-gce-metadata-probe-delay

Conversation

@cy-yun

@cy-yun cy-yun commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Fix: Prevent unnecessary GCE metadata probing during SpannerClient initialization**

When a SpannerClient is instantiated, it automatically calls configureMetrics(). Previously, this method unconditionally resolved the Google Compute Engine (GCE) location by calling $this->getLocation().

In non-GCE environments (such as local development environments, emulators, or external CI pipelines), this triggers an HTTP probe to the GCE metadata server (169.254.169.254) that ultimately times out. This timeout introduces a completely unnecessary ~1.5 second delay to every single client instantiation.

Because the Spanner system tests repeatedly instantiate the SpannerClient, this delay heavily impacted the overall test suite execution time.

Changes Made

  • Moved the $location = $this->getLocation(); resolution below the early return for when built-in metrics are disabled (which is the default behavior).
  • Added testSpannerClientInstantiatesWithoutDelayWhenMetricsDisabled() to assert that the SpannerClient instantiates in under 1.0 second when metrics are disabled, ensuring no future regressions re-introduce the metadata probe delay.

Impact

By completely bypassing the metadata server probe for the default use case, this instantly recovers 1.5 seconds per SpannerClient instantiation in non-GCP environments. This dramatically speeds up local system test runs like BackupTest.php and improves local developer velocity.

Issue


Additional Spanner Emulator System Test Fix

This PR addresses the flaky ConflictException (ALREADY_EXISTS) failures occurring in the Spanner emulator system tests, specifically in testPdml and testRunTransaction.

The Problem:
When running against the emulator (or real backend), operations like runTransaction can occasionally fail with transient errors (such as ABORTED). When this happens, the transaction closure is automatically retried. However, if the initial mutation was actually applied before the abort occurred, a subsequent retry using insert or insertBatch will fail with an ALREADY_EXISTS error.

The Fix:
Changed the mutation methods from insert and insertBatch to their idempotent equivalents insertOrUpdate and insertOrUpdateBatch in:

  • PartitionedDmlTest.php
  • PgPartitionedDmlTest.php
  • TransactionTest.php
  • PgTransactionTest.php

This ensures that retried transactions or reused test databases won't fail due to existing rows.

@Hectorhammett

Copy link
Copy Markdown
Collaborator

This looks GTM but for some reason the Emulator System Tests are failing, I don't think this is related though 🤨

@cy-yun
cy-yun force-pushed the fix-gce-metadata-probe-delay branch from b04d25e to 0dcf2c8 Compare July 8, 2026 19:53
@cy-yun

cy-yun commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

This looks GTM but for some reason the Emulator System Tests are failing, I don't think this is related though 🤨

You're right. It's not related. It's a flakey test that I'm addressing in #9329

The Spanner Emulator occasionally drops connections and triggers our automatic retry logic after it has already committed a row, causing the retry attempt to crash. I'll be swapping those tests to use idempotent insertOrUpdate() calls to fix the flakiness.

Changes insert to insertOrUpdate and insertBatch to insertOrUpdateBatch in Transaction and PartitionedDML system tests. This prevents ConflictException (ALREADY_EXISTS) when tests are retried or a transaction closure is retried after a transient emulator error (e.g. ABORTED) where the initial mutation was still applied.
@bshaffer
bshaffer enabled auto-merge (squash) July 20, 2026 21:56
@cy-yun
cy-yun disabled auto-merge July 20, 2026 22:02
@cy-yun
cy-yun enabled auto-merge (squash) July 20, 2026 22:03
@cy-yun
cy-yun merged commit 920e1bc into main Jul 20, 2026
41 checks passed
@cy-yun
cy-yun deleted the fix-gce-metadata-probe-delay branch July 20, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: spanner Issues related to the Spanner API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants