Skip to content

Bound host-metadata discovery on Config init to the configured timeouts#1483

Closed
tejaskochar-db wants to merge 1 commit into
mainfrom
tejaskochar-db/fix-host-metadata-resolution-timeout
Closed

Bound host-metadata discovery on Config init to the configured timeouts#1483
tejaskochar-db wants to merge 1 commit into
mainfrom
tejaskochar-db/fix-host-metadata-resolution-timeout

Conversation

@tejaskochar-db

Copy link
Copy Markdown
Contributor

Summary

Build the host-metadata discovery client on Config initialization from the configured retry_timeout_seconds / http_timeout_seconds, so an unreachable host no longer blocks Config() / WorkspaceClient() for ~5 minutes.

Why

On initialization, Config resolves missing fields (account_id, workspace_id, discovery_url, cloud, token_audience) from the host's /.well-known/databricks-config discovery endpoint, via _resolve_host_metadataget_host_metadata. This probe is best-effort: any failure is caught, logged, and the explicitly-provided configuration is used instead.

The problem is the client used for the probe. get_host_metadata's default client is a no-argument _BaseClient(), so the probe ignores retry_timeout_seconds and http_timeout_seconds and always uses the default 300-second retry budget. When the host is unreachable — behind a proxy/firewall that drops connections, or a workspace that is down — initialization blocks for the full ~5 minutes before the probe gives up and falls back, even when host, token, and auth_type were all supplied explicitly and there was nothing to discover. Setting the timeouts on Config does not help today, because the probe never receives them.

This is the same class of problem as #1046 (and its fix #1085), which addresses the oidc_endpoints probe; this PR covers the separate _resolve_host_metadataget_host_metadata probe.

What changed

Interface changes

None.

Behavioral changes

_resolve_host_metadata now builds its _BaseClient from the configured retry_timeout_seconds / http_timeout_seconds, matching how ApiClient constructs its client. Callers that set shorter timeouts now get a fast, bounded failure against an unreachable host instead of a fixed ~5-minute block. The probe remains best-effort with the same fallback to explicit configuration, behavior is unchanged for reachable hosts, and the default timeout is unchanged (still 300s when not configured).

Internal changes

None beyond the above.

How is this tested?

  • New unit test (test_resolve_host_metadata_uses_configured_timeouts) asserts the discovery client carries the configured timeouts; the existing _resolve_host_metadata suite still passes.
  • Manual: against an unreachable host (https://192.0.2.1, RFC 5737 TEST-NET), Config(host=..., token=..., auth_type="pat", retry_timeout_seconds=2, http_timeout_seconds=1) returns in ~2.6s and logs the fallback warning, instead of blocking on the default 300s retry budget.

Config.__init__ resolves config fields from the host's
/.well-known/databricks-config endpoint. The discovery client was built with
no arguments, so it ignored retry_timeout_seconds / http_timeout_seconds and
always used the default 300s retry budget. An unreachable host therefore
blocked Config() initialization for ~5 minutes before falling back to the
explicit configuration, even when the user had set shorter timeouts.

Build the discovery client from the configured timeouts, matching how
ApiClient constructs its _BaseClient. The fetch stays best-effort with a
fallback to explicit config on failure.

Co-authored-by: Isaac
@github-actions

Copy link
Copy Markdown

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-py

Inputs:

  • PR number: 1483
  • Commit SHA: c32c41824c692699dfff8de91ecc79907c43b81a

Checks will be approved automatically on success.

@tejaskochar-db tejaskochar-db deleted the tejaskochar-db/fix-host-metadata-resolution-timeout branch June 25, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant