fix(thrift): retry transient transport failures on GetOperationStatus/CloseOperation polls

[diegofanesi]

Summary

Fixes #1642.

Long-running queries on the Thrift client path (UseThriftClient=1) could fail with Query has been timed out due to inactivity under sustained concurrency. A single transient transport-level failure (stale pooled connection, TCP reset, load-balancer idle drop) on a GetOperationStatus poll abandoned the still-running server operation — the driver surfaced a TTransportException and stopped polling, so the server kept running the query until its inactivity window elapsed and reaped it. The same failure class on CloseOperation / CancelOperation could leak completed operations until the server reaped them too.

Root cause

DatabricksHttpRetryHandler.retryRequest only retries HTTP status-code errors (429/503) that arrive wrapped in a DatabricksRetryHandlerException. A raw IOException (stale pooled connection, connection reset) maps to status code 0 via getErrorCodeFromException, which isStatusCodeRetryable rejects — so it is never retried. The Thrift polling loop in DatabricksThriftAccessor has no retry of its own either, so one transient poll failure permanently abandons an otherwise-healthy server operation.

Change

Adds a bounded, jittered exponential-backoff retry (withTransportRetry) in DatabricksThriftAccessor and routes the idempotent Thrift RPCs through it:

• GetOperationStatus — both the main execution polling loop and the metadata polling loop (also covers the Thrift heartbeat, which shares the same accessor method)
• CloseOperation and CancelOperation

Each RPC invocation builds a fresh transport, so a retry naturally leases a new pooled connection while the broken one is discarded. This lets a still-running server operation be re-polled — or a completed one be re-closed / cancelled — instead of being abandoned after a single connection blip.

Details:

• Bounded to 5 retries with full-jitter backoff (1s → 16s cap), interrupt-aware (a thread interrupt during a backoff sleep restores the interrupt flag and aborts the retry loop).
• The original TTransportException is rethrown once retries are exhausted, so existing caller-side failure handling (buildTransportFailureException, etc.) is fully preserved.
• Only transport-level failures are retried; non-transport TExceptions propagate on the first attempt.
• Statement submission (ExecuteStatement) is deliberately excluded from the retry path — re-sending it could double-execute the query. Only read-only / idempotent RPCs retry.

Scope

• Thrift client path only (UseThriftClient=1). All changes live in DatabricksThriftAccessor.
• The SEA client path is unchanged — it already retries this class of IOException via the SDK's idempotent-request retry strategy (GET /api/2.0/sql/statements/* is treated as idempotent, and an IOException maps to a retriable synthetic 523).

Files changed

• src/main/java/com/databricks/jdbc/dbclient/impl/thrift/DatabricksThriftAccessor.java — retry helper + call-site wiring
• NEXT_CHANGELOG.md — Fixed entry

Testing

• Verified the retry helper's generics, lambda capture, exception flow, and backoff math via a standalone compile + run.
• Note: the full Maven build was not run locally (no network access to Maven Central in the authoring environment); please rely on CI for the full build + spotless check. If spotless flags formatting, it is cosmetic and auto-fixable with mvn spotless:apply.

This pull request and its description were written by Isaac.

[diegofanesi]

tests completed. ready to merge on my end.

I run 3 parallel tests on three large warehouses for a total of almost 10000 queries launched in parallel on a 10 TB dataset. this lasted 24h and I had only 1 query failure related to a different issue. Normally I would have seen multiple queries failing under such conditions.

I can conclude that the stability has been improved significantly with this patch.