Skip to content

fix(desktop): give the SSH readiness probe room for high-RTT links - #7815

Open
Exotic209093 wants to merge 2 commits into
pingdotgg:mainfrom
Exotic209093:fix/ssh-readiness-probe-timeout
Open

fix(desktop): give the SSH readiness probe room for high-RTT links#7815
Exotic209093 wants to merge 2 commits into
pingdotgg:mainfrom
Exotic209093:fix/ssh-readiness-probe-timeout

Conversation

@Exotic209093

@Exotic209093 Exotic209093 commented Aug 21, 2026

Copy link
Copy Markdown

Summary

Fixes #7733. T3 Code Desktop connects to remote environments over SSH tunnels, then polls a backend-readiness HTTP endpoint through the tunnel before considering the connection up. Each individual probe attempt had a hardcoded 1000ms deadline. On a high-RTT link (satellite, in-flight wifi), the round trip through the tunnel alone can exceed that, so every single attempt failed on the per-request timeout before a response could ever arrive — even though the tunnel and remote server were both healthy and answering. The app retried forever inside the existing 20s overall connect budget and never succeeded.

Fix

Raised SSH_READY_PROBE_TIMEOUT_MS in packages/ssh/src/tunnel.ts from 1s to 8s. This value also feeds the remote-side reuse/launch scripts' own readiness probe (T3_READY_PROBE_TIMEOUT_MS), but that probe only ever checks 127.0.0.1 on the remote host itself — it never crosses the slow link, so the larger bound there is harmless: it can only make an already-slow remote boot wait a little longer before giving up, never make a healthy one fail faster. Left the overall 20s connect budget untouched — this only gives each individual attempt real room to succeed on a slow link, so connecting gets slower, not impossible, matching the issue's expected behavior.

Test plan

  • Added a regression test in packages/ssh/src/tunnel.test.ts using a mocked HTTP client that responds successfully after a simulated 1.5s delay (TestClock-driven) — longer than the old 1000ms default. Confirmed it fails (times out) before the fix and passes after.
  • vp test run packages/ssh — 27 passed, including the existing "bounds each HTTP readiness probe so retries cannot hang on one request" test (which explicitly overrides probeTimeoutMs per-call and is unaffected by the default change).
  • vp run --filter @t3tools/ssh typecheck — clean.
  • vp lint on changed files — clean.

Note

Low Risk
Timeout-only change to SSH readiness probing; no auth, data, or protocol changes. Worst case is slightly slower failure on a hung remote.

Overview
SSH tunnel HTTP readiness no longer fails every attempt on high-RTT links. Each probe used a 1s deadline, so a healthy remote behind satellite or in-flight wifi could never answer in time and the 20s overall connect budget was wasted on retries.

Raises SSH_READY_PROBE_TIMEOUT_MS from 1s to 8s (overall 20s budget unchanged). The same constant still feeds remote loopback probes, which only wait longer on a slow boot. Adds a TestClock regression that a 1.5s successful response now succeeds.

Reviewed by Cursor Bugbot for commit 3d707b0. Configure here.

Note

Raise SSH tunnel readiness probe timeout to 8000ms for high-RTT links

The SSH tunnel readiness check was using the default per-probe timeout, which could fail on high-RTT links. Introduces SSH_TUNNEL_READY_PROBE_TIMEOUT_MS = 8000 in tunnel.ts and passes it as probeTimeoutMs to waitForHttpReady in startSshTunnel. Adds a test in tunnel.test.ts that verifies success when a probe response takes ~1500ms.

  • Risk: each probe now waits up to 8s instead of the previous default; on genuinely unreachable hosts, readiness detection may take longer before failing.
📊 Macroscope summarized 8041aab. 1 file reviewed, 1 issue evaluated, 1 issue filtered, 0 comments posted

🗂️ Filtered Issues

packages/ssh/src/tunnel.ts — 0 comments posted, 1 evaluated, 1 filtered
  • line 1120: The 8-second override is applied only to the initial readiness check. A subsequent ensureEnvironment checks the same established tunnel with waitForHttpReady({ timeoutMs: 2_000 }), which retains the 1-second default probe timeout. On the high-RTT links this change targets (for example, a healthy response after 1.5 seconds), that reuse check still fails, closes the healthy tunnel/managed remote server, and recreates the environment instead of reusing it. Pass the tunnel-specific probe timeout to the existing-entry readiness check as well. [ Out of scope ]

Each readiness probe crosses the real SSH tunnel (desktop -> remote),
so its 1000ms deadline left no room for links with RTT above roughly
that, e.g. satellite or in-flight wifi. Every attempt failed on the
per-request timeout before a response could arrive, and the connection
never succeeded even though the tunnel and remote server were healthy.

Raise the probe deadline to 8s so a slow link only makes connecting
slower, not impossible, while leaving the overall 20s connect budget
alone.

Fixes pingdotgg#7733
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9ef7d091-1635-4072-9bef-1e07b6d5896f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:XS 0-9 changed lines (additions + deletions). labels Aug 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d707b0d6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/ssh/src/tunnel.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3d707b0. Configure here.

Comment thread packages/ssh/src/tunnel.ts Outdated
@macroscopeapp

macroscopeapp Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 8041aab

Macroscope's review found this PR approvable — Straightforward timeout configuration fix that raises the SSH tunnel readiness probe timeout from 1s to 8s for high-RTT connections. The author properly addressed prior review feedback by keeping the remote loopback probe at 1s while only increasing the desktop-side tunnel probe timeout, and includes regression test coverage.

You can add or adjust custom eligibility rules. Learn more.

…s deadline

Macroscope caught a real regression in the first pass: raising the
shared SSH_READY_PROBE_TIMEOUT_MS to 8s also fed the remote-side
reuse/launch scripts' own loopback probe. That script's deadline loop
only checks its overall timeout *between* attempts, so one in-flight
probe bounded by 8s could blow through the reuse check's 2s budget by
several seconds — a real regression the previous version didn't
account for.

Split it: SSH_READY_PROBE_TIMEOUT_MS stays 1s for the remote script
(always loopback, no RTT to account for, and must stay under
REMOTE_REUSE_READY_TIMEOUT_MS), and a new
SSH_TUNNEL_READY_PROBE_TIMEOUT_MS (8s) applies only to the desktop-side
probe that actually crosses the SSH link at initial connect. That path
uses Effect's own timeoutOption, which genuinely interrupts an
in-flight request at its outer deadline, so it can't overshoot a short
timeoutMs elsewhere the way the remote script's loop can.
@Exotic209093

Copy link
Copy Markdown
Author

Good catch — the shared constant also fed the remote-side reuse/launch scripts' own probe loop, which (unlike the Effect-based path) doesn't interrupt an in-flight request when its own deadline passes, so one slow attempt bounded by the new 8s could blow through the 2s reuse-check budget. Split it into two constants: SSH_READY_PROBE_TIMEOUT_MS stays 1s and is only used for the remote loopback probe now; a new SSH_TUNNEL_READY_PROBE_TIMEOUT_MS (8s) applies only to the desktop-side probe that actually crosses the SSH link at initial connect.

@github-actions github-actions Bot added size:S 10-29 changed lines (additions + deletions). and removed size:XS 0-9 changed lines (additions + deletions). labels Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S 10-29 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: SSH remote environment never connects when link RTT exceeds ~1s, eg on in-flight wifi (readiness probe deadline is 1000ms)

1 participant