fix(ops): verify restart PID and served build SHA#81
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91711091ae
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Refs ADA-76
wizzoapp
Bot
force-pushed
the
codex/p0-restart-delivery
branch
from
9171109 to
89cd20f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89cd20f995
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if [[ "$rc" -ne 0 || -z "$sha" ]]; then | ||
| last_reason="environment descriptor did not include serverBuildSha" |
There was a problem hiding this comment.
When the descriptor request fails or returns non-JSON for the whole timeout, fetch_descriptor_sha exits nonzero but this branch reports the same CHECK restart FAIL environment descriptor did not include serverBuildSha text that t3-daily-restart treats as a legacy unstamped artifact. On a same-SHA restart where HTTP is still coming up or returning 5xx, this causes an unnecessary stop/rebuild retry instead of surfacing a restart/HTTP failure; distinguish the missing-serverBuildSha case from fetch/parse failures before allowing the legacy rebuild path.
Useful? React with 👍 / 👎.
1 participant
Summary
t3-daily-restartalways receives service/origin/smoke/probe settings explicitlyT3CODE_BUILD_SHAand expose it in the environment descriptorverify-restartand fail/rollback unless MainPID changes and descriptorserverBuildShamatches the target commitRefs ADA-76. Related narrower symptom: ADA-71.
Evidence
KillMode=control-group+TimeoutStopSec=2s: restart changed PID (3923300 -> 3923345), proving systemd itself is not silently no-oping.T3CODE_BUILD_SHA=dddd... vp run --filter t3 build:bundle;apps/server/dist/bin.mjscontains the injected 40-char SHA.Tests
./scripts/ops/daily-restart/verify-restart.test.sh./scripts/ops/daily-restart/t3-nightly-cycle.test.sh./scripts/ops/daily-restart/t3-daily-restart.test.sh(102 passed)vp test run src/environment/ServerEnvironment.test.ts --reporter verbosevp run --filter t3 typecheckvp check(0 errors; existing mobile unused-symbol warnings)