Skip to content

test(ci): split platform/fixture tests into an informational CODEHUB_PLATFORM lane#192

Merged
theagenticguy merged 1 commit into
mainfrom
chore/platform-test-lane
Jun 4, 2026
Merged

test(ci): split platform/fixture tests into an informational CODEHUB_PLATFORM lane#192
theagenticguy merged 1 commit into
mainfrom
chore/platform-test-lane

Conversation

@theagenticguy
Copy link
Copy Markdown
Owner

What

Audit P1 follow-up to the package collapse (#189): isolate the vendor/runtime/fixture tests into a separate, informational CI lane so environment flakiness can't mask or block the logic-regression signal on the release-gating test job.

This is the structural answer to the Windows chmod/path/file-url churn that dominated #189 — those tests are legitimate (they caught real bugs), but they fail for environment reasons unrelated to OpenCodeHub's logic, so they shouldn't sit on the required check.

Mechanism: env-var { skip } gate

A per-suite/per-case { skip: platformSkip } keyed on CODEHUB_PLATFORM, generalizing the existing CODEHUB_INTEGRATION idiom in sagemaker-embedder.integration.test.ts.

Chosen over the two alternatives because both break on the Node 20 floor:

  • --test-skip-pattern is added: v22.1.0 — absent on Node 20.
  • node --test glob expansion is post-20, so a *.platform.test.ts file-glob split would match nothing on Node 20 (vacuous pass).

The env gate uses only core-stable test-runner APIs, behaves identically on Node 20/22/24 and on Windows cmd + POSIX shells, and needs zero package.json / invocation changes — both lanes run the identical pnpm -r test; only CODEHUB_PLATFORM differs.

What's tagged (verified per-case, not file-level)

Interleaved logic cases stay in the required lane:

File Gated Stays as logic
mcp/run-smoke.test.ts all 25 ToolResult-shape smokes
cli/setup-embeddings.test.ts weight-download suite
ingestion/wasm-grammar-resolution.test.ts 15 vendored-.wasm disk-presence cases php→php_only / cobol routing
cli/scip-downloader.test.ts 6 download/extract+chmod mechanics pin-mismatch, UnsupportedPlatform, placeholder-hash, SHA-mismatch, dotnet-SDK, install ordering
cli/embedder-downloader.test.ts 5 download mechanics Sha256MismatchError contract, retry/backoff policy

CI

New test-platform job (3 OS × Node 22) runs the same suite with CODEHUB_PLATFORM=1 + continue-on-error: true, not a required check. A Windows fixture quirk turns it amber without blocking a logic-correct PR or the release-please PR. The required test job runs the identical command with the gate off and carries the logic-regression signal alone.

Verified

  • Logic lane (default): full suite exit 0 with platform cases skipped (cli 11 skipped, mcp 25, ingestion's 15-case wasm suite).
  • Platform lane (CODEHUB_PLATFORM=1): full counts run (cli 303, mcp 218, ingestion 577), exit 0.
  • lint + typecheck + banned-strings 0.

Also fixes a pre-existing useTemplate lint nit in init.test.ts touched by the same sweep.

🤖 Generated with Claude Code

…PLATFORM lane

Audit P1 follow-up to the package collapse: isolate the vendor/runtime/
fixture tests so environment flakiness (the Windows chmod/path/file-url
failures that dominated the collapse PR) can't mask or block logic-
regression signal on the release-gating `test` job.

Mechanism — env-var `{ skip }` gate (generalizes the existing
CODEHUB_INTEGRATION idiom in sagemaker-embedder.integration.test.ts).
Chosen over a file-rename/glob split and over `--test-skip-pattern`
because BOTH of those break on the Node 20 floor: `--test-skip-pattern`
is `added: v22.1.0`, and `node --test` glob expansion is post-20 (the
repo's double-quoted globs would match nothing on 20). The env gate uses
only core-stable test-runner APIs and behaves identically on 20/22/24 and
on Windows cmd + POSIX shells. Zero package.json / invocation changes —
both lanes run the identical `pnpm -r test`; only CODEHUB_PLATFORM differs.

Tagged (verified per-case, not file-level — interleaved logic cases stay
in the required lane):
- mcp/run-smoke.test.ts — all 25 ToolResult-shape smokes (whole file).
- cli/setup-embeddings.test.ts — the weight-download suite (whole file).
- ingestion/wasm-grammar-resolution.test.ts — the 15 vendored-.wasm
  disk-presence cases (the php->php_only/cobol routing logic stays).
- cli/scip-downloader.test.ts — the 6 download/extract+chmod mechanic
  cases (pin-mismatch, UnsupportedPlatform, placeholder-hash, SHA-mismatch,
  dotnet-SDK, install ordering stay as logic).
- cli/embedder-downloader.test.ts — the 5 download-mechanic cases
  (Sha256MismatchError contract + retry/backoff policy stay as logic).

CI: new `test-platform` job (3 OS × Node 22) runs the same suite with
CODEHUB_PLATFORM=1 + `continue-on-error: true`, NOT a required check — a
Windows fixture quirk turns it amber without blocking. The required
`test` job runs the identical command with the gate off and carries the
logic-regression signal alone. Also fixed a pre-existing useTemplate lint
nit in init.test.ts touched by the same sweep.

Verified: logic lane (default) full suite exit 0 with platform cases
skipped (cli 11 skipped, mcp 25, ingestion's 15-case wasm suite); platform
lane (CODEHUB_PLATFORM=1) runs full counts (cli 303, mcp 218, ingestion
577), exit 0; lint + typecheck + banned-strings 0.
@theagenticguy theagenticguy merged commit 71dbfe0 into main Jun 4, 2026
42 checks passed
@theagenticguy theagenticguy deleted the chore/platform-test-lane branch June 4, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant