Skip to content

refactor(cli): move the async leaves and the calibration child runner onto Effect - #554

Merged
Makisuo merged 3 commits into
mainfrom
refactor/cli-effect-async
Aug 20, 2026
Merged

refactor(cli): move the async leaves and the calibration child runner onto Effect#554
Makisuo merged 3 commits into
mainfrom
refactor/cli-effect-async

Conversation

@Makisuo

@Makisuo Makisuo commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

The CLI is already Effect at the command layer — Effect CLI v4, Context.Service for MapleConfig/Mode, Schema.TaggedError throughout — but raw async below it. Most of that raw code is straight-line filesystem work that happens to use await and is fine as-is. Three spots are not, and this converts those.

Scope is deliberately tiered. src/server/{checkpoints,archives,local-store-migrations} (~15k lines) and serve.ts stay raw; see Out of scope below.

Why these three

  • Nothing was interruptible. runCandidateChild was a hand-rolled new Promise with a settled flag, a setTimeout watchdog and a 500ms setInterval disk poller. Ctrl-C mid-calibration left the poller running and the spawned process group orphaned — nothing killed it.
  • Failures were silently discarded. credential-store.ts swallowed every error in a bare catch, and its callers used Effect.promise, which discards failures again. A broken keychain was indistinguishable from "not logged in".
  • A latent self-kill. const pgid = child.pid ?? 0, then process.kill(-pgid, "SIGKILL"). -0 is 0, and POSIX kill(0, sig) signals the caller's own process group — the CLI would have SIGKILLed itself.

What changed

Leaves (each had only Effect callers, so no bridging seams were introduced):

  • core/credential-store.tsChildProcessSpawner. MapleConfig captures the spawner in make, the way it already captures fs, so MapleConfigValues signatures stay R = never.
  • commands/auth.ts stdin → Stdio.stdin.
  • core/update.tsChildProcess for tar/xattr, FileSystem for the fs work.

Calibration child-process path (commands/archive.ts) — runCandidateChild, runBoundCalibrationMatrix, runCalibrationMatrix and the handler, converted as one unit. The watchdog and poller become a forked killer fiber; the group reap becomes a scope finalizer so it also runs on interruption and defects.

No new dependencies: BunServices.layer was already in MainLayer and provides ChildProcessSpawner, FileSystem, Stdio.

Reviewer notes

Three near-misses worth a look, since each would have been a silent regression:

  1. Terminal.readLine is the wrong primitive for piped stdin. It waits for a readline "line" event and never resolves on EOF, so printf tok | maple auth login --with-token would hang forever. Stdio.stdin terminates at EOF and splitLines flushes the trailing partial line. Verified against the real runtime for no-trailing-newline, extra-lines, CRLF and empty input.
  2. handle.exitCode fails with a PlatformError on signal death, and every watchdog kill is a signal death. That is collapsed to a null code, so a killed candidate lands in the same code !== 0 branch as before. Without it, one killed candidate would abort all six signals instead of eliminating a single matrix cell.
  3. mapFsError detected EACCES via a bare .code, which a PlatformError does not expose — the actionable "re-run the installer" message would have vanished for every converted fs call. It now also checks the PermissionDenied reason tag and the wrapped cause.

Behaviours deliberately preserved verbatim: fail-closed peak-RSS parsing, nonzero exit failing even when a metrics JSON line is present, the 1600-char diagnostic truncation, the fail-loud disk-poll read error (its catch lives inside the poll, so a read error can't silently kill the poller and downgrade fail-loud to fail-late), and gating completion on the pipes draining rather than on exit.

Two intentional deviations:

  • core/executor.ts was left alone. Its throw new Error sits behind WarehouseSqlClient.insert, a Promise<void> port owned by @maple/query-engine, and the executor already wraps it in Effect.tryPromise with mapWarehouseError. Converting it would change the shape without changing the behaviour.
  • The closing reconcile keeps its ArchiveError rather than being orDied into a defect, and a close failure now only decides the outcome when the matrix itself succeeded — so unlike the original finally, it no longer masks the matrix error.

A bug this turned up

Exercising the refactored credential path against a real keychain surfaced a pre-existing bug (present before this branch — the original Bun.spawn version reproduces it identically): on macOS, maple auth login recorded credentialStore: "keychain" and stored an empty password, so the token was lost. saveRemoteCredential skips the file fallback whenever the keychain write reports success, so the next command saw no token and the user looked logged out.

security add-generic-password -w with no argument does not read the secret from stdin as the old comment claimed. It prompts for it — which is what keeps the secret out of the process list — and then asks for it to be retyped. A single piped line fails that confirmation, and security stores an empty password and exits 0 anyway. The secret is now fed twice, and the write is verified by reading it back, so a partial write falls back to file storage instead of silently losing the token.

This is exactly the class of failure the old bare catch was hiding, which is why it is fixed here rather than filed separately.

Testing

typecheck, oxlint, oxfmt --check and bun test (493 pass) are green in apps/cli.

New coverage for code this branch converted that previously had none:

  • runCandidateChild — four tests (signal death, nonzero exit alongside a valid metrics JSON line, the group kill, output written immediately before exit). It was an unexported promise closure reachable only from the shell probes. The group-kill test is mutation-verified: swapping the reap for a child-only kill makes it fail.
  • extractTar and mapFsError. The mapFsError test is mutation-verified too — it fails against the old bare .code check, confirming a PlatformError really does hide EACCES.

Manual, against the real runtime:

  • Keychain write/read/delete round-trip on a throwaway origin (no automated test: it would either touch a developer's real login keychain or need a fake spawner whose partial stub test/ cannot typecheck).
  • maple auth login --with-token with a token piped without a trailing newline — the case Terminal.readLine would hang on — plus empty stdin, extra lines and CRLF.

Native probes, run locally against a binary compiled from this branch, all passing:

Probe Result
native-archive-calibrate-probe.sh PASS — 24 results, held-out validation, six-metric predicted-vs-observed within tolerance, manifest identity verified, no debris
native-archive-calibrate-crash-probe.sh PASS — SIGKILL at the sampling boundary reconciled, unrelated pin survived
native-checkpoint-smoke.sh PASS
native-archive-smoke.sh PASS

Out of scope

checkpoints.ts pins/locks/restore, durable-files.ts, serve.ts, and the archives/migrations bulk. Every conversion here obeys one direction — Effect wraps raw, raw never wraps Effect — and those modules are called from six still-promise-based callers, so converting them would mean either compatibility shims or a 15k-line diff. Splitting withMaintenanceLock / withRestoredCheckpoint into acquireRelease shapes is the natural next tier; it also unblocks runCalibrateSample.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

… onto Effect

The CLI is Effect at the command layer but raw async below it. Three of those
raw spots cost real correctness, so convert them and leave the rest alone.

Leaves, each with only Effect callers:

  * credential-store spawns `security`/`secret-tool` through ChildProcessSpawner
    instead of Bun.spawn behind a bare catch. That catch made a broken keychain
    indistinguishable from a machine that has none; the cause is now logged
    before degrading. MapleConfig captures the spawner in `make`, the way it
    already captures `fs`, so MapleConfigValues keeps R = never.

  * `maple auth login --with-token` reads stdin through Stdio rather than a
    hand-rolled Promise over stdin events. Deliberately NOT Terminal.readLine:
    that waits for a readline "line" event and never resolves at EOF, so
    `printf tok | maple auth login --with-token` would hang forever.

  * update.ts drives tar/xattr through ChildProcess and its filesystem work
    through FileSystem. mapFsError detected EACCES via a bare `.code`, which a
    PlatformError does not expose, so the actionable "re-run the installer"
    message would have silently disappeared — it now also reads the
    PermissionDenied reason tag and the wrapped cause.

Calibration child runner (archive.ts), converted as one unit so raw code never
wraps Effect:

  * The `settled` flag, the setTimeout watchdog and the 500ms setInterval disk
    poller become a forked killer fiber racing a sleep against a sleep-first
    poll loop. The poll keeps its fail-loud catch INSIDE the poll so a read
    error still kills the candidate rather than silently killing the poller.

  * The group reap moves into a scope finalizer, so it runs on interruption and
    defects too. Previously it only ran from inside a timer callback, and a
    Ctrl-C mid-candidate orphaned the Maple grandchild.

  * `pgid` could be 0, and POSIX kill(0, sig) signals the CALLER's own process
    group — the CLI would have SIGKILLed itself. Guarded.

  * Completion still gates on the pipes draining, not on exit: exitCode alone
    resolves on "exit", which Node emits before stdio is guaranteed to drain.
    exitCode also FAILS on signal death, and every watchdog kill is a signal
    death, so that is collapsed to a null code — otherwise one killed candidate
    would abort all six signals instead of eliminating one matrix cell.

  * The closing reconcile keeps its ArchiveError instead of being orDie'd, and
    no longer masks a matrix failure the way the original `finally` did.

runCandidateChild had no unit coverage at all (it was an unexported promise
closure reachable only from the shell probes); it is exported now with four
tests, including a group-kill test that fails if the reap is child-only.

checkpoints.ts pins/locks, durable-files, serve.ts and the archives/migrations
bulk stay raw: their callers are still promise-based, and converting them would
mean either shims or a 15k-line diff.
Testing the refactored credential path end-to-end turned up a pre-existing bug
that predates the Effect conversion: `maple auth login` on macOS recorded
`credentialStore: "keychain"` and stored an EMPTY password, so the token was
lost. saveRemoteCredential skips the file fallback whenever the keychain write
reports success, so the next command saw no token at all and the user looked
logged out.

`security add-generic-password -w` with no argument does not read the secret
from stdin as the old comment claimed. It PROMPTS for it — which is still what
keeps the secret out of the process list — and then asks for it to be RETYPED.
A single piped line fails that confirmation, and `security` stores an empty
password and exits 0 anyway. Feeding the secret twice answers the prompt, for
both create and update.

Neither helper reports a partial write through its exit status, so also read the
secret back and require an exact match before claiming the keychain owns it. A
mismatch now returns false and the caller falls back to file storage, which
works. That also covers a secret `security` echoes back hex-encoded (it does
this for non-ASCII data), where the keychain is genuinely not usable as-is.

Verified by round-tripping a realistic ASCII token against the real keychain on
a throwaway origin. Left untested automatically: a keychain test would either
touch a developer's real login keychain or need a fake spawner whose partial
stub `test/` cannot typecheck.

Also cover extractTar and mapFsError, both of which this branch converted with
no tests behind them. The mapFsError test is the useful one: it fails against
the old bare `.code` check, confirming that a PlatformError really does hide
EACCES and that the install-dir advice would otherwise have gone missing.
…osity

The candidate-child drain test passed on macOS and failed on Linux CI. The
diagnostic is `stderr\n stdout\n <time report>` and truncation keeps only the
first and last 800 characters, so the surviving tail is the time report — and
GNU `time -v` writes ~721 bytes where BSD `time -lp` writes a few lines. With a
200-line payload the assertion on the FINAL stdout line landed in the discarded
middle on Linux. Passing on macOS was luck, not evidence.

Split into two tests that each hold on both platforms:

  * drain: a payload small enough that the diagnostic never truncates on any
    platform, asserting the first AND last line survive. The last line is the
    one that matters — a stdout cut short at exit would still produce the head.

  * truncation: the oversized payload, asserting the truncation marker, a length
    bound, and that the head survives. All three hold whatever the report says.

Verified by running the suite under Ubuntu as a non-root user (494 pass): the
previous version reproduces the CI failure there, this one does not.
@Makisuo
Makisuo merged commit 738d37b into main Aug 20, 2026
32 checks passed
@Makisuo
Makisuo deleted the refactor/cli-effect-async branch August 20, 2026 22:54
@Makisuo
Makisuo deployed to pr-preview August 20, 2026 22:54 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit 67e9479 · View workflow run

Makisuo added a commit that referenced this pull request Aug 20, 2026
This branch carried its own copy of the CLI Effect refactor, which landed
on main separately as #554. Every conflict was that duplicate, and main's
copy is the later one in each case, so the conflicted files are resolved
to main verbatim:

- credential-store.ts — main writes the secret twice, because `security
  -w` asks the caller to retype it and a single piped line silently stores
  an empty password while exiting 0, then reads the credential back to
  prove the keychain actually owns it. This branch predates both.
- update.ts — main's `__testables` also exports `extractTar` and
  `mapFsError`.
- archive-candidate-child.test.ts — main splits drain-to-EOF and
  truncation into separate cases with a payload small enough that
  `/usr/bin/time`'s output cannot make the assertion platform dependent.

What remains of this branch after the merge is one line: project.yml's
MapleSwift pin, 0.2.1 -> 0.3.1. Main already resolved maple-swift to 0.3.1
in Package.resolved, so the pin was the half left behind.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant