Skip to content

fix(fspy): unify shared memory on a sparse temp file across all platforms - #576

Merged
wan9chi merged 1 commit into
mainfrom
fspy-sparse-file-shm
Aug 10, 2026
Merged

fix(fspy): unify shared memory on a sparse temp file across all platforms#576
wan9chi merged 1 commit into
mainfrom
fspy-sparse-file-shm

Conversation

@wan9chi

@wan9chi wan9chi commented Jul 28, 2026

Copy link
Copy Markdown
Member

Motivation

Codex CLI's and Claude Code's default sandboxes deny the primitives fspy's Unix shared memory was built on. The macOS Seatbelt profile denies shm_open (#563), and both sandboxes block Unix domain sockets, which the Linux memfd broker depended on. Plain files in the temp directory work under both sandboxes; the IPC lock file lives there today.

This PR replaces all three platform backends with one file-backed implementation, mapped through memmap2 on every platform. The API has three types, each one platform concept:

  • ShmKeeper is the name. create returns it, it carries the identifier (the backing file's absolute path), and dropping it removes the file with remove_file.
  • ShmHandle is the opened file. create returns one, so the creator never looks its own file up by name, and open returns one to everybody else. map can be called more than once.
  • Mapping is the bytes. It keeps them alive until dropped and cannot affect the name.

Removal works on every platform because modern Windows deletes with POSIX semantics: the name goes away at once, existing handles keep working, and mapped views keep the data alive. CI probes on Windows Server confirmed both, with a live writable view and with an open share-delete handle. The docs reserve the right to fail the delete while a view is mapped, and Windows versions without POSIX delete do fail it, so the keeper falls back to reopening the file with FILE_FLAG_DELETE_ON_CLOSE and closing it. Unit tests pin the full removal semantics: name gone with a live mapping, name gone with an open handle, and the handle still mapping the same bytes afterwards.

Name removal is cleanup. The channel invalidates contents inside the shared bytes, so nothing depends on removal timing. Backing files sit directly in the system temp directory as vite-task-fspy-<uuid>.shm with mode 0o600; a shared subdirectory would belong to whichever user created it first and lock everyone else out. The identifier is resolved to an absolute path at creation, so a relative TMPDIR in the creating process cannot mislead an opener with a different working directory.

There is no broker, no global object name, no tokio requirement, and no hand-written mapping code: the Windows-specific parts shrink to the sparse-file FSCTL and the creation flags.

If the keeper's process is killed, the file stays behind: on Unix for the temp reaper, on Windows until a cleanup tool runs. It costs about as much disk as the run wrote into it.

Refs #563.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -3.38%  [-10.37% ..  +3.15%]  overhead   +62.98%
dynamic/access             change  -3.87%  [-47.75% ..  +4.98%]  overhead   +14.47%
dynamic/access-relative    change  -8.21%  [-39.54% ..  +3.61%]  overhead   +33.93%
static/launch              change  +0.68%  [ -6.79% ..  +9.33%]  overhead  +145.99%
static/access              change  +1.08%  [-11.92% .. +12.81%]  overhead +2174.33%
static/access-relative     change  -0.14%  [ -2.98% ..  +2.76%]  overhead +2450.83%

macos

dynamic/launch             change  +0.82%  [ -3.65% ..  +5.20%]  overhead  +241.10%
dynamic/access             change  +0.86%  [ -6.73% .. +10.42%]  overhead    +4.88%
dynamic/access-relative    change  +2.31%  [ -4.75% ..  +9.50%]  overhead  +248.77%

windows

dynamic/launch             change  -0.40%  [ -7.51% ..  +8.21%]  overhead   +27.96%
dynamic/access             change  +0.18%  [ -8.57% .. +17.71%]  overhead    +0.75%
dynamic/access-relative    change  +0.35%  [ -3.74% ..  +9.58%]  overhead    +2.89%

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f029ee6c9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/fspy_shm/src/file_backed.rs Outdated
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch 6 times, most recently from 5e527ad to f572f0d Compare July 30, 2026 02:14
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch from f572f0d to ca1932a Compare July 30, 2026 03:24
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread crates/fspy_shm/src/file_backed.rs Outdated
Comment thread CHANGELOG.md Outdated
…orms

Codex CLI's and Claude Code's default sandboxes deny the primitives fspy's
Unix shared memory was built on: the macOS Seatbelt profile denies
`shm_open` (`ipc-posix-shm-write-create`), and both sandboxes block
Unix-domain sockets, which the Linux memfd broker depended on. Plain files
in the temp directory are writable under both.

Replace all three backends with one file-backed implementation attached by
path: a sparse temporary file plus `memmap2`, with the file's absolute path
as the identifier. No broker, no tokio requirement, no global object names.

Lifetime semantics converge too: dropping the owner makes the name
disappear and later opens fail, while existing views stay usable — now on
Windows as well, where closing the delete-on-close handle applies the delete
disposition even while other processes hold views.

Refs #563.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the fspy-sparse-file-shm branch from 8622417 to 1c604c5 Compare August 10, 2026 06:59
@wan9chi
wan9chi merged commit 29fcbd6 into main Aug 10, 2026
20 checks passed
@wan9chi
wan9chi deleted the fspy-sparse-file-shm branch August 10, 2026 07:06
wan9chi added a commit that referenced this pull request Aug 10, 2026
## Motivation

Codex CLI's and Claude Code's default sandboxes block Unix domain
sockets, so `vp run` fails while setting up task communication before
any task code runs (#562). Named FIFOs are plain files, and both
sandboxes allow them.

This PR reapplies the FIFO transport from #565 with a rename, a review
pass, and new coverage:

- The crate is `pipe_socket`: socket-style server-client IPC,
implemented on named pipes rather than Unix domain sockets. It carries
no task-runner specifics, so it sits outside the `vite_*` prefix, like
`fspy` and `pty_terminal`. It exposes `Server::bind`, `Server::name`,
`Server::accept`, and `Client::connect`; FIFOs on Unix and named pipes
on Windows stay implementation details.
- `Client::connect` now fails instead of hanging when the server is
gone. The rendezvous open is nonblocking, so a missing reader turns into
a connection-refused error at once. While waiting for the server's ready
byte, the client watches the rendezvous write end and probes the server
every 100ms; a dead server turns into an error within one probe
interval. macOS needs the probe because its `poll` does not always
report FIFO events; Linux reports the death at once. The module
documents the handshake step by step.
- A new integration test pins the no-hang behavior for both cases:
server gone before the connection attempt, and server dying in the
middle of one.
- The Codex sandbox snapshot now records the end state this PR and #576
were built for: the task runs inside the sandbox, fspy traces its reads,
and editing the traced file causes a cache miss on the next run.
Regenerated against the real `codex` CLI on macOS. The Claude sandbox
snapshot is carried over from #565 unchanged; it needs an `srt` binary
to regenerate, and the recorded behavior does not depend on the
shared-memory backend that changed underneath it.

Closes #562.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant