feat(dev): the local Compute and bucket emulator daemons#160
Conversation
…or daemon layer The daemon registry was pinned to ~/.prisma-composer/emulators/ with no seam for tests, which would pollute the real home directory. ensureDaemon and stopDaemon accept an optional registryRoot override whose only caller is tests; production code never passes it. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New lowering-layer package (spec local-dev § 2), domain prisma-cloud, layer lowering, plane control. Registered in architecture.config.json and tsconfig.depcruise.json ahead of any source so pnpm lint:deps can verify the plane placement before building on it. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
daemon.ts (spec local-dev § 2): ensureDaemon/stopDaemon, the machine-scoped registry at <registryRoot>/<name>.json (defaulting to ~/.prisma-composer/emulators/, overridable for tests only), version-skew detection against the running daemon own /health response, and the pinned start/failure sequence. client.ts: typed loopback clients (computeClient/bucketsClient) resolving a running daemon from the registry, surfacing the pinned not-running error for an absent or dead daemon. segments.ts and state-file.ts are small shared internals: the <app>/<id>/<name> path-segment hygiene both daemon programs enforce, and temp-then-rename JSON state writes serialized behind one in-process queue. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compute-main.ts (spec local-dev § 2, subpath /compute-main): the loopback JSON admin API for service registration and deployment, spawning `bun bootstrap.js` with exactly the requests env. Restart rules (running restarts iff hash/env changed; stopped/held/never-ran always start, clearing held), crash supervision (1s*2^n backoff capped at 30s, held after 5 consecutive sub-30s exits), per-service log files with [emulator]-prefixed supervision events, and follow streaming. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
buckets-main.ts (spec local-dev § 2, subpath /buckets-main): the S3 wire over @internal/s3-protocols createS3Handler + fsStore, resolver fed from /_pcdev/ registrations (physical bucket names <app>--<name>). Admin API for bucket + credential registration and app teardown, SigV4 verified against any accepted credential, and the pinned 501 for multipart. Plain node:http throughout, per the no-new-runtime-dependencies behavior contract. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers the plan S3 Proves list: ensure -> health -> version-skew restart; deployment lifecycle (spawn, restart-on-hash-change-only, noop on an identical redeploy, a stopped service always restarts, backoff, held after 5 fast crashes, a redeploy clears held, log follow); multi-app isolation for both compute and buckets (same service/bucket names, distinct ports/dirs, no collisions); bucket admin plus a real SigV4 round-trip via @aws-sdk/client-s3; the daemon surviving its parent process exit. Every test uses a temp registryRoot and temp state/data dirs, and stops every daemon it started in afterEach. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ensureDaemon spawns the daemon detached + unref()d so it survives the calling process exiting -- correct for the happy path, but on a health-poll timeout it left that same child running forever with nobody supervising it. Found via a stray-process audit after the squatted-port test: the spawned compute-main took over the port only after the test process moved on, leaking a process every run. Kill the child before throwing the pinned failure error. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spec 2s Start step still hardcodes ~/.prisma-composer/emulators for the spawned daemons state-dir and log path, left over from before registryRoot was introduced on the Registry bullet two paragraphs above. Taken literally this defeats registryRoots whole purpose (test isolation from the real home directory). Implemented with registryRoot governing all three paths together, matching its stated default and this slices own explicit test-isolation requirement; recorded here for the spec text to be tidied to match. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ensure kills its child Resolves the Open Questions entry recorded during S3: the Start step still hardcoded ~/.prisma-composer paths from before the registryRoot override existed, defeating its test-isolation purpose taken literally. Also pins the daemon-leak fix S3 surfaced: a health-poll timeout must kill the detached child before throwing, or a failed ensure leaves an unsupervised daemon running forever. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The design branch resolved the registryRoot-scope question S3 recorded (commit 9d6adf9, cherry-picked): registryRoot governs the registry file, state dir, and log path together, and a failed ensure kills its child before throwing -- exactly what S3 implemented. Reverts the Open Questions section to match the design branch byte-for-byte. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…le registry entry Three cheap fixes worth taking since these daemons are long-lived: - compute-main.ts: the running+unchanged deployment PUT (a true no-op, 204 with nothing restarted) mutated svc.address/svc.port in memory without persisting, so a daemon restart before any later persist would silently revert them. Move the mutation past the no-op return instead of persisting on every no-op PUT. - compute-main.ts: DELETE /apps/<app> dropped each services RuntimeInfo without closing its open log fd, leaking one fd per deleted service for the life of the daemon process. ServiceLog gains close(), called before the RuntimeInfo is discarded. - daemon.ts: a failed ensure now also removes the registry entry after killing the just-spawned child, so nothing is left pointing at a pid that is already dead. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 39 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
WalkthroughThis PR introduces the 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
commit: |
…stics, stale-reference fixes Three of the four CodeRabbit findings on #158, all valid. Bucket-emulator SigV4 now authenticates against only the credentials owned by the target bucket's app (derived from the <app>-- prefix) — cross-app access was possible with a known bucket name, violating the stated multi-tenant isolation; credential registration records the owning app and a foreign re-registration 409s. The pinned prisma dev error masks connection-URL credentials before embedding captured output (the no-value-logging contract applies to diagnostics). The ADR index one-liner and ADR-0041 Related section still described the pre-pivot model (dev.state, process table, CLI supervision) — rewritten to the emulator model. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…app hole CodeRabbit found the real isolation gap: SigV4 verified against ANY accepted credential regardless of which app registered it, so a known bucket name plus a valid signature from a DIFFERENT apps credential could read/write it. Pinned in spec commit b1d7324 (cherry-picked). - PUT /_pcdev/apps/<app>/credentials now records the owning app and 409s naming neither secret when the accessKeyId is already owned by a different app; the same app rotating its own secret still succeeds. - The S3 wire derives the target buckets owning app from its registration (the authoritative source for the <app>--<name> split) and verifies SigV4 against only that apps credentials. An unknown bucket or a valid signature from another apps credential both fall through the same 403 path as a bad signature -- identical status, identical body, nothing about the buckets existence leaks. Writing the requested restart round-trip test surfaced a real adjacent bug: both daemons SIGTERM-handlers called process.exit() without waiting for an in-flight state write to land, so a daemon killed right after a mutation could lose it. StateFile gains flush(), awaited in both compute-main.ts and buckets-main.ts before exit. Tests: cross-app rejection with a genuinely valid signature compared byte-for-byte against a bad-signature response, same-app access (already covered by the existing isolation test plus a same-app multi-credential test), the 409 conflict plus same-app secret rotation, and the credential owner + secret round-tripping across a real daemon restart. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…not the physical name The segment rule permits internal hyphen runs, so <app>--<name> is not reversible by string-splitting; the registration record stores the owner and is authoritative. Matches the S3 implementation. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Per-app credential scoping is now enforced (4725393), closing the multi-tenant hole CodeRabbit found on #158's spec: credential registration records the owning app (cross-app re-registration of an Also fixed while testing: both daemons' SIGTERM/SIGINT handlers exited without awaiting an in-flight state write, which could silently drop a mutation made just before a kill — Gates re-run twice: 39/39 package tests, empty process-leak audit both runs, cast ratchet delta 0. |
… port stability, publish-safe entries Port stability across prisma dev stop/start is verified on the primary path; the --db-port fallback retires unneeded. Build-only extensions (every node kind:build, no providers/application/provisions/container) have nothing to emulate and are exempt from dev capability — one shared isBuildOnlyExtension predicate serves mergedDevProviders and the CLI check. The @internal/dev-emulators runtime resolution is not publish-safe; S5 gains the pinned fix (caller-supplied entry paths + public daemon-entry subpaths). Two layer-order accommodations in the local providers are pinned as protocol matches with a recorded follow-up for a shared home. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…logy wording Second-round CodeRabbit findings on #158, both valid. Two concurrent prisma-composer dev processes could both observe no healthy daemon, double-spawn, and collide on a port allocation — the ensure critical section is now serialized across processes with an atomic mkdir lock (stale-holder recovery, bounded wait, re-read after acquire, allocation inside the lock). The ADR index over-claimed one emulator per node kind: Compute/buckets are machine-global daemons, Postgres is one prisma dev instance per Database under the ORM CLI's own manager. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rectory lock CodeRabbits second finding on #158: two concurrent prisma-composer dev processes could both observe no healthy daemon, double-spawn, and collide on a port allocation -- the daemon registry had no inter-process claim protocol. Pinned in spec commit c83b169 (cherry-picked, along with two prerequisite commits that had landed on the design branch since the last sync: 1ecba54 and 90f4383, both docs-only and already matching this implementation). daemon.ts: an atomic mkdir lock (<registryRoot>/.lock-<name>) now wraps ensureDaemons observe-spawn-persist section per daemon name. An optimistic unlocked read still short-circuits the common already-healthy case; anything else acquires the lock, RE-READS the registry (the previous holder may have already finished), then proceeds. EEXIST with a dead holder pid is stale and removed for an immediate retry; a live or still-being-written holder is polled every 250ms up to a 10s budget before the pinned timeout error. Port allocation now happens inside the lock. Release is unconditional via finally, including the health-poll-timeout throw path. Tests: two concurrent ensureDaemon calls from separate spawned OS processes (the mutex is inter-process, so two promises in one process would not exercise it) resolve to one daemon and agree on its URL; a stale lock (dead holder pid, a real spawned-then-reaped process) is broken and ensure proceeds; a held lock with a live scratch holder process times out with the pinned error message. Along the way: `pgrep -a` means "include ancestors" on macOS, not "show full args" as on GNU/Linux -- the process-count assertion needed `-fl`, and a short poll instead of one snapshot to tolerate transient verification lag under a heavily loaded machine, not because the lock itself is unreliable (confirmed clean over many repeated runs). Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Carries the second-round CodeRabbit resolutions: the inter-process daemon-ensure lock protocol (implemented on #160) and the precise emulator-topology wording, plus the S4-findings pins. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…p/ps Root cause (CI run 29989129545): verification-tooling platform difference, not a lock failure. macOS pgrep -a means "include ancestors" (already worked around locally); GNU pgrep on the Linux runner reads -a as "show full args" instead, so the same -fl invocation renders differently there and the process-count poll never matched, expiring the 3s waitFor. The other two lock tests, which never touch pgrep or ps, pass on both platforms -- daemon.ts's mkdir-atomicity is not in question. Removed OS process inspection from the test entirely per directive, replaced with portable evidence: - Each ensure-and-print fixture process now also reads back the registry entry it observed and prints its pid alongside the URL. - The test asserts both processes printed the same pid and URL, that pid is alive (process.kill(pid, 0)), and the registry file exists. - compute-main.ts and buckets-main.ts now log a single unambiguous "listening on 127.0.0.1:<port>" line to their own stdio log once actually bound -- the stdio log already existed for supervision; this just gives it one greppable, count-once marker line. The test asserts the log contains exactly one such line, which a lock that failed to serialize the two racing calls would violate (a second spawn logs its own listening line even after being killed). Confirmed via direct reproduction (15/15 clean runs of the exact race in a tight loop, plus repeated bun:test runs) that daemon.ts itself correctly serializes concurrent callers. A further reproduction uncovered an unrelated, expected edge case worth recording: if the first spawn attempt genuinely fails (e.g. a squatted port), the process that was waiting on the lock correctly gets its own turn and may succeed with a different pid -- intended lock behavior, not a bug, and not something the shipped tests exercise (they use fresh, uncontended registryRoots). Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Pins the fix for the parallel-test flake mechanism: a FRESH daemon allocation (no persisted port) retries up to 5 next-free candidates on a bind failure while holding the ensure lock; a previously persisted port never moves and keeps the existing manual-recovery failure. Also carries the teardown slug-glob wording fix (S4-scope, code-wise). Taken wholesale from the design branch tip; verified byte-identical. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bind failure Pins spec commit 4ce7160 (§ 2 step 5, rewritten): root cause of the parallel-suite flake S5 confirmed pre-existing -- isolated test registryRoots all start their port search at the same MIN_PORT, so two daemons (or a daemon and any unrelated process already on that port) race for the same OS port and the loser times out. daemon.ts: a FRESH allocation (no persisted port, hence no frozen endpoint anywhere yet) now retries up to 5 distinct port candidates inside the same ensure-lock critical section when the spawned child exits with a bind error before ever reporting healthy. A previously persisted port never moves -- deploy state may already reference it as a frozen endpoint -- and keeps the existing single-attempt failure. awaitHealthy() replaces pollUntilHealthy(): it watches the child's own exit event so a dead child is detected and retried immediately rather than exhausting the full 10s budget; only a live-but-not-yet-healthy child still gets the full budget. It also now verifies the health response's OWN version, not just that something answered -- if the child failed to bind, whatever foreign process already held the port may itself answer /health successfully, and treating that as "our daemon is healthy" would persist the wrong pid. Tests: a scratch listener occupies the first candidate a fresh registryRoot always picks, and the daemon comes up on the next free port, persisting it and logging exactly one listening line; a persisted port occupied by a scratch listener with a dead pid recorded still fails with the pinned error, no port move. Diagnosing this surfaced real external interference from concurrent agents sharing this sandbox (another worktree's real, long-running compute-main daemon on the default port, and separately-owned processes transiently on compute service ports) -- not a flaw in this code, but it made several existing daemon-focused tests fragile in the same way this fix targets. Added a shared ensureFreshDaemon() test helper (steers a fresh registryRoot's first allocation to a port this process has verified is actually free, by pre-seeding fake registry entries for the machine-contended range below it) and applied it everywhere a test starts a brand-new daemon, plus switched one existing test off a hardcoded port 4300. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…service contention, verify stop truthfulness skipContendedServicePorts and a dual-interface isPortFree (checking both 127.0.0.1 and 0.0.0.0 bind scopes) steer compute-deployment.test.ts and compute-multi-app.test.ts past real service ports already held by other processes on the machine, the same class of contention the daemon-port retry feature already handles one layer down. Also empirically verified the POST /apps/<app>/stop truthfulness concern: a live reproduction against compute-main.ts's actual stop path confirmed killChild already awaits the full SIGTERM -> 5s grace -> SIGKILL sequence, and the HTTP handler awaits it, before the listing ever reports `stopped` with the pid cleared - no code change was needed there. Added the requested regression test with a fixture that ignores SIGTERM: the listing only shows `stopped` once the pid is genuinely dead. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Branch updated (f6ffe17): the Linux CI failure is fixed — root-caused to BSD-vs-GNU |
…equential The dual-scope isPortFree probed 127.0.0.1 and 0.0.0.0 concurrently via Promise.all. Binding two sockets to the same port at the same time makes the probes collide with EACH OTHER, not with any real occupant - the kernel can refuse the overlapping bind even when nothing else was ever listening there. That self-collision apparently doesn't happen on macOS (both probes coexisted fine when reproduced locally) but does on Linux CI, which is why a fresh CI runner - genuinely free on almost every port - still failed to find one in [4300, 4500). The old code also treated any bind error as "taken", silently. Replaced with attemptBind, which resolves 'taken' only for EADDRINUSE/EACCES and rethrows anything else, so a real surprise errno fails loudly instead of being misread. checkPort now binds loopback then wildcard sequentially (no concurrent same-port binds at all), and findFreePort's exhaustion error reports the observed errno counts for one-run diagnosis. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
wmadden
left a comment
There was a problem hiding this comment.
Seems reasonable, but were there really no ecosystem alternatives we could have leveraged instead of hand-rolling everything?
…endency razor pins) Takes .drive/projects/local-dev/spec.md wholesale from design-branch commit 904d23a, which pins two things this round implements: the POST /apps/<app>/start session-resume endpoint (70c0ef1) and the dependency-razor decision retiring the no-new-deps contract in favor of proper-lockfile and get-port for the daemon lock and free-port probing (904d23a itself). Verified empty diff against 904d23a exactly. The design branch has since moved to d407c1f ("the #162 rework wave — operator review reversals"), which is out of scope for this round and not chased here. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oint Alchemy correctly skips reconcile when nothing diffed, so a warm dev restart never fires a deployment PUT and nothing else restarts services a previous session stopped. This adds the symmetric inverse of /stop: starts every service that has a stored deployment spec and isn't already running, reusing the deployment PUT's exact start rules (same spawn path, same supervision reset, clears `held` the same way an explicit deployment does). A service with no stored deployment is skipped without error; a service already running is a no-op. client.ts gains a matching startApp(app) on ComputeClient. Also swaps compute-main.ts's own service-port allocation (smallestUnusedServicePort) from pure bookkeeping to get-port for the actual is-it-free check — the same dependency-razor decision this round applies to daemon.ts's port allocation (see the next commit), landing here too since it's the same function family in the same file as the /start work. Tests: stop then start brings children back up on the same ports with new pids; a never-deployed service is skipped; a held service resumes; start on an already-running app changes nothing. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t for free-port checks Dependency razor (spec § 2, design commit 904d23a): commodity infrastructure with latent edge cases uses maintained libraries. Our own hand-rolled versions of both already produced real cross-platform bugs this project hit directly — a BSD/GNU pgrep divergence (round 5) and a Linux-only port-probe self-collision (the errno-driven fix two commits back on this branch). Retires the earlier no-new-deps contract for exactly this class of code. daemon.ts: the concurrent-ensure protocol's mkdir+pid-file lock is replaced by proper-lockfile locking a stable per-daemon target file under registryRoot. Its own staleness/compromise semantics are adopted wholesale — no pid liveness checking on top. Our wrapper keeps only what spec § 2 pins: a ~10s bounded wait (proper-lockfile's retries option), the pinned timeout error on ELOCKED exhaustion, re-reading the registry after acquiring, and releasing in a finally on every path. Port allocation (smallestUnused) now calls get-port for the actual bind check; persistence and the >= 4300 range/exclusion policy stay ours. ensureDaemon's signature gains an explicit entry parameter — ensureDaemon(name, entry, opts) — instead of resolving @internal/dev-emulators/<name>-main internally. This mirrors what the S5 branch (bot/claude/local-dev-s5-dev-command) already needs for its own reason (a published dist can't import.meta.resolve a private workspace package), threading it now so that cascade needs no manual reconciliation inside the retry loop. Test helpers: isPortFree/findFreePort's hand-rolled bind probing is deleted; every scratch-port need in the test suite now calls get-port directly. The two lock-behavior tests in daemon.test.ts are rewritten to proper-lockfile's actual on-disk mechanism (an aged .lock directory for staleness; a real second process genuinely holding the same lock via proper-lockfile itself for the live-holder timeout) rather than asserting against the old hand-rolled pid-file format — their assertions (one daemon exists, a stale lock is broken, a live holder times out with the pinned message) are unchanged. New dependencies of @internal/dev-emulators (private, workspace-only): proper-lockfile, get-port, @types/proper-lockfile (dev). tsdown's skipNodeModulesBundle keeps both external in dist/*.mjs, confirmed by inspection of the built output — this package gains no new *transitive* runtime deps for its current (private, in-repo) consumers. Packaging note for S5: when the public package re-emits these daemon entry points for npm consumers (spec § 2's publish note), that public package will need proper-lockfile and get-port as its OWN dependencies too, since they are not inlined — flagging now, no action taken here since no public consumer exists yet on this branch. CI read for the next merge (not fixed here — the cascade replaces S5's daemon.ts with this one): diffed this branch's post-swap daemon.ts against bot/claude/local-dev-s5-dev-command's daemon.ts. The retry loop (exit-detection via awaitHealthy, port-exclusion via usedPorts, error classification via exitedBeforeHealthy) is otherwise identical between the two branches; the only structural difference is the entry-parameter threading this commit adopts. The Linux-only CI failure on #164 (run 30007836065, "a bind failure on a fresh allocation retries the next free port", 529ms fail, "Expected length: 1, Received length: 0" on the listening-line count) is therefore not explained by any logic divergence in the retry path itself between the two branches. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…surface pin) Takes .drive/projects/local-dev/spec.md wholesale from design-branch commit 62d1a6a, which revises the Postgres providers section (operator review of #162, S7 pulled forward): the CLI shell-out is replaced by a third emulator daemon, postgres-main, hosting @prisma/dev's startPrismaDevServer() programmatically. Verified empty diff against 62d1a6a exactly. The design branch has since moved to 5c6eb71 ("the dev seam is a lazy reference — no dev code in production bundles"), out of scope for this round and not chased here. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ma/dev programmatic
Adds postgres-main.ts (subpath /postgres-main), the local Postgres
emulator: hosts @prisma/dev's startPrismaDevServer() in-process, one
named persistent server per Database resource, several servers in the
one daemon process. Same daemon layer/registry/lock/port machinery as
compute-main.ts and buckets-main.ts (daemon.ts gains the 'postgres'
name, health at /health).
@prisma/dev is imported dynamically from a caller-resolved path (each
PUT carries prismaDevModulePath) — the daemon itself has no @prisma/dev
dependency of its own, so the app stays in charge of its own Prisma
version. Resolution failure surfaces the pinned "add \"prisma\" to your
app's devDependencies" message, naming the given path without leaking
anything else from the underlying error (only its first line, and
credential-masked).
Admin surface exactly per the pinned spec:
- GET /health
- PUT /apps/<app>/databases/<id> { prismaDevModulePath } -> { url },
idempotent. Instance name pcdev-<app>-<id> (each half slugged,
combined name capped at 63 chars). databasePort persists in this
daemon's own state and follows spec § 2 step 5's fresh-allocation
retry pattern (a fresh allocation tries up to 5 candidates on a bind
conflict; a persisted port never moves).
- GET /apps/<app>/databases -> listing.
- DELETE /apps/<app> -> closes the app's live servers and deletes their
persisted PGlite data via @prisma/dev's ./internal/state export
(resolved by reading that package's own declared exports map, walked
up from the caller-given path — never by guessing at its dist/ file
layout).
Every server also needs its own HTTP/shadow-database/streams ports
(@prisma/dev's other ServerOptions, each defaulting to a fixed value) —
allocated fresh and kept unique among servers live in this one process,
tracked in memory only since nothing outside this daemon depends on
their exact values (only databasePort, and the connection string it
produces, are part of the contract).
client.ts gains postgresClient() (ensureDatabase/listDatabases/
deleteApp), exported from the package index alongside the other two
clients.
Port-conflict detection: @prisma/dev exports a PortNotAvailableError
class, but `instanceof` against it does not match across the dynamic-
import boundary (confirmed empirically — the daemon's own module graph
and the dynamically-imported one produce two separate class
identities for the same logical error). Duck-typed instead: the
documented `port: number` property, matched against the exact
candidate just tried.
Tests (src/__tests__/postgres.test.ts): ensure/list/delete lifecycle
with a real `pg` connection; ensure is idempotent; a daemon restart
(stopDaemon + re-ensure) drops the in-process server but PUT restores
it on the SAME databasePort with a real row (written via `pg` before
the restart) still readable after; two apps' same-id databases are
genuinely separate PGlite instances; a bogus prismaDevModulePath
surfaces the pinned 500 naming the given path; a bind conflict on a
fresh databasePort allocation retries the next free port. Every test
uses a distinctly-prefixed app name and calls DELETE at the end — this
machine has real, unrelated @prisma/dev usage under the same
machine-global app-data directory (`~/Library/Application
Support/prisma-dev-nodejs`), confirmed untouched by the leak audit.
New devDependencies of @internal/dev-emulators (private, workspace-
only, tests only — the daemon itself imports @prisma/dev only via a
caller-resolved runtime path, never as a static import): @prisma/dev,
pg, @types/pg. Not present anywhere else in the workspace; added here
per the coordinator's instruction since no example already depended on
it.
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.ts`:
- Around line 168-185: Update the test named “surfaces 500 naming the resolution
failure without leaking unrelated paths” to assert that the rejected error
exposes HTTP status 500 and that its message does not contain unrelated path
details. Preserve the existing assertions for the resolution hint and bogusPath,
using the returned error object and its status property where available.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 4e0143f8-72f0-4ad2-86b5-66e97a7b18f0
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (30)
.drive/projects/local-dev/plan.md.drive/projects/local-dev/spec.mdarchitecture.config.jsondocs/design/10-domains/local-dev.mddocs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.mddocs/design/90-decisions/README.mdpackages/1-prisma-cloud/0-lowering/dev-emulators/package.jsonpackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/buckets.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/daemon.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/fixtures/ensure-and-print.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/fixtures/fake-versioned-daemon.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/fixtures/spawn-and-exit.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/buckets-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/client.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/daemon.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/buckets-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/compute-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/postgres-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/postgres-main.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/segments.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/state-file.tspackages/1-prisma-cloud/0-lowering/dev-emulators/tsconfig.jsonpackages/1-prisma-cloud/0-lowering/dev-emulators/tsdown.config.tstsconfig.depcruise.json
…ce interpolation Two alerts on #160, both in this round's code: js/polynomial-redos (postgres-main.ts): the instance-name slug used `/[^a-z0-9]+/g` followed by `/^-+|-+$/g` for the trim — an alternation of two `+`-quantified anchored patterns, the shape CodeQL's check flags regardless of whether this particular instance is provably linear. Rewritten as a per-character replace (no `+`), a bounded `{2,}` collapse, and plain index-walking for the trim — no regex at all for the boundary case. Moved into a new instance-name.ts (was inline in postgres-main.ts): that file calls its own main() at module load, so testing the slug logic by importing it directly would have run the daemon's own argv parsing and crashed outside a real subprocess. Added a test feeding a 10,000-hyphen input and asserting both the correct slug and sub-100ms handling. Checked for the same regex shape elsewhere: no other daemon (compute-main.ts, buckets-main.ts, daemon.ts, segments.ts) has any regex-based slug/trim logic. @internal/local-target, mentioned as a second known instance, does not exist in this worktree — presumably lives on a branch not yet merged here; nothing to fix on this branch. js/bad-code-sanitization (helpers.ts): servingBootstrap's generated child-process source embedded `body` twice — once properly through JSON.stringify for the Response body, once raw inside a single-quoted console.log string. Both interpolations now go through the same JSON.stringify'd literal, embedded as a JS expression rather than concatenated raw text, so no unsanitized value reaches the generated source. Gates: bun test 56/56 (one unrelated one-off ConnectionRefused flake on the first run, traced to a leftover dangling process from an earlier debugging cycle in this session, not from this change — clean on rerun with no dangling processes), lint:casts delta 0. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Re: ecosystem alternatives — this was a deliberate call, recorded in the spec ( On the remaining open item: CodeRabbit's single actionable comment (postgres.test.ts assertion gap) is tracked as its own inline thread and will be addressed there. |
… derivation Delta review finding A (#160): LocalDatabaseProvider re-derived the pcdev-<app>-<database-id> instance name with its own slug, which dropped the leading/trailing-dash trim postgres-main's own derivation (now dev-emulators' instance-name.ts, extracted in the CodeQL linear-regex fix) applies. An app name or database id ending in a hyphen — a shape the wire protocol's own segment rule already allows through — produced a DIFFERENT concatenated name here than the one the daemon actually created the server under (a doubled dash at the boundary), so LocalConnectionProvider's listing-based lookup threw noRecordedInstanceError even though the server existed. postgres.ts drops its own slug/instanceName and imports instanceNameFor directly from @internal/dev-emulators instead — one implementation, so this can no longer drift. instance-name.ts's slug/instanceNameFor are exported through dev-emulators' main entry (a pure function module, no daemon main() to run on import — verified importing it is side-effect free). Adds a cross-boundary test against the real postgres-main daemon: ensure through LocalDatabaseProvider, Connection-resolve through LocalConnectionProvider, for an app name and database id each ending in a hyphen — the exact pathological pair that threw noRecordedInstanceError before this fix. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, never interpolated into source
The previous JSON.stringify fix did not clear the js/bad-code-sanitization
alerts at helpers.ts:45 (two separate alert instances, same line). Kills
the alert class structurally instead of trying to sanitize the
interpolation: SERVING_BOOTSTRAP is now a fixed constant with NO dynamic
value in it at all — it reads process.env['FIXTURE_BODY'] at runtime,
the same way it already read process.env['PORT']. servingBootstrapEnv(body)
returns the { FIXTURE_BODY } entry to merge into the child's env at spawn
time.
Every call site that built a fixture from servingBootstrap(body) now
uses the fixed SERVING_BOOTSTRAP source plus servingBootstrapEnv(body)
folded into its putDeployment env — compute-deployment.test.ts and
compute-multi-app.test.ts. No template literal in helpers.ts builds
code from a variable anymore.
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n a bogus prismaDevModulePath The test only checked that SOME error was thrown and that its message contained the resolution hint and the given bogus path — it never asserted the 500 status, and never checked that nothing ELSE leaked. Writing the "no other path" assertion surfaced a real leak: a dynamic import() failure's own message routinely names a SECOND path (bun's "Cannot find module '<target>' from '<importer>'"), and the daemon's prior error text embedded that whole message verbatim — leaking its own internal file location, not anything the caller supplied. Fixed at the source rather than just asserting around it: importPrismaDev's resolution-failure branch no longer embeds the underlying import() error text at all — it names only the pinned message plus the caller-given prismaDevModulePath. (The startup-failure branch, once the module resolves fine, is unchanged and still surfaces its underlying text verbatim, credential-masked, per spec § 2 — that's a different, correctly-scoped case.) Test now asserts: the message contains "(500)" explicitly, and every multi-segment path-shaped substring in the message is the one path the client itself supplied. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@coderabbitai resolve (All three review threads are implemented and carry Done replies: the code-sanitization alerts are fixed structurally — fixture source is now a fixed constant reading FIXTURE_BODY from the environment, zero dynamic interpolation — and the bogus-module-path test asserts both the 500 status and that no filesystem path beyond the caller-supplied one appears in the body, which also surfaced and fixed a real path leak in the daemon's error text. Thread resolution is delegated to this command because the bot token lacks the repo scope for the resolve mutation.) |
… derivation Delta review finding A (#160): LocalDatabaseProvider re-derived the pcdev-<app>-<database-id> instance name with its own slug, which dropped the leading/trailing-dash trim postgres-main's own derivation (now dev-emulators' instance-name.ts, extracted in the CodeQL linear-regex fix) applies. An app name or database id ending in a hyphen — a shape the wire protocol's own segment rule already allows through — produced a DIFFERENT concatenated name here than the one the daemon actually created the server under (a doubled dash at the boundary), so LocalConnectionProvider's listing-based lookup threw noRecordedInstanceError even though the server existed. postgres.ts drops its own slug/instanceName and imports instanceNameFor directly from @internal/dev-emulators instead — one implementation, so this can no longer drift. instance-name.ts's slug/instanceNameFor are exported through dev-emulators' main entry (a pure function module, no daemon main() to run on import — verified importing it is side-effect free). Adds a cross-boundary test against the real postgres-main daemon: ensure through LocalDatabaseProvider, Connection-resolve through LocalConnectionProvider, for an app name and database id each ending in a hyphen — the exact pathological pair that threw noRecordedInstanceError before this fix. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts (1)
88-98: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winMake
waitForHttpabort before exhaustingtimeoutMs.The deadline is only inspected after
fetch()rejects, so a server that accepted the TCP connection but never sends a response can leave one poller stuck. Pass an abort signal based on the remaining timeout into eachfetch()so stale in-flight requests are canceled on retry or final timeout.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts` around lines 88 - 98, Update waitForHttp so each fetch attempt receives an AbortSignal derived from the remaining deadline, canceling in-flight requests when the timeout expires. Recalculate remaining time before each attempt, preserve retries while time remains, and throw the final timeout-related error once the deadline is reached.packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.ts (1)
20-26: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winDo not swallow daemon cleanup failures.
catch(() => undefined)lets the suite pass while a daemon remains alive or its state cleanup fails, undermining the process-cleanup coverage and potentially contaminating later tests. Only ignore a narrowly identified “already stopped” condition; rethrow all other shutdown errors.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.ts` around lines 20 - 26, Update the afterEach cleanup around stopDaemon for the postgres daemon so it only ignores a specifically identified “already stopped” condition; rethrow all other shutdown or state-cleanup errors. Preserve registryRoot usage and ensure cleanup failures cause the test suite to fail.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/1-prisma-cloud/0-lowering/dev-emulators/src/instance-name.ts`:
- Around line 22-35: Update instanceNameFor to guarantee distinct canonical
app/id inputs produce distinct names despite slug collisions and the
63-character limit. Preserve the existing readable prefix while appending a
deterministic short hash derived from the original app and id, reserving space
for it before truncation; add injection tests covering separator collisions and
long inputs.
In `@packages/1-prisma-cloud/0-lowering/dev-emulators/src/postgres-main.ts`:
- Around line 163-168: Update importPrismaDev so its dynamic-import catch
distinguishes a genuinely missing target module from failures during module
evaluation or runtime wrappers. Only emit the existing NOT_INSTALLED_MESSAGE for
an error clearly indicating the requested prismaDevModulePath cannot be
resolved; otherwise preserve and rethrow the original error, including its
message and cause.
---
Outside diff comments:
In `@packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts`:
- Around line 88-98: Update waitForHttp so each fetch attempt receives an
AbortSignal derived from the remaining deadline, canceling in-flight requests
when the timeout expires. Recalculate remaining time before each attempt,
preserve retries while time remains, and throw the final timeout-related error
once the deadline is reached.
In
`@packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.ts`:
- Around line 20-26: Update the afterEach cleanup around stopDaemon for the
postgres daemon so it only ignores a specifically identified “already stopped”
condition; rethrow all other shutdown or state-cleanup errors. Preserve
registryRoot usage and ensure cleanup failures cause the test suite to fail.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e0836b26-60d7-466b-997e-5ae7d918e0a7
📒 Files selected for processing (6)
packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/postgres.test.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/instance-name.tspackages/1-prisma-cloud/0-lowering/dev-emulators/src/postgres-main.ts
|
✅ Action performedComments resolved and changes approved. |
Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The decision: local infrastructure runs as emulator daemons — long-lived, shared across apps — and this PR builds them.
Context in one breath: we're building
prisma-composer dev(ADR-0041, in the base branch), which runs a whole Composer app locally with no cloud credentials. This is the second layer of that stack of PRs, on top of #159 (the S3 protocol package). Stacked PR: the diff shown here is only this layer.The model
A Composer app declares infrastructure of three kinds: compute services, Postgres databases, and S3 buckets. Locally, each kind is backed by an emulator — the firebase/supabase model: a daemon that outlives any one dev session, holds your data, and makes every restart warm. Postgres needs nothing built here, because the ORM's
prisma devalready is exactly this. This PR builds the other two:helduntil the next deployment), and serves per-service logs with follow streaming.Both daemons are deliberately ignorant of the framework. They receive fully prepared inputs and run them. What they add is what a platform adds: stable ports (bindings freeze endpoint URLs in deploy state, so a moving port would break real config — stability is correctness, not comfort), restart-on-change, supervision, logs.
A small shared layer handles daemon lifecycle: a registry under
~/.prisma-composer/emulators/, readiness polling, and version-skew replacement (a daemon left running from an older package version is detected via its health endpoint and swapped). Because twoprisma-composer devprocesses can race to start the same daemon, the start path is serialized across processes with an atomic directory lock; the loser re-reads the registry and adopts the winner's daemon. A daemon that never becomes healthy on a freshly allocated port retries the next free port a bounded number of times; a port that was ever handed out never moves.Tested
36→45 tests driving the daemons' real HTTP APIs: full deployment lifecycle including restart-only-on-change and the crash/held policy, log following, multi-app isolation (same service and bucket names under two apps, no collisions, no cross-app credential access), SigV4 round-trip with a real AWS SDK client, version-skew replacement, the inter-process lock raced by two real OS processes, and daemons surviving their parent process exiting. A stop is only reported once the child is genuinely dead — proven with a fixture that ignores SIGTERM. Process-leak audit after every run. Workspace gates green, including Linux CI.
Alternatives considered
🤖 Generated with Claude Code