feat(ocap-jsonrpc-vat): line-delimited JSON-RPC over a Unix-style socket - #1009
feat(ocap-jsonrpc-vat): line-delimited JSON-RPC over a Unix-style socket#1009FUDCo wants to merge 37 commits into
Conversation
`runQueueLengthCache` uses a negative value to mean "unknown, re-read from the DB", but enqueueRun/dequeueRun adjusted it arithmetically without materializing it first. An enqueue while the cache was -1 (its value at daemon startup) produced 0 for a queue that actually held an item, and since 0 isn't negative it was never re-read: the run loop then saw an empty queue, went to sleep, and stranded the queued messages forever, with no error and no log. Also wake the run loop on any non-empty queue rather than only on the empty->1 transition, so a drifted count cannot silently lose the wakeup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds registerAnonymousKernelObject/releaseAnonymousKernelObject: a kref is allocated and entered in the by-kref routing table, but deliberately not in the service-name index, so the object has no name in the global service namespace and cannot be requested via a cluster config's `services` list. Authority comes from holding the reference. Needed for IOListener.accept(), where each accepted connection is a per-session object that should be reachable only by reference. Returned krefs are handed to kslot() so a kernel service method can return one; krefOf has no allocation path of its own. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…channels Splits the point of contact from the connection, BSD-style. An IOListener is what a cluster config's `io` entry now creates; its accept() yields one IOChannel per peer, each wrapped in its own exo and hosted as an anonymous kernel object, so the vat receives a Presence per connection. Sessions are isolated because they are distinct objects: holding one connection conveys no way to reach another, and `direction` is enforced per connection. IOManager tracks accepted connections per subcluster and releases them when the subcluster (or the listener) goes away. accept() resolves null once the listener is closed, so an accept loop can terminate rather than hang. **BREAKING:** Kernel's `ioChannelFactory` option becomes `ioListenerFactory`, and `IOChannelFactory` is replaced by `IOListener`/`IOListenerFactory`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces makeSocketIOChannel with makeSocketIOListener. The server hands each connection to accept() as its own IOChannel whose buffer, decoder, line queue, and reader queue are all local to that connection, so any number of peers can be served at once. Connections that arrive before accept() is called are queued rather than dropped. Gone with the single-client design: currentSocket, pendingSessionEnd, the merged lineQueue, and the socket.destroy() that rejected every second connection. Session boundaries need no latch now — one channel serves one peer, so the end of the socket simply is the end of the channel. **BREAKING:** makeIOChannelFactory becomes makeIOListenerFactory; makeSocketIOChannel becomes makeSocketIOListener. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…eers The io-vat's `repl` endowment is now an IOListener, so it accepts connections and addresses them by index, letting a test drive several peers independently. The integration test drops its hand-rolled duplicate channel in favour of the real makeIOListenerFactory, and adds a case covering two concurrent peers end to end through a real kernel — neither reading the other's data nor receiving the other's writes. That case was unrepresentable before: the second connection was destroyed on arrival. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…nSchema
Adds an `InterfaceJsonSchema` variant — `{ type: 'interface', description?,
methods }` — describing an object whose methods can be invoked, so a method
that hands back an object reference can declare the returned object's API
inline and a client need not make a second round-trip to discover it. The
`methods` field is recursive, so a returned interface can itself return
interfaces.
The schema describes an *interface*. Whether the reference to that object is
unforgeable is a property of the reference plumbing, not of the description,
so the same schema serves either case.
service-discovery-types converts the new variant to a `RemotableSpec` via
`interfaceJsonSchemaToRemotableSpec`, which means `remotable` is no longer
among the kinds `JsonSchema` cannot express.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The interface case validates that the value is a non-null object and nothing more — the declared `methods` describe the object for the caller rather than a shape to enforce here, since whether the object honours them is only discoverable by invoking it. Covers both halves: any object passes regardless of its methods, and every non-object is rejected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A general-purpose building block: a vat that serves line-delimited JSON-RPC 2.0 on an IOListener endowment, so local non-vat processes can reach kernel objects without shell-execing the CLI per call. Two methods. `redeemURL(url)` redeems an OCAP URL through the kernel's `ocapURLRedemptionService` and returns a name for the resulting reference. `send(target, method, args)` invokes `E(target)[method](...args)`, expanding names in `args` to live references and substituting any remotable in the result for a name. Names are `@@j<n>` sigil strings scoped to one connection. That scoping is load-bearing rather than incidental: the client is outside the ocap world, so the names it holds are plain forgeable strings, and confining them to a connection is what stops one client naming another's references. A forged name simply misses that client's own table. Each connection therefore gets its own bridge, and the accept loop serves connections concurrently without one client's traffic blocking another's. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
…fetimes Three findings from review: - Closing a listener dropped its sockets but left every accepted connection's kref pinned, since release only ran from a connection's own `close()`. The listener service now tracks what it handed out and releases the outstanding ones when it closes. - A connection's `close()` signalled EOF and only then flushed the receive buffer, so a trailing partial line could still be handed to a later `read()` after EOF had been reported. Closing now discards buffered data first; a peer-initiated end still flushes, since that data arrived before the peer went away. - `releaseAnonymousKernelObject` now deletes the kernel object once nothing references it, rather than leaving it to `collectGarbage`, which skips kernel-owned objects (per review; a no-op at the current refcount baseline, correct once #1006 changes that). Peer disconnect still does not release on its own, and that is deliberate: the holder's c-list still names the kref, so releasing there would make a later call on the dropped reference reach `invokeKernelService`, find nothing registered, and throw — taking down the run loop. That is worse than a leak bounded by the listener's lifetime. Documented at the call site, pending #1006. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three review findings, all cases where a client could be left with either no reply or a reply that is neither success nor error: - A void method returned `undefined`, which `JSON.stringify` drops, so the response carried neither `result` nor `error`. Normalized to `null`. Only `undefined` is substituted, so `0`, `''`, and `false` still report as themselves. - An unparseable request line was logged and dropped with no reply, so a client awaiting an answer on this request/reply socket waited forever. It now gets `PARSE_ERROR` with a null id, the id being unknowable from a line that would not parse. - A method may return a passable with no JSON form — a `bigint`, say — which `substituteRemotables` passes through untouched and which then throws in `JSON.stringify`. That was treated as a write failure and closed the connection. Encoding is now separate from writing, and an unencodable result yields an `INTERNAL_ERROR` reply instead. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ener Follow-up to review on the previous commit: setting `ended` inside `close()` made `handleEnd` return early and skip `onClosed`, so a channel closed by its holder stayed registered with the listener — a long-lived listener would accumulate every session it ever served. The flush-or-discard decision now lives in `handleEnd` and is keyed on `closed`, so both paths reach `onClosed` exactly once while a trailing partial line is still flushed for a peer-initiated end and discarded for a holder close. `makeConnectionChannel` is exported so this is testable directly; the package's public surface is unchanged, since `io/index.ts` does not re-export it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The project was still called `llm-mediator-vat`, the package's name before it was renamed, so its tests were mislabelled in monorepo output and in `--project` filters. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per review: `rollbackCrank` also invalidates the length cache, and a rollback is normally followed straight away by enqueueing an error or termination message — which is precisely the sequence that trips the bug. That path is more likely in practice than the startup one the entry originally described. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…el ends Node can still emit 'data' after `socket.destroy()`, and `handleData` checked neither flag. A late chunk therefore refilled the queue that `close()` had just cleared, and since `read()` drains the queue before consulting the flags, it would hand that line out after EOF had been reported. Data that arrived before the end is unaffected — it is already queued and stays readable, which is what a peer-initiated end owes its reader. Both halves are now covered by tests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…fications `isJsonRpcRequest` accepted a request with no `id` — a JSON-RPC notification — but `dispatch` always produces a response and the vat always writes it. On a persistent line-delimited socket that extra reply sits in the client's buffer and is read as the answer to some later request, scrambling request/response pairing from then on. Requiring an id keeps the invariant that every line in gets exactly one line back, which is what keeps the stream in step. Notifications would be pointless here anyway, since both methods exist to return a value. It also makes the type predicate honest: `JsonRpcRequest.id` is `JsonRpcId`, which does not include `undefined`. An explicit null id is still accepted, being legal in a request; only an absent one is rejected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts: # packages/kernel-node-runtime/src/kernel/make-kernel.ts # packages/ocap-kernel/CHANGELOG.md
…ous incarnation Per review. `registerAnonymousKernelObject` recorded its object only in the in-memory routing table, but `initKernelObject` and `pinObject` both write to the store — so an anonymous object survived a restart while its routing entry did not. Unlike a named service there is no name to re-register it under, leaving it unreachable but still pinned, accumulating with every restart. Worse, it stayed owned by `'kernel'`, so a delivery to a stale connection kref would reach `invokeKernelService`, find nothing registered, throw, and kill the run loop — the same failure this PR's other fix exists to prevent. Anonymous objects are now recorded in the store and swept at init, before the run queue starts so nothing can be delivered to a stale kref in the meantime. These host things that cannot outlive the process — an accepted socket connection, say — so a survivor is unambiguously garbage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A promise has no own enumerable properties, so the response walker
turned one into `{}` and JSON.stringify accepted it.
…vice A throw escaped the crank and killed the run loop. The init sweep cannot prevent this: a (1,1) refcount baseline keeps the object alive.
JSON.stringify turns NaN and ±Infinity into null, which is indistinguishable from the null a void method returns.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a29b09a. Configure here.
A request that failed partway left its @@j<n> names in the table, and sequential names make an undisclosed one guessable.
| @@ -0,0 +1,407 @@ | |||
| /** | |||
There was a problem hiding this comment.
shouldn't we add some tests that two connections get separate tables, that a name minted on A misses on B, or that a stalled connection doesn't block accept()?
There was a problem hiding this comment.
Yes to all three, and they were a real gap: src/vat/index.ts had no test file at all, so the accept loop — the part your three cases are about — was entirely uncovered. Added in d0a600b as src/vat/index.test.ts, driven through buildRootObject/bootstrap with a stand-in listener, so it exercises the real wiring rather than re-testing the bridge in isolation.
Four tests, one per property you named plus a cleanup one:
- Separate tables / a name from A misses on B — connection 1 redeems a URL and is given
@@j1; connection 2 then forges@@j1and getsINVALID_PARAMS. It also asserts the message namesconnection 2, so the label actually earns its keep. - Independent minting — both connections redeem their own URL, both are independently given
@@j1, and each@@j1resolves to its own reference (ocap://alphavsocap://beta). That is the stronger statement: not just that a foreign name misses, but that the same name means different things per connection. - A stalled peer does not block
accept()— connection 1 never sends and never hangs up (itsread()never settles); connection 2 is still served, andaccept()still reaches the call that drains the queue. - Cleanup — a connection is closed once its peer goes away.
I verified these fail for the right reasons rather than trusting them green: awaiting serveConnection fails only the liveness test, and hoisting the bridge so connections share one table fails only the two isolation tests. Restored the module byte-identical afterward.
Two notes on how they are built. E() is non-functional under mock-endoify (HandledPromise.applyMethod is not a function), so the test substitutes identity for E — every target it reaches is a local plain object, and what is under test is loop shape, not eventual-send semantics. That is the same reason bridge.ts takes redeem/invoke as hooks. And the mock baggage/listener/connection helpers went in test/helpers.ts per the repo convention for single-package test utilities; test/ is now in tsconfig.json but not tsconfig.build.json, so it stays out of dist.
| 'params.target must be a string', | ||
| ); | ||
| } | ||
| const match = /^@@([A-Za-z0-9]+)$/u.exec(bag.target); |
There was a problem hiding this comment.
| const match = /^@@([A-Za-z0-9]+)$/u.exec(bag.target); | |
| const match = MARKER_PATTERN.exec(bag.target); |
There was a problem hiding this comment.
Applied in d0a600b. MARKER_PATTERN is character-identical to the literal it replaces, and it has no g/y flag, so sharing it with expandMarkers carries no lastIndex state — behavior-preserving, and now there is one definition of what a marker looks like instead of two that could drift.
| if (!match) { | ||
| throw new BridgeRpcError( | ||
| JSON_RPC_ERROR.INVALID_PARAMS, | ||
| 'params.target must be a marker string like "@@j1"', |
There was a problem hiding this comment.
| 'params.target must be a marker string like "@@j1"', | |
| `params.target must be a marker string like "${MARKER_PREFIX}j1"`, |
There was a problem hiding this comment.
Applied in d0a600b. Same reasoning — the message now derives the sigil from MARKER_PREFIX rather than hardcoding @@, so it cannot contradict the constant. The rendered text is unchanged, which the existing tests confirm.
sirtimid
left a comment
There was a problem hiding this comment.
Some comments but we could merge as is imo
| - The chip/orchestration-demo branch is checked out at the same path | ||
| as before (openclaw plugins install with `-l` from the workspace, | ||
| so the branch update is picked up automatically). | ||
| - `yarn workspace @metamask/kernel-cli build` and | ||
| `yarn workspace @ocap/ocap-jsonrpc-vat build` have run at least | ||
| once since the branch update. |
There was a problem hiding this comment.
Did you mean to commit this file?
There was a problem hiding this comment.
No, I think that's a mistake on the bot's part.
There was a problem hiding this comment.
Good catch — removed in d0a600b. It should not have been in this PR: it documents rehearsing the orchestration demo on the VPS, referencing reset-everything.sh, ~/.ocap-consumer, the chip/orchestration-demo branch, and the openclaw plugins (with a stale -l flag, which is a further sign of where it came from). None of that belongs in a general-purpose package landing on main.
It still exists on chip/orchestration-demo, where it is accurate and useful, so nothing is lost by dropping it here — I checked before deleting. Nothing referenced it either.
I also checked the two sibling files under scripts/ for the same problem, since they arrived by the same route: probe.mjs and start-ocap-jsonrpc-vat.sh are both free of demo-specific references, so the removal stops at this one file.
…iveness Also reuse MARKER_PATTERN/MARKER_PREFIX instead of literals, and drop the demo-specific VPS rehearsal doc, which belongs on the demo branch.

Adds
@ocap/ocap-jsonrpc-vat, a general-purpose building block extracted fromchip/orchestration-demo. Nothing demo-specific here — the demo happens to be its first consumer.What it is
A vat that serves line-delimited JSON-RPC 2.0 on an
IOListenerendowment, so local non-vat processes can reach kernel objects over a persistent socket instead of shell-execing the CLI once per call. Two methods:redeemURL(url)— redeem an OCAP URL via the kernel'socapURLRedemptionService, returning a name for the resulting referencesend(target, method, args)— invokeE(target)[method](...args), expanding names inargsto live references and substituting any remotable in the result for a nameThe vat has no other public facet; the socket is the whole interface. Its authority is exactly the redemption-service endowment, whatever the URLs redeem to, and whatever those references hand back.
The part worth reviewing carefully
Names are
@@j<n>sigil strings scoped to a single connection, and that scoping is load-bearing rather than incidental.The client is outside the ocap world. It can't hold a reference, so it holds a string — and strings are forgeable. Confining the name table to one connection is what makes forgery harmless: a made-up name misses that client's own table and resolves to nothing. If the table were shared across connections, any client could name another's references simply by guessing, which would hand out authority nobody granted.
So each connection gets its own
makeBridge, and the accept loop serves connections concurrently — deliberately not awaiting each one, so a single long-lived or stalled client can't keep others out.Two consequences worth knowing:
The
@@jprefix is deliberately noto—o<n>reads like a vref (o+N/o-N) andkoNlike a kref, and these are neither. They're connection-local nicknames, never kernel references, and the vat never sees a kref at any point.Validation
Full monorepo with this package added: 31/31 builds, 53/53 test tasks, lint clean. The lockfile change is a single additive workspace entry with no dependency resolution churn; all four runtime dependencies already exist in the monorepo. The package is
private: true, so it is not published and carries no semver obligations yet.🤖 Generated with Claude Code
Note
Medium Risk
New local IPC surface that redeems OCAP URLs and invokes arbitrary methods on redeemed references; security hinges on per-connection name scoping and atomic name disclosure, which are explicitly tested but warrant careful review.
Overview
Introduces
@ocap/ocap-jsonrpc-vat, a new workspace package (private, not published) that lets local non-vat clients reach kernel objects over a line-delimited JSON-RPC 2.0 Unix socket instead of one-off kernel-cli RPCs.The vat exposes
redeemURL(viaocapURLRedemptionService) andsend(E(target)[method](...args)), mapping live remotables to connection-local@@j<n>sigils in both directions. Each accepted socket gets its own bridge and name table, served concurrently so a stalled client does not block new peers; names reset on disconnect. The bridge commits new names only after a successful, JSON-encodable reply and rolls them back on errors or non-serializable results (unsettled promises, non-finite numbers, etc.) so clients cannot guess undisclosed references.Also ships
makeOcapJsonrpcClusterConfig, launch/probe scripts, Vitest coverage for bridge + vat accept loop, and monorepo/tsconfig/yarn workspace wiring.Reviewed by Cursor Bugbot for commit d0a600b. Bugbot is set up for automated code reviews on this repo. Configure here.