feat: runtime plugin system — capability host, marketplace, install lifecycle (server + web)

[ccdwyer]

Summary

Adds a runtime plugin system to t3code — install third-party features (server + web/Electron) at runtime from a marketplace, sandboxed behind curated, capability-scoped host APIs. No fork required to extend the app.

A plugin is a signed tarball with a manifest declaring the capabilities it needs. The host serves its web bundle same-origin behind an import map (sharing React/Effect/atoms as singletons), runs its server module in an isolated Effect scope, and exposes only the host APIs the manifest declared — validated at install time with an explicit consent step.

What's included

Built as reviewable slices (happy to land these as a stacked series if you'd prefer — see "Splitting" below):

1. Thread ownership — projection_threads.owner (default user), so plugin-created agent threads are attributable and filterable without disturbing existing flows.
2. Server host + transport + scopes — per-plugin lifecycle (verify → import → register → migrate → recover → mount), a WS RPC envelope (plugins.call/subscribe) with a plugins:manage scope, and plugin-aware auth scopes bound atomically to each method.
3. Curated capability façades — database (plugin-prefixed tables gated at migration time), filesystem (workspace-scoped, realpath-contained), httpClient (HTTPS-only, DNS-pinned SSRF guard), secrets, vcs, sourceControl, agents (owner-injected, built on the orchestration engine — never raw provider access), terminals, projections.read, environments.read, textGeneration. Nothing raw is handed out; each façade is a narrow, auditable surface.
4. Web host + extension points — PluginUiHost loads/renders plugin web bundles; plugins can contribute routes, sidebar sections, settings pages, command-palette entries, and per-project sidebar actions. Plugins ship their own compiled CSS isolated in a lowest-priority cascade layer.
5. Marketplace + install lifecycle — sources, catalog browse, install-consent dialog (shows every requested capability), install/enable/disable/uninstall/upgrade, streaming-safe download with sha256 + gzip-bomb protection, single-writer lockfile, safe-mode crash counter.
6. Example fixture — an in-repo hello-board plugin plus a real install-and-drive integration test that installs it through the real installer and round-trips an RPC.

Real-world example plugin

The plugin APIs are proven by porting a substantial existing feature — a board-as-state-machine workflow engine — into a standalone installable plugin with 46 RPC methods (boards, tickets, agent-driven lanes, work-source sync, webhooks, self-improve proposals), built entirely on the capability APIs above:

👉 https://github.com/ccdwyer/workflow-boards-plugin

It's distributed separately (not in this PR) so the core stays generic; it's the reference for what these APIs can carry. Two capability methods here exist to serve it and are generic-additive in the same spirit as the rest: vcs.diffRefToWorkingTree (diff an arbitrary base ref against the working tree) and textGeneration.generateBoardProposal (structured generation under a per-provider no-tool posture — the model provably cannot run tools or write files).

Testing

• Unit tests across capabilities, transport, scopes, install lifecycle, and the no-tool text-generation postures.
• The in-repo HelloBoardFixture install-and-drive integration test exercises the real module loader → runtime build → RPC dispatch path.
• The example plugin was separately verified end-to-end against the real runtime (all 46 methods installed + dispatched), and the no-tool board-proposal path confirmed against a real model.

Splitting

This is a large PR because a plugin system touches many layers at once. If you'd rather review incrementally, I can restructure it as a stack: (1) thread ownership → (2) server host + transport + scopes + capabilities → (3) web host + extension points → (4) marketplace + install + fixture. Just say the word.

Notes

• Full-trust-in-process is the security model: the façades defend a well-behaved plugin against malicious data (traversal, SSRF, prompt-only egress), not a sandbox against malicious plugin code — consistent with a curated marketplace + install consent. Documented inline.
• Rebased onto latest main.

https://claude.ai/code/session_011H86UHL1RPuWBX3LBTUPsk

---

Note

High Risk
Changes authentication scope satisfaction and token exchange, adds a full-trust in-process plugin host with outbound HTTP, and alters thread visibility in projections—security- and data-visibility sensitive areas.

Overview
Introduces a server-side plugin runtime wired to @t3tools/plugin-sdk: derived pluginsDir, lockfile/catalog/host lifecycle (activate, migrations, capability gating, crash/safe-mode behavior), HTTP route registry, outbound SSRF validation for plugin HTTP clients, and a hello-board install-and-RPC integration test.

Auth moves from environment-only scopes to AuthScope (core + plugin:<id>:read|operate + plugins:manage). Token exchange and pairing delegation use satisfiesScope so full standard-client grants implicitly satisfy plugin scopes without listing them verbatim; pairing links and sessions persist the wider scope set.

Threads gain an owner column (default user, migration 033). Plugin-owned threads (plugin:…) stay in the command read model but are hidden from user-facing projections (counts, shell snapshots, startup thread pick) while remaining reachable by id.

Static index.html responses inject plugin host head HTML for shared web singletons. DB migration 034 adds plugin_migrations tracking.

^{Reviewed by Cursor Bugbot for commit b2e4df3. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add runtime plugin system with capability host, marketplace, and install lifecycle

• Introduces a full plugin subsystem on the server: host lifecycle (PluginHost), runtime registry, lockfile store, catalog, marketplace, installer, module loader with ESM singleton hooks, RPC dispatcher, HTTP route serving, and per-plugin SQLite migrations
• Adds a @t3tools/plugin-sdk (server) and @t3tools/plugin-sdk-web (web) package exposing plugin capabilities: agents, VCS, terminals, database, filesystem, HTTP client, secrets, environments, source control, and text generation
• Exposes plugin management over WebSocket RPC (list, call, subscribe, installBegin/Confirm/Abort, setEnabled, uninstall, upgradeBegin/Confirm, checkUpdates, source CRUD) with scope-based authorization via the new satisfiesScope helper
• Extends the auth scope model with plugins:manage and per-plugin plugin:<id>:{read|operate} scopes; standard-client token holders implicitly satisfy plugin scopes without explicit enumeration
• Adds plugin UI hosting in the web app: import-map injection into index.html, dynamic route and settings rendering, sidebar sections, project actions, and command palette integration
• Introduces thread owner field (migration 033) so plugin-owned threads are excluded from user-facing lists and agent awareness relay
• Adds generateBoardProposal to all text-generation providers, running Claude/Codex in a strict no-tool posture from a scoped temp directory; Cursor and Grok explicitly reject the operation
• Risk: large surface area added in a single PR; ESM loader hook registration (node:module.register) is best-effort and emits a warning when unavailable rather than failing hard

^{Macroscope summarized b2e4df3.}

[macroscopeapp]

One Effect service convention finding on the reviewed changes. See inline comment.

^{Posted via Macroscope — Effect Service Conventions}

[macroscopeapp]

When wrapping a real register() failure, PluginRegistrationError discards the underlying error: detail is set to Cause.pretty(cause) and message (line 92) is then built from that stringified cause. Per the error conventions, the immediate failure should be preserved as cause and the wrapper message derived only from stable structural attributes—not from a stringified/pretty-printed cause. Cause.pretty also copies unbounded defect/stack text into a caller-visible field.

Every other error in this package (e.g. PluginModuleLoadError, PluginLockfileLockError, PluginLockfileWriteError) already carries cause: Schema.Defect(). Add a cause field to PluginRegistrationError and construct it here with the real cause plus a stable detail (e.g. "plugin register() failed"), leaving detail for the pure validation constructions.

^{Posted via Macroscope — Effect Service Conventions}

[ccdwyer]

Thanks for the continued review — round-3 fixes pushed in b416af370. Mapping each item to its change:

Lifecycle / interrupt-safety

• Same-version upgrade destroyed the live plugin — beginUpgrade now rejects a same-version request before staging, so the destructive move can never run against the live version dir.
• loadPlugin persisted failed on interrupt-only teardown — a clean shutdown / concurrent lifecycle change now propagates the interruption and leaves persisted state intact. Shared a new handleLoadFailureCause across the hot-activation and pre-scope start paths so all three call sites behave identically (matching the round-2 resolveRegistration interrupt handling). Genuine errors still persist failed.
• Runtime could go live after a concurrent disable/uninstall — re-check lockfile state immediately before registry.put; if no longer active, abort via Effect.interrupt (closes the scope → removes any partial HTTP registration, skips the put + active publish).
• Hung register()/recover() could stall startup — both bounded by a 30s Effect.timeout (T3_PLUGIN_HOST_REGISTER_TIMEOUT_MS override); a timeout flows into the normal markFailure path.
• uninstall published disabled — deactivatePlugin now re-reads and publishes the actually-persisted state (pending-remove vs disabled).

Install / lockfile

• stageTarball decompress leak — wrapped in the same staging-dir cleanup the sibling steps use.
• Advisory-lock finalizer could delete a foreign lock — the lock now carries a per-holder pid:uuid owner token; the finalizer only removes a lock still carrying its own token, so a reclaimer's now-valid lock is never clobbered by a slow prior holder.
• addSource didn't rewrite a legacy credentialed URL — a matched duplicate source's URL is rewritten to the credential-stripped canonical form (opaque sourceId preserved).

Capability API

• available() used Effect.die — now a typed Effect.fail(PluginCapabilityUnavailable), so a plugin calling an undeclared capability can catch/degrade instead of crashing on a defect (SDK already types the failure channel).
• observeThread snapshot→live race — now subscribes the filtered live event stream into a bounded (256, back-pressured) queue before reading the snapshot, then emits snapshot followed by buffered events filtered to sequence > snapshotSequence (dedups anything already in the snapshot). The engine commits the projection update inside the txn before publishing to the domain-event PubSub, so this closes the gap; a hard scheduler-independent guarantee would need a subscription-readiness signal from streamDomainEvents (an engine-level change ws.ts's subscribeThread also lacks) — documented inline rather than touching ws.ts.

Pairing-link plugin scopes (display-only)

• Widened AuthPairingLink.scopes AuthEnvironmentScopes → AuthScopes (propagates to the pairingLinkUpserted event + AuthAccessSnapshot.pairingLinks); PairingGrantStore list/emit views pass plugin scopes through instead of filtering them, and the web AccessScopeSummary renders plugin:<id>:* verbatim. The grant/consumption/security path is untouched.

All green: full 18-package typecheck, apps/server src/plugins (127 tests) + src/auth suites. New regression tests added for the upgrade guard, decompress cleanup, lock-ownership finalizer, credentialed-URL rewrite, pending-remove publish, typed-capability catch, and the observeThread dedup.

[ccdwyer]

Round-4 fixes pushed in e0f6b6240. Mapping each finding to its change (full 18-package typecheck + apps/server src/plugins/src/auth = 179 tests, all green; the two subtlest fixes were mutation-verified to fail on the old code):

Lifecycle

• start-loop aborted on one plugin's interrupt (High) — this was a regression from round-3: propagating an interrupt-only cause is right for a single on-demand activation, but inside the startup loop the pre-registry.put state re-check firing Effect.interrupt for one plugin skipped all remaining plugins. The loop now logs and continues on an interrupt-only cause (non-interrupt causes still go through handleLoadFailureCause); a genuine host-shutdown interrupt still stops the loop via re-interrupt + the trailing ignoreCause. The single-plugin activatePlugin path is unchanged. New two-plugin test proves a later plugin still activates after one is interrupted.
• Activation published a stale active — the success path now re-reads and publishes the actually-persisted lockfile state (mirroring the round-3 deactivate fix), so a concurrent disable/uninstall landing after registry.put isn't contradicted.

Install / lockfile

• compareSemver mis-ordered prerelease/build metadata — replaced with a semver.org §11-correct comparator (release outranks prerelease, numeric identifiers compared numerically and ranked below alphanumeric, longer identifier set wins on tie, build metadata ignored), so checkUpdates can't advertise a wrong "latest" (1.0.0-rc.10 vs rc.2, rc.1 vs 1.0.0). Unit-tested across the full §11 chain.
• openLock partial-write orphaned the lock — if writeAll/sync fails or is interrupted after the wx create, the file is now removed, so it can't block all lockfile mutations until STALE_LOCK_MS. Tested with an injected FS whose writeAll fails.

Agents capability

• requireOwnedThread conflated not-found with ownership — a missing thread now yields AgentsThreadNotFoundError, a real owner mismatch yields AgentsThreadOwnershipError, owned-by-us still passes. The guard is not weakened; callers' not-found branches are now reachable.
• startTurn dropped bootstrap when it created the thread — it now forwards the bootstrap with only createThread stripped, so the first turn keeps its prepareWorktree/runSetupScript.
• turnAliases leaked on a failed start for an existing thread — the error path now always drops the pending alias, and only rolls back a self-created thread when applicable. (Mutation-verified.)
• observeThread snapshot read race — the detail and the global snapshotSequence are now read sequentially, sequence first, so the dedup threshold can never exceed what the detail reflects → no dropped updates. A bounded duplicate within the read window is idempotent on the revision-gated client. Being simultaneously gap-free and dup-free isn't attainable without a per-thread snapshot sequence (getThreadDetailById exposes none), so this deliberately chooses at-least-once — a lost update is worse than an idempotent re-render — matching how core subscribeThread tolerates duplicate thread-detail events. Documented inline.

[ccdwyer]

Round-5 fixes pushed in 9ae34eee0 (full 18-package typecheck + apps/server src/plugins/src/auth green; the marker-clear regression is mutation-verified — removing it fails the new test).

All three findings trace back to the interrupt handling from the prior rounds, and the root cause of the first two is the same: Effect.interrupt was overloaded for both a per-plugin self-cancel and a genuine host-shutdown interruption, so they couldn't be told apart. The fix introduces a typed sentinel:

• [both bots] Interrupt-only teardown left activatingSince stale → a cleanly-cancelled activation was miscounted as a crash by reconcilePendingState on the next start(), bumping crashCount toward failed. Both the interrupt-only teardown and the new sentinel teardown now clear activation.activatingSince (preserving state/crashCount/lastError) — a clean cancellation isn't a crash. A genuine hard crash mid-activation still leaves the marker set (no teardown ran), so safe-mode's crash-loop protection is unchanged.
• [Cursor] The start-loop swallow also caught host-shutdown interrupts → the pre-put lifecycle re-check now fails with an internal typed PluginActivationCanceled sentinel instead of Effect.interrupt. So the loop continues only on that sentinel (one plugin's concurrent-disable doesn't skip the rest), while a genuine interrupt-only cause is re-raised through handleLoadFailureCause and propagates out of the loop — the trailing ignoreCause ends start promptly on shutdown instead of plodding through the remaining plugins.
• [Macroscope, secondary] turnAliases unbounded for turns started but never awaited → bounded by a generous FIFO cap (4096). Normal turns prune on terminal read well before the cap, so eviction only affects long-abandoned un-awaited turns (a later awaitTurn then degrades to the not-found path — not a crash — only under pathological >cap concurrent un-awaited turns).

Cancel/interrupt matrix now: (a) concurrent disable during activation → sentinel → marker cleared, no failed, crashCount untouched, loop continues; (b) host shutdown mid-activation → interrupt-only → marker cleared, re-raised, loop stops promptly; (c) genuine activation error → markFailure (unchanged).

[macroscopeapp]

Effect service review: one error-modeling convention violation found in PluginLockfileStore.ts.

^{Posted via Macroscope — Effect Service Conventions}

[ccdwyer]

Round-6 fixes pushed in 67ca9e848 (full 18-package typecheck + apps/server src/plugins/src/auth green; the cause predicates are locked by unit tests over hand-built mixed causes).

• [both bots] The activation-cancel check was "contains" when it must be "only" — and interrupts must win. Scope.close can produce a mixed cause (the PluginActivationCanceled sentinel combined with a finalizer/teardown error, or with a fiber interrupt), and the round-5 .some(...) check folded those into the benign-cancel branch — silently dropping a real teardown failure and masking a shutdown interrupt. Replaced with two predicates: causeContainsInterrupt (Cause.hasInterrupts) checked first → re-raise so shutdown stops promptly even when mixed; then causeIsActivationCanceledOnly (every reason is the sentinel) → benign cancel; then everything else — including the sentinel combined with a teardown error — goes to markFailure and is never dropped. Both predicates are exported and unit-tested (sentinel-only → cancel; sentinel+die → failed; sentinel+interrupt → re-raised; pure interrupt → re-raised).
• [Macroscope] startTurn idempotent-retry alias leak. The engine dedups dispatch by commandId, so a retry with the same caller-supplied commandId persists no new turn — but the old code returned fresh random turnId/messageId and aliased to a messageId that was never persisted, so a later awaitTurn couldn't correlate and timed out. When a commandId is supplied, turnId/messageId are now derived deterministically from it (sha256, opaque, same id prefixes — no raw commandId leak), so a retry resolves the same alias the first call persisted. The no-commandId path is unchanged. Tested: same-commandId retry returns identical ids and the real-engine dedup leaves one turn whose pendingMessageId equals the returned messageId.
• [Effect Service Conventions] PluginLockfileCorruptError.message now derives from the stable path only (was echoing String(cause), which could leak corrupt lockfile contents); the underlying failure is still preserved on cause, and the redundant detail field is dropped — matching the sibling read/write/lock errors.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 67ca9e8. Configure here.}

[macroscopeapp]

One convention finding in the plugin HTTP client capability. See inline comment.

^{Posted via Macroscope — Effect Service Conventions}

[ccdwyer]

Round-7 fixes pushed in b2e4df306 (full 18-package typecheck + apps/server src/plugins/src/auth = 191 tests green).

Activation concurrency (both High):

• activatePlugin had no single-flight — two concurrent calls both observed an empty registry, both ran loadPlugin, and registry.put overwrote the first runtime's entry, orphaning its scope (forked services / terminal finalizers / HTTP routes stayed live but unreachable). Now serialized per pluginId by a single-permit semaphore, get-or-created atomically under a SynchronizedRef<HashMap<pluginId, Semaphore>>, with the registry.get double-check inside the lock. deactivatePlugin shares the same per-plugin lock so activate/deactivate can't interleave (neither calls the other under the lock → no deadlock).
• activatePlugin swallowed a lockfile read failure — a transient/corrupt read was replaced with an empty lockfile, so it early-returned as "success" while the plugin never activated (installs/enables looked done but weren't). The error now propagates (signature widened; PluginInstaller.setEnabled maps it to an activation-failed error). A genuinely missing lockfile is still EMPTY_PLUGIN_LOCKFILE via readLockfile, so only real read/parse errors surface.

Agents capability:

• awaitTerminalTurn subscribe/read race (High) — forkScoped returns before the watcher subscribes, so a turn that reached terminal in that gap produced no waking event and awaitTurn hung to the timeout. Now races the event-driven wake against a bounded re-poll, so a missed event still makes progress; the outer timeoutOption still bounds it.
• startTurn idempotent-retry (follow-up to round 6) — a retry with the same commandId but no messageId derived a fresh id and overwrote the alias, breaking correlation when the first call supplied an explicit messageId. It now reuses the messageId the first call persisted (located via the deterministic turnId's existing alias).

Conventions: HttpClientError's transport-failure message now uses a stable reason instead of echoing cause.message (the underlying error stays on data.cause).

---

A few older Bugbot threads re-surfaced this round — these are already addressed in the current tree (they date from the pre-round-1 e3a2cb60f review):

• "Pairing check ignores implicit scopes" → the pairingCredential handler uses satisfiesScope (honors the implicit standard-client marker), not verbatim .has, since round 1.
• "observeThread misses live events" → observeThread subscribes into a bounded buffer before reading the snapshot (round 3), then replays filtered by sequence.
• "Healthy window counts as crash" → the success path clears activatingSince immediately via markActivated(false); it does not linger until the healthy-delay job.

[macroscopeapp]

🟡 Medium plugins/PluginInstaller.ts:824

In confirmInstall, the lockfile entry is committed with state: "active" before host.activatePlugin() runs. When activatePlugin fails, the error is mapped to activation-failed and returned, but the lockfile entry is left in the active state even though no runtime was started — the plugin is recorded as running when it is not. Consider updating the lockfile entry to reflect the activation failure (for example, transitioning to an inactive/error state) before returning the error.

🤖 Copy this AI Prompt to have your agent fix this:

In file @apps/server/src/plugins/PluginInstaller.ts around line 824:

In `confirmInstall`, the lockfile entry is committed with `state: "active"` before `host.activatePlugin()` runs. When `activatePlugin` fails, the error is mapped to `activation-failed` and returned, but the lockfile entry is left in the `active` state even though no runtime was started — the plugin is recorded as running when it is not. Consider updating the lockfile entry to reflect the activation failure (for example, transitioning to an inactive/error state) before returning the error.

[ccdwyer]

Superseded by an 8-PR stacked series — the same runtime plugin system, re-sliced into small, dependency-ordered, individually-green PRs for easier review. Before submission each slice was re-reviewed by a four-model tribunal (GPT-5.5, Grok, GLM-5.2, Fable) that traced execution rather than the diff — surfacing a batch of real issues (SSRF-via-redirect, a stderr credential leak, a shell-stream visibility leak, an Effect.exit interrupt bug, an actually-broken DNS pin, a gh exit-code merge-safety bug, and ~25 more), all fixed and folded in. Closing this in favor of:

1. Plugin system (1/8): thread ownership #3725 — thread ownership (standalone)
2. Plugin system (2/8): host foundation + plugin SDK #3726 — host foundation + plugin SDK
3. Plugin system (3/8): RPC transport + plugin-aware auth scopes #3727 — RPC transport + auth scopes
4. Plugin system (4/8): mechanical capability façades #3728 — mechanical capability façades
5. Plugin system (5/8): agents + vcs capabilities #3729 — agents + vcs capabilities
6. Plugin system (6/8): web host + extension points #3730 — web host + extension points
7. Plugin system (7/8): marketplace + install lifecycle #3731 — marketplace + install lifecycle
8. Plugin system (8/8): filesystem/httpClient capabilities + hardening #3732 — filesystem/httpClient + hardening

Merge bottom-up (start with #3725). Thanks for the 7 rounds of Bugbot/Macroscope review here — every finding is carried forward into the stack.

https://claude.ai/code/session_011H86UHL1RPuWBX3LBTUPsk