Live share for q2 preview - #464
Open
shikokuchuo wants to merge 22 commits into
Open
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
shikokuchuo
force-pushed
the
feature/preview-live-share
branch
from
August 8, 2026 16:48
12981af to
7cca372
Compare
…bd-9gam4jqe)
Add crates/quarto-p2p to the workspace: public API stubs only
(PreviewShareTicket, TunnelHost/TunnelClient + handles, TunnelStatus,
TunnelError), all bodies todo!("Phase 1 (bd-v8mwzpmi)"). Deps per plan:
iroh 1.0.3 (default features), iroh-tickets 1.0, subtle 2, rand 0.9,
tokio with explicit io-util+net, tracing, thiserror. Adds the
[workspace.dependencies.quarto-p2p] entry for Phase 2's consumer and
commits the epic plan file.
Gate 0 static checks re-confirmed on the real wiring:
- WASM closure clean: cargo tree -i iroh from wasm-quarto-hub-client
fails to match any package
- dep set identical to what the gate measured: Cargo.lock additions are
name+version-identical to the spike branch's (141 external packages;
iroh 1.0.3 + iroh-tickets 1.0.0) — no Q4 re-measure needed
- Windows compile leg pending a pushed CI run (test-suite CI has no
Windows matrix); the gate proved this exact dep set on windows-latest
2026-08-04 (run 30894960520)
cargo build --workspace green; cargo xtask verify --skip-hub-build
passed all steps.
Plan: claude-notes/plans/2026-08-03-q2-preview-live-share-iroh.md (Phase 0)
Lands the plan's Phase 1 test specs before any implementation (TDD):
ticket::{roundtrip,rejects_garbage_and_foreign_kinds,debug_redacts_token},
tunnel::{http_roundtrip_loopback,websocket_frames_survive,
wrong_token_rejected,client_redials_after_connection_loss,
half_close_propagates,clean_shutdown,idle_pooled_conn_survives_quic_keepalive}.
All 10 fail via todo!() stubs (verified: cargo nextest run -p quarto-p2p
-> 10 FAIL). Hermetic iroh only: presets::Minimal, RelayMode::Disabled,
loopback binds. Public API surface extended with TunnelHostConfig/
TunnelClientConfig (EndpointPreset::HermeticLoopback for tests),
TunnelClientHandle::status(), TicketParseError re-export.
…-v8mwzpmi) Implements the tunnel under the tests landed in c146ca6 (TDD; all 10 were failing via todo!() stubs, now 10/10 pass): - ticket.rs: iroh_tickets::Ticket impl, KIND "q2preview", postcard wire format following the versioned-enum convention (Variant1 {id, addrs, token}); Display/FromStr via encode_string/decode_string; manual Debug redacts the token. - host.rs: Router accept loop; per stream read_exact of the 32-byte token under a 10 s timeout, subtle::ConstantTimeEq compare, then terminated splice copy_bidirectional(join(recv, send), tcp). Bad or short token => stream reset + connection close + warn log with remote_id().fmt_short(). N0 preset wraps online() in a 10 s timeout and degrades to direct/LAN-only with a warning. - client.rs: MemoryLookup seeded from the ticket; local TcpListener; one TCP conn = one token-prefixed bi-stream. A supervisor task parked on conn.closed() re-dials with expo backoff (250 ms..5 s, 10 s per attempt) and drives the Connected/Reconnecting status watch; per-conn handlers wait on the watch with a 30 s budget, then drop the conn. - Shutdown: host router.shutdown() (handles JoinError); client awaits the aborted accept-loop task so the local port is provably unbound, then closes the endpoint. Hermetic test posture: EndpointPreset::HermeticLoopback = presets:: Minimal + RelayMode::Disabled + loopback binds; TunnelHostConfig's secret_key/token/bind_addr overrides exist for the restart-same- identity re-dial test. No n0 infrastructure in CI. Verification (output inspected): cargo nextest run -p quarto-p2p 10/10; cargo build --workspace; cargo nextest run --workspace 10873 passed; cargo xtask verify --skip-hub-build all green; cargo tree -i iroh from wasm-quarto-hub-client still fails (WASM closure clean).
Host side of live share: `q2 preview --share` spawns a quarto-p2p
TunnelHost in front of the preview server's loopback port and prints a
capability banner + ready-to-paste `q2 preview --join q2preview…` line
(join line last, bare, for copy-paste through terminal wrapping).
- CLI: `--share` flag; `--join <TICKET>` declared hidden with
conflicts_with("share") + a runtime Phase 3 bail; first clap parse
tests for the q2 CLI (try_parse_from harness in main.rs).
- quarto-preview: new `share` module — start_share_session() +
format_share_banner() + share_target(); PreviewConfig::share;
session spawned in run_with_on_ready before the server starts and
shut down after run_server_with returns (before the CLI's TempDir
drop). Banner carries the direct/LAN-only notice when the ticket
has no relay addr (quarto-p2p's tracing::warn is invisible at the
default `quarto=warn` filter).
- quarto-p2p: PreviewShareTicket::has_relay_addr(); tunnel-client
example (reference guest until Phase 3's real --join).
TDD: CLI tests failed first via E0026 missing-field compile errors;
share-glue tests 4/4 failed on todo!() stubs, then went green.
Verified: cargo nextest run --workspace 10883 passed; cargo xtask
verify --skip-hub-build all green; cargo tree -i iroh from
wasm-quarto-hub-client still fails (WASM closure clean). Recorded
end-to-end run (host --share + example guest + Playwright browser:
/health identical through tunnel, render 1.47s, live edit propagated
1.07s, SIGINT exit 0) in the plan.
Plan: claude-notes/plans/2026-08-03-q2-preview-live-share-iroh.md
Guest side of live share: `q2 preview --join <ticket>` parses the
q2preview join string, dials the host over iroh, and serves the shared
session on a local loopback proxy — no local project, TempDir, or hub.
CLI (crates/quarto):
- --join unhidden with the full conflict matrix (path, --share,
--no-project, --allow-edit, --data-dir, --preview-dir rejected;
--port/--host/--no-browser compose); --ui joins the matrix in Phase 4
with the flag itself
- run_join: clear error UX for malformed tickets, unreachable hosts
(bounded 10 s dial), and rejected tokens; status lines from the
tunnel watch channel ("connected via direct connection|relay",
"connection lost — reconnecting…"); Ctrl-C teardown (29 ms connected,
3.0 s while reconnecting — iroh's close budget)
- browser-open gated on the first GET /health *through the tunnel*
(wait_until_healthy; a local TCP accept would lie when the host is
gone), open-anyway-on-timeout floor as in host mode
quarto-p2p:
- TunnelStatus::Connected now carries a PathKind (Direct/Relay/Unknown)
fed by a per-connection paths_stream() watcher; conn-generation guard
keeps a dying connection's straggler snapshot from overwriting the
re-dialed connection's kind
- terminal TunnelStatus::Rejected: the client maps the host's
unauthorized close (shared ERROR_CODE_UNAUTHORIZED) to a no-re-dial
terminal state instead of spinning on a token that can never succeed
Tests (landed failing-first; full suite 10897 passed, xtask verify
--skip-hub-build green, WASM closure still iroh-free):
- cli_parse_tests: 7 new conflict/compose tests
- quarto-p2p: status_reports_direct_path_kind,
rejected_token_flips_status_terminal
- money test quarto-preview::join_tunnel::guest_syncs_project_through_tunnel:
real hub in-process + hermetic tunnel; /health via guest port matches
direct, samod dial_websocket through the tunnel syncs the files map
- wait_until_healthy unit tests (200 / non-200 keeps polling / dead)
Single-machine e2e with two concurrent --join guests recorded in the
plan (browser render + live-edit propagation, screenshots inspected).
Cross-machine n0-relay leg still open — needs a second machine or a
GH-Actions guest (push approval), tracked on the strand.
No snapshot (.snap) changes.
Plan: claude-notes/plans/2026-08-03-q2-preview-live-share-iroh.md
…the real n0 relay (bd-6y0p1bne) Run 31092359776: real q2 --join guest on ubuntu-latest against a live --share host on the dev machine. 'connected via relay' on both concurrent guests, first render 12.7s / ~47.5MB through the relay, live-edit propagation median ~1.0s over 4 bumps, screenshots inspected. Throwaway workflow + secret + remote branch cleaned up.
Serve the full hub-client editor from the preview server via
--ui <viewer|editor> (clap ValueEnum, default viewer; conflicts with
--join). The editor boots into the hub-client share route
(#/share/{docId}?server=%2Fws&file=…&name=…) built in an on_ready
closure — the index doc id only exists server-side, so editor mode
defers the URL print + browser-open gate into the callback.
UI × write policy stays a strict 2×2: --ui editor without --allow-edit
is the sandbox mode (session edits sync live, disk stays authoritative)
and prints the ephemeral-edits note; the DiskWritePolicy mapping is
untouched by the UI choice.
Embedding: hub-client's new build:preview-embed script (auth off, sync
server pinned to relative /ws, PWA service worker disabled via new
VITE_DISABLE_PWA so ephemeral origins don't precache ~67 MB) emits
dist-preview-embed/, built by the new cargo xtask
build-hub-client-embed. quarto-preview's build.rs embeds a filtered
copy: files byte-identical to the viewer dist at the same rel path are
stripped (64/187 files, 45.7 MB incl. the 38.4 MB wasm) and served
through the viewer embed by the runtime editor→viewer fallback.
Measured release q2 delta: +22.2 MB (vs ~+69.6 MB naive double-embed).
Placeholder fallback (naming the xtask) keeps unbuilt trees working.
Tests first (observed failing as the structural compile errors, per
the Phase 2 precedent): CLI parse/conflict tests, boot-URL builder +
file-picker units, write-policy 2×2 sweep, embed-contract units, and
an editor-mode server integration test. Workspace suite 10914 passed;
full cargo xtask verify green; browser e2e (both --allow-edit legs)
recorded in the plan.
Plan: claude-notes/plans/2026-08-03-q2-preview-live-share-iroh.md
…f4ryvuq) q2 preview --ui editor boots hub-client via a share URL, but App.tsx gates rendering on project-set status: a fresh browser (the default — preview binds an ephemeral port, so every origin has fresh IndexedDB) landed on the ProjectSetSetup create/migrate page instead of the preview. Onboarding for a synced project list is pointless against a throwaway per-session hub. The preview server's boot URL now carries ephemeral=true on the share route (build_editor_boot_url). hub-client captures the flag once at mount (before the share handler clears the URL), then mirrors the join-collection invite-first pattern: silently establish the personal root set against /ws (createProjectSet on needs-setup, migrateProjects on needs-migration) and bypass the needs-setup/needs-migration/error ProjectSetSetup gates. Production share links never carry the param (buildShareableUrl/buildHashRoute unchanged), so the production onboarding flow is untouched. Verified: new Rust + vitest cases failed pre-implementation and pass after; cargo xtask verify — all 14 steps. E2E against the real binary (headless Chromium, fresh profile): editor mounts with index.qmd and ProjectSetSetup never renders; control run without the param shows the setup page as before. hub-client changelog entry waived for this change.
q2 preview --join against a host running --ui editor --share opened the browser at the guest proxy's root route, which carries no document coordinates: a fresh profile hit the ProjectSetSetup gate, and even past it the app landed on ProjectsHome — the share handler that joins the document only runs for #/share/… URLs, so guests never joined. The host's editor-mode on_ready now stashes its boot params (index doc id, file, project name) via quarto_preview::set_editor_boot, and GET /api/preview/config carries them as editorBoot. The guest fetches the config through the tunnel after its /health readiness probe and boots the same share URL the host printed — ephemeral=true included, so the bd-zf4ryvuq machinery skips project-set onboarding — built from the same editor_share_route helper as the host's URL. Viewer-mode and older hosts answer without editorBoot and keep the root URL, so both skew directions degrade to today's behavior. The ticket can't carry the doc id (minted before the hub boots), which is why the params ride the config endpoint instead; the doc id was already exposed to guests via /health. Verified: TDD (compile-red confirmed) — new quarto-preview integration test plus 5 CLI unit tests; cargo nextest run --workspace 10939 passed; cargo xtask verify --skip-hub-build green. E2E with real host+guest binaries: guest prints the share URL with the host's doc id and ephemeral=true; fresh-profile headless Chromium at the guest URL loads the editor on index.qmd through the tunnel with ProjectSetSetup never rendering, while the root-URL control on the same session still shows the setup gate.
…(bd-4062e8tn)
q2 preview --allow-edit --share aborted after a few minutes: the
periodic sync's sync_document hit a MissingOps panic at
automerge-0.10.0 op_set2/change/collector.rs:761, the unwind escaped
with_document, samod's WithDocGuard::drop panicked on top ("dropped
without comitting"), and the destructor panic during unwind aborted
the process.
Root cause is upstream automerge#1327: fork_at's dependency walk marks
hashes as seen when popped, not when scheduled, so some DAG shapes
(sync_document's every-5s fork_at(checkpoint) + merge against
splice-heavy browser edits produces them) schedule a dep twice; the
duplicate change metadata makes the change collector return MissingOps
and its unwrap panics. Verified with the issue's deterministic repro,
which panics identically on our pinned 0.10.0 and passes with PR
#1366's fix; kept as fork_at_after_sync_and_merge_does_not_panic.
Two-part fix:
- [patch.crates-io] pins automerge to quarto-dev/automerge
rust/automerge-0.10.0-fork-at-missingops = the 0.10.0 tag + the
#1366 cherry-pick (one-hunk schedule-time seen-marking + its
regression test; the fork's own suite passes). Remove once upstream
merges #1366 and ships a release.
- sync_document now runs its automerge work under run_contained:
catch_unwind + in-place heal (save → load rebuilds clean internal
indices; actor id restored explicitly since save/load doesn't
round-trip it), turning any future collector-class panic into an
ordinary sync error instead of a process abort. The collector panic
paths are all read paths, so the document is never half-mutated by
the panic itself.
Verified: containment unit tests (panic caught, doc healed, actor and
heads preserved; Ok/Err pass through untouched); cargo nextest run
--workspace 10942 passed; cargo xtask verify --skip-hub-build green
(automerge is not in the hub-client/WASM dependency chain). cargo tree
confirms a single automerge source across quarto-hub, samod, and
samod-core.
Under DiskWritePolicy::ReadOnly (q2 preview without --allow-edit), the sync checkpoint pairs current heads with the *disk* content hash, so the doc state at those heads has diverged from the never-written-back file. The next sync's fork-apply then rewrote the fork to the stale disk content and the merge deleted every doc-side edit older than the previous checkpoint — with the 5s periodic sync, edits in a --ui editor --share session vanished on host and guest alike within a tick or two. Gate the fork-apply-merge on the filesystem actually having changed. With no disk delta there is nothing to merge; the checkpoint still advances heads so the next sync doesn't re-fire, and a later disk edit still converges the doc to disk per the documented ReadOnly semantics. WriteBack is unaffected: its checkpoint pairs heads with the content written back, so the fork-apply is already a no-op when only the doc changed. Adds regression test test_sync_readonly_repeated_doc_edits_are_not_clobbered (fails pre-fix: the second periodic sync clobbers the first edit). Existing ReadOnly tests — disk-authoritative convergence, no-write-back, repeat-sync stability — all still pass.
shikokuchuo
force-pushed
the
feature/preview-live-share
branch
from
August 9, 2026 10:05
7cca372 to
9fb1ade
Compare
Host, --share, and --join all auto-opened the boot URL in the system default browser via open::that, with --no-browser as the only opt-out. --browser <name> routes through open::with instead (open -a on macOS), so users pick a browser per invocation. clap makes it conflict with --no-browser; a failed named-browser open warns and stays non-fatal, same as the default path.
…s off q2 preview --ui editor without --allow-edit is an ephemeral sandbox: edits sync live to everyone connected but are never written to disk. The only notice was one line printed at CLI startup — easy to miss while working in the editor, and lost edits were a silent surprise. The hub-client now fetches /api/preview/config once at boot (bd-ov4gqk3m) and, when allowEdit is false, shows a persistent banner under the editor header. The endpoint exists only on preview servers (a --join guest's TCP proxy splices through to the host, so guests see the banner too); a standalone hub 404s or answers with SPA-fallback HTML, both treated as "not a preview", so the banner never appears against a real hub.
…le pick, dedupe - --share no longer blocks the host's own preview on the tunnel's relay wait (up to ~10 s when no relay is reachable): the share task is gated on the server's on_ready via a watch channel instead of being awaited before the listener binds (bd-jhvkwosw). The join banner now prints after the boot URL in both UI modes, keeping the join line last on the terminal. A tunnel start failure degrades to a stderr warning rather than failing the preview, and Ctrl-C during the relay wait aborts the parked task instead of hanging exit. New integration tests pin the gate (no banner/tunnel before ready; quiet exit when the server dies first). - pick_editor_file prefers a root index.qmd over the sorted-first fallback (bd-jt1etjbn follow-up); verified end to end through `q2 preview --ui editor` (boot URL now carries file=index.qmd). - build.rs embed dedupe cheap-rejects on metadata/size before reading files — non-matches no longer slurp both copies (~76 MB of reads for the shared WASM). Embed output verified byte-identical pre/post. - Extract spawn_browser_open_when_ready shared by both UI modes, drop the dead suppress param from open_browser_or_log, and consolidate the duplicated share-route doc contract onto editor_share_route. - Recast the quarto-p2p tunnel-client example as a debugging driver now that `q2 preview --join` is the real guest. Verified: cargo xtask verify --skip-hub-build (all 14 steps); e2e `q2 preview --share` serves /health 0.4 s before the banner prints and exits cleanly on SIGINT.
The conflict matrix already lets guests pass --browser (covered by preview_browser_composes_with_join); the --join doc comment only named --no-browser.
The two share.rs tunnel tests were the only current-thread #[tokio::test] fns running an iroh endpoint; every other tunnel test (tunnel.rs, join_tunnel.rs) is multi_thread. On the ubuntu CI runner (slowest cores, fullest suite) the single runtime thread starved the endpoint actors until the 20s step caps were exceeded: share_glue_tunnels... failed at 100.9s, share_task_waits... at 63.5s. - multi_thread flavor for both share.rs tunnel tests (matches the suite convention; locally 19-23s -> 12-14s under in-crate load) - STEP_TIMEOUT 20s -> 60s in share.rs and p2p support.rs: iroh endpoint work inflates 5-10x under full-suite load (each bind pays netmon setup + handshake crypto on contended cores); the caps only bind when something is genuinely broken - join_tunnel convergence poll 15s -> 30s (same load-inflation class)
…r component A --join guest's boot fetch of /api/preview/config can race the tunnel's connect handshake; since the config is fetched once per boot, a dropped request hid the ephemeral-session banner for the whole session. Retry once on transport failure only — definitive answers (non-ok, HTML fallback, malformed body) stay single-shot so standalone hubs see no extra request. The banner moves from inline JSX in Editor (untestable in jsdom — Monaco imports) to a tiny EphemeralSessionBanner component with a render test pinning its copy, status role, and hover text.
Catches up the branch's hub-client changes (the --ui editor embed, ephemeral project-set boot, and the ephemeral-session banner) plus the config-fetch retry.
TunnelHostHandle::shutdown is graceful but not synchronous with the UDP socket's release (iroh gives endpoint tasks a grace window after close), so the immediate rebind raced the teardown and failed with EADDRINUSE on the loaded ubuntu CI runner. Poll the respawn until the port is free (10s deadline). A real host restart can't hit this — process exit releases the socket.
`q2 preview --join` prints "Received Ctrl-C, leaving the shared
session…", but the host side printed nothing — the hub's own shutdown
notice goes to tracing::info, hidden at the CLI's default filter. Arm
a small ctrl_c printer alongside the server (tokio allows multiple
listeners; the hub's handler still drives the graceful shutdown and
final filesystem sync), with a share-specific variant ("ending the
shared session…") when --share is active.
Covered by a unix subprocess test that boots the real binary, waits
for the port to accept, sends SIGINT, and asserts the line and the
clean exit.
shikokuchuo
marked this pull request as ready for review
August 9, 2026 15:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
q2 previewcan now share a running preview session with other people.q2 preview --share index.qmdprints a join string.q2 preview --join <string>and opens the printed link. They see the same preview, live.--ui editoropens the full hub-client editor instead of the viewer, for host and guests alike.--browser <name>opens the session in a specific browser instead of the system default — works for host,--share, and--joinalike (conflicts with--no-browser).Who can do what
--allow-edit. The share banner says exactly what the join string grants.--ui editorwithout--allow-editallows "ephemeral" editing: edits sync live to everyone in the session but are never written to disk. The editor shows a persistent "Ephemeral session — edits won't be saved to disk" banner in this mode (driven by/api/preview/config; guests see it too), so session-only edits can't be missed.How it works
quarto-p2pcrate: share tickets, a tunnel host in front of the preview server's loopback port, and a tunnel client that serves a local proxy for the guest. All traffic goes over one encrypted iroh connection; if no relay is reachable, the banner says guests can join over LAN/direct connections only.--joinis a thin local proxy — the host's preview server serves everything through the tunnel, so host and guests share one Automerge document set./api/preview/config, so guests land in the same document instead of a setup screen.Fixes included
--allow-edit, the hub's 5-second disk sync deleted any edit older than the previous sync tick, on host and guests alike. The sync now skips its disk-merge step when the file on disk has not changed. Regression test included.fork_atfix and contain collector panics so a bad doc cannot take down the server.Testing
build:all, unit (881), and WASM changelog-render gate all green.--joinhelp text mentions--browser.share.rstunnel tests sometimes exceeded their 20s timeouts on the loaded ubuntu runner. They now run multi-threaded like every other tunnel test, with 60s caps.