Skip to content

docs(designs): forge-poll driver design record (SEA-1810) - #219

Open
seal-agent wants to merge 5 commits into
mainfrom
compass-server-1810-forge-poll-driver-design
Open

docs(designs): forge-poll driver design record (SEA-1810)#219
seal-agent wants to merge 5 commits into
mainfrom
compass-server-1810-forge-poll-driver-design

Conversation

@seal-agent

@seal-agent seal-agent commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Design record for the compass-server forge-poll driver: a ctx-cancellable board-ingestion poll loop that drives the existing internal/ingest library against a real GitHub forge so tracker issues appear live in the server projection, built ON the DL-053 forge-subscription machinery (Matt ruled OQ-A = Option 2).

Merge-ready — Status: Active, all load-bearing Open Questions resolved, ledger delta in this PR.

Four tasks:

  • T1 — a hand-rolled net/http GitHub read client (OQ-6 frozen: no new dep): ListIssuesPage/HasNext over RFC-5988 Link, If-None-Match/304 conditional requests, per_page=100, a 403 rate-vs-dead-token discriminator (retry-after/x-ratelimit-remaining), and an x-ratelimit-* budget gate that fails fast rather than sleeping.
  • T2 — migration 0015_forge_subscriptions.sql (four tables, DDL below) + store/forge_cursors.go list-cursor + repo-subscription methods + store.ForgeProviderLinear = 4; pgtest-covered.
  • T3 — ingest.Driver: the DL-053 conditional-poll driver, per-pass target enumeration via PollStore.ListEnabledRepos, sink-gated per-page cursor advance, nil on ctx cancel.
  • T4 — serve boot wiring behind an all-optional ForgeConfig (SeedRepos + Poll), the boot seed reconcile, the DL-052 server_only token behind a short TTL, driver under the existing serve errgroup.

Resolved Open Questions (Matt)

  • OQ-A = Option 2 (2026-08-07): build on the DL-053 machinery from day one; the board is a distinguished subscriber.
  • OQ-C = TABLE (2026-08-08): the board's poll targets are forge_repo_subscriptions rows (dynamically add/removable), not config; DL-053's forge_subscriptions lands RENAMED agent_forge_subscriptions.
  • OQ-D1 = land now / OQ-D2 = keep Linear (2026-08-08): all four forge tables land in 0015, every provider CHECK admits the full declared enum IN (1, 2, 3, 4).
  • OQ-B (live-demo content) and OQ-E (the Linear-ingestion issues-CHECK prerequisite) are explicit non-load-bearing deferrals.

Red-team (design-critic)

Second red-team over the table/seed re-cut: sound-with-folds, no high, no needs-rework, no Option-1 vestige. All 10 findings folded (F1 seed ON CONFLICT DO NOTHING; F2 polling-disabled boot Warn; F6 CursorStore → PollStore; F3–F5, F7–F10). Two carry a small author-resolved call flagged here for your ratification at review, rather than a separate round-trip:

  • F1 — seed conflict semantic = ON CONFLICT DO NOTHING (bootstrap-only insert; the table is authoritative after first insert; a soft-disabled row stays disabled across restarts). Rejects DO UPDATE SET enabled = TRUE, which would silently re-enable an incident-disabled repo on a routine restart. Weighed in Alternatives (h); flagged in OQ-C.
  • F7 — the DL-053 rename's ownership-layer annotation. DL-163 records the rename in the ledger (this PR). The proposed one-line forward-annotation at the frozen ownership-layer DDL (compass-server-ownership-layer/design.md:983) is not applied here — that record is frozen Active, and I don't rewrite a frozen record without your call. Say the word and it lands as its own delta.

Ledger delta (in this PR)

  • DL-161 — the poll driver + forge_list_cursors FETCH-cursor model.
  • DL-162 — repo targets as forge_repo_subscriptions table rows; --forge-repos becomes a declarative boot seed.
  • DL-163 — 0015 lands four tables unconditionally; provider CHECK IN (1, 2, 3, 4).

Ledger-impact: DL-161, DL-162, DL-163 appended (Storage section).

Refs SEA-1810

Co-authored-by: Matt Wilkinson matt@sealedsecurity.com

@linear-code

linear-code Bot commented Aug 7, 2026

Copy link
Copy Markdown

SEA-1810

@seal-agent
seal-agent force-pushed the compass-server-1810-forge-poll-driver-design branch 2 times, most recently from 50bc8d7 to d9e3860 Compare August 7, 2026 20:00
@seal-agent seal-agent closed this Aug 7, 2026
@seal-agent seal-agent reopened this Aug 7, 2026
seal-agent and others added 5 commits August 7, 2026 16:41
@seal-agent @mattwilkinsonn
docs(designs): forge-poll driver design record (SEA-1810)
64e1cd3 
Design record for the compass-server forge-poll driver: a ctx-cancellable board-ingestion poll loop that drives the existing `internal/ingest` library against a real GitHub forge so tracker issues appear live in the server projection.

Three tasks:

- **T1** — a hand-rolled `net/http` GitHub read client (OQ-6 frozen: no new dep), with `sort=updated&since` incremental fetch, `If-None-Match`/304 conditional requests over a per-URL response+`Link`-chain cache, and an `x-ratelimit-*` budget gate that fails fast rather than sleeping.
- **T2** — an `ingest.Poller` scheduler: immediate first pass then ticker, sequential per-repo, log-and-continue on error, `nil` on ctx cancel.
- **T3** — serve boot wiring behind an all-optional `ForgeConfig`, the token resolved (behind a short TTL) via the DL-052 `server_only` secret path, poller run under the existing serve errgroup.

Design layer only — no implementation ships here. Red-teamed by design-critic; its findings are folded (the `since`/ETag-efficacy fix, 304 page-chain caching, resolve-cost TTL, the added test cases) and the PR-C double-fetch consequence + a durable-cursor Option 1.5 are promoted into OQ-A.

Freeze is blocked on **OQ-A** — the relationship to the frozen DL-053 subscription model (distinct board-poll loop vs building on the DL-053 machinery). Parked for Matt; OQ-B (live-demo content) is a non-load-bearing deferral.

Ledger-impact: none in this PR — Draft record; the DECISIONS.md ledger delta lands at freeze, once Matt rules OQ-A and the record goes Active.

Refs SEA-1810

Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
@seal-agent @mattwilkinsonn
docs(designs): fold forge-poll review findings — restore idempotency …
ce0b76d 
…invariant, 403 discriminator, no-clobber test

Review (skill://review, 1 round, 1 high + 2 medium + 2 low) folded:

- HIGH: the since-watermark added in the design-critic pass advanced on
  fetch success, but Ingester.Ingest sinks-then-stops (ingest.go:60-62) and
  forge.Issue carries no timestamp (provider.go:37-53), so an in-client
  watermark could not gate on sink outcome — a mid-pass sink failure stranded
  every issue below it, voiding the re-poll idempotency invariant. v1 now uses
  ETag/304 only; a correct durable incremental cursor is homed in OQ-A
  Option 1.5. Idempotency stated as a load-bearing Global Constraint.
- MEDIUM: single-rule 403 disambiguation (rate-limit vs bad-creds vs
  permission) stated explicitly in Approach step 3, reconciled with step 7.
- MEDIUM: T3 no-clobber pgtest respecified to set a human State baseline via
  store.SetIssueState first, so it can go red (was vacuous over a zero row).
- LOW: trimmed the DL-129 reopen over-claim; fixed garbled poller prose.

Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
@seal-agent @mattwilkinsonn
docs(designs): re-cut forge-poll driver on DL-053 (Option 2) + fold r…
cbe9b26 
…ed-team

Matt ruled OQ-A = Option 2: build the forge-poll driver ON the DL-053
forge-subscription machinery from day one, board as a distinguished
subscriber. Re-cut T1-T4 around a durable sink-gated forge_list_cursors
fetch cursor; DL-053/DL-129 added as frozen constraints.

Folds the design-critic red-team (0 high / 6 medium / 3 low): M1 boundary
probe for the stored-has_next walk gap; M3 forge.Issue.UpdatedAt widened at
T1 for the DL-129 recency guard; M4 TokenSource invalidation seam; M6 OQ-C
owns the LIST-cursor class; L1 multi-replica precondition clause; L2
advanced_at rename on the list cursor; L3 dual startup error text. OQ-D
reframed to two sub-forks (M2+M5): land the two spec-d DL-053 tables now vs
defer, and the provider-domain CHECK.

Refs SEA-1810.

Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
@seal-agent @mattwilkinsonn
docs(designs): fold forge-poll red-team + freeze (Status: Active + DL…
90bda02 
…-161/162/163)

Fold the 10 design-critic findings into the SEA-1810 forge-poll driver
record (sound-with-folds verdict) and flip it to Active with its ledger
delta, making the design PR merge-ready:

- F1: seed reconcile ON CONFLICT DO NOTHING (bootstrap-only insert; the
  table is authoritative after first insert; a soft-disabled row stays
  disabled across restarts), DO-UPDATE weighed + rejected in Alternatives.
- F2: one boot slog.Warn when polling is disabled but enabled target rows
  exist for the bound (provider, host); never fail-fast.
- F3: justify the deferred issues-CHECK widening (OQ-E) vs the Go
  ForgeProviderLinear constant closed now.
- F4/F5: GitHub repo-string lowercase normalization at the seed boundary;
  seed rows keyed under --forge-host (host change abandons prior rows).
- F6: rename the driver store interface CursorStore -> PollStore.
- F7/F8/F10: freeze-delta names both ledger + ownership annotation;
  forge_artifact_cursors GC deferred with its writers; forge_repo_subscriptions
  gains updated_at touched on every upsert/enable-flip.
- F9: T4 tests assert the seed-visible-before-first-pass ordering directly.

Ledger: append DL-161 (the poll driver + forge_list_cursors FETCH-cursor),
DL-162 (repo targets as forge_repo_subscriptions table rows, --forge-repos
becomes a boot seed), DL-163 (0015 lands four tables, CHECK IN (1,2,3,4)).

Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
@seal-agent @mattwilkinsonn
docs(designs): fold PR #219 review — accurate ledger-state prose, def…
36a3fc4 
…er DL-053 annotation (SEA-1883), citation drifts, empty-probe row

Review round (design-critic + review agent) surfaced 2 medium + 2 low, all
documentation/ledger-cross-reference; the poll/cursor/seed/budget model passed
every soundness lens.

- M1: correct the stale 'Draft'/'Ledger-impact: none' prose — the record is
  Active and its ledger delta (DL-161/162/163) rides this PR.
- M2: the DL-053 ownership-layer forward-annotation is deferred to SEA-1883
  (editing a frozen record needs Matt's call); stated explicitly in OQ-C, and
  the mis-cited target line :983 corrected to :978.
- L1: DECISIONS.md citations drifted +3 by this PR's own DL rows — :173->:176
  (DL-129), :126->:129 (DL-051).
- L2: an empty boundary-probe page writes no cursor row (skip the upsert on a
  zero-issue page), so the probe never persists an unprunable empty tail row.

Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>
@seal-agent
seal-agent force-pushed the compass-server-1810-forge-poll-driver-design branch from d9e3860 to 36a3fc4 Compare August 7, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattwilkinsonn mattwilkinsonn Awaiting requested review from mattwilkinsonn mattwilkinsonn is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@seal-agent