feat(carve S4): Linear issue-context surface — attachments, PDF/image screening, auth health - #655
feat(carve S4): Linear issue-context surface — attachments, PDF/image screening, auth health#655isadeks wants to merge 3 commits into
Conversation
eb91f42 to
30aa161
Compare
9e2dda7 to
4f947ee
Compare
30aa161 to
0df8c2c
Compare
4f947ee to
77677a0
Compare
0df8c2c to
c58d51b
Compare
Automated review — carve S4 (#655)Reviewed as its own diff only ( Slice standalone-ness: PASS
Security posture — the parts that are rightWorth recording, since this is the slice where it matters most:
1. MAJOR — the widened markdown regex is quadratic; a large issue description exhausts the webhook timeout
MARKDOWN_LINK_OR_IMAGE_PATTERN = /!?\[([^\]]*)\]\(<?(https:\/\/[^)>]+)>?\)/gWith
Growth is quadratic rather than exponential — so I would not call it catastrophic backtracking — but the practical outcome is the same: a ~150 KB description blows a 30 s webhook-processor timeout, and an attacker (or an unlucky user pasting a large bracketed table) needs no special crafting.
Cheapest fixes: cap description length before scanning, or make the 2. MAJOR —
|
77677a0 to
fcc18a3
Compare
c58d51b to
efd1765
Compare
fcc18a3 to
4b4a3d6
Compare
efd1765 to
430576f
Compare
4b4a3d6 to
868197c
Compare
430576f to
227058d
Compare
Review findings addressedI had missed this review initially — apologies. Two of these were real bugs that would have shipped. MAJOR #1 — quadratic markdown regex. Confirmed by measurement and fixed. I reproduced your numbers before touching anything (4 KB ≈ 7 ms, 12 KB ≈ 56 ms, 50 KB ≈ 940 ms), and confirmed your two supporting points: the scan cap does not bound it because the backtracking happens inside a single MAJOR #2 — Both fixes were mutation-verified — reverting either one fails its own test and nothing else. MAJOR #3 — MAJOR #4 (pdf-parse bundling), #5 (private review citations), #6 (context probe untested), #7 (dynamic import types as Gates: 2545 CDK tests, |
63ae670 to
fd0834a
Compare
2a8e7b2 to
dadf00f
Compare
Second-round review: my regex fix was wrong in both directionsA second adversarial pass found the label bound both introduced a worse failure and failed to fix the one it targeted. Both verified by measurement. It silently dropped attachments. A 300-char label bound makes a link with longer alt text stop matching entirely — the file vanishes, nothing is logged, and the agent plans against incomplete material. Driven through the real entry point, a 350-char label went from 1 attachment to 0. That is the same silent-data-loss failure this slice's other fix exists to prevent, and I introduced it while fixing a slow scan. Trading a DoS for lost user data is a bad trade. The bound is now 4096 — far past any real label — with a test pinning that a ~780-char descriptive label still yields its attachment. And it did not fix the slow scan. BOTH variable parts of the pattern backtrack per start position, and the URL part dominates. On
Identical. My new test only asserted The real bound is on the input: a description over 64 KB has its tail left unscanned, and the truncation is logged rather than passed over silently. That fixes every hostile shape at once and is honest about the trade. Both mutation-verified: restoring the tight bound fails the long-label test; removing the cap fails the URL-shape test (4.9 s against a bounded budget). Two timing assertions replaced with assertions on observable work. Both passed alone and failed under full-suite load — a wall-clock budget inside a parallel suite measures CPU contention as much as the code, so those would have been flaky gates rather than guards. They now assert the truncation warning fired, which also proves the cap engaged and that it is announced. On the revocation marker being dead code — the reviewer is right that making it opt-in leaves Verified as still-correct: |
… screening, auth health Bring the Linear surface's issue-context and operator tooling onto main. All of this is reachable today through the existing Linear webhook path; none of it depends on the sub-issue orchestration arc, so it stands alone. - linear-attachments.ts / linear-issue-context-probe.ts: pull an issue's file attachments and recent comments into the task context, so the agent sees the screenshots and discussion a human would. - attachment-screening.ts: screen fetched attachments (images and PDFs) before they reach the model. This carries the pdf-parse v2 upgrade as one unit — the source uses the v2 `PDFParse` class, which requires pinning the dependency and dropping the hand-written module declaration that described the v1 function export. Splitting any of the three breaks the build, so they travel together. - error-classifier.ts: distinguish transient compute faults from user-visible build failures, and report whether work was preserved when a run does not produce a pull request. - screenshot-url.ts + github-screenshot-integration.ts: resolve screenshot URLs for the review path. - CLI: `linear-auth-health.ts` + `platform-doctor.ts` surface an expired or revoked Linear authorization as a diagnosable condition instead of silently dropped events; `linear.ts` / `platform.ts` expose it. The comments and test names in these files are rewritten to explain the what and why directly rather than pointing at internal tracker ids, and two CLI strings that printed an internal issue number to user-facing output are reworded. Stacked on the agent-runtime slice. Full CDK suite 2598 passing, CLI 675 passing, both type-checks clean.
…pping an inert marker **A quadratic regex could blow the webhook timeout.** Making the leading `!` optional in the markdown-link pattern stopped the engine anchoring each attempt on a literal `!`, so the unbounded label quantifier retried from every `[`. Measured on a description of unmatched brackets: 4 KB ≈ 7 ms, 12 KB ≈ 56 ms, 50 KB ≈ 940 ms — and around 150 KB exceeds the processor's 30 s timeout. No crafting is needed; a large pasted table does it, and the existing scan cap does not bound it because the backtracking happens inside a single `exec()` before any match is produced. The label quantifier is now bounded, which caps the work per start position while still matching every real markdown label. **Two attachments whose names differ only by `.` versus `-` silently became one.** The upload id collapsed `.`, `-` and `/` onto the same character and then squashed runs, so `design.v1.png` and `design-v1.png` produced an identical id — and both de-dupe sites resolve a collision by discarding the second file. A user who attached both got one, with nothing logged and no warning, and the agent planned against incomplete material. The id now carries a short digest of the full pathname, keeping the readable stem for logs. Both fixes are pinned by tests that were mutation-verified: reverting either change fails its own test and nothing else. **The revoked-authorization marker was on by default and could never succeed.** Every Lambda that resolves a token holds read-only registry access, and no slice in this arc grants write, so the marker's write failed AccessDenied on every revoked refresh and the failure was swallowed. The feature read as working while being permanently inert, which is worse than being visibly absent. It is now opt-in: a caller that holds the grant passes a recorder explicitly, and the default is to attempt nothing. When the grant lands, flip the default in that same change.
…ound lost attachments A second review pass found my markdown-regex fix was wrong in both directions. **It silently dropped attachments.** A 300-char label bound makes a link with longer alt text stop matching entirely, so the file vanishes with nothing logged and the agent plans against incomplete material. That is the failure this slice argues elsewhere is the worst kind, and I introduced it while fixing a slow scan. The bound is now 4096 — far past any real label — and a test pins that a ~780-char descriptive label still yields its attachment. **And it did not fix the slow scan.** BOTH variable parts of the pattern backtrack per start position, and the URL part dominates: on `[](https://a` repeated, a 100 KB description takes ~820 ms with or without a label bound, and larger inputs still exceed the webhook timeout. Bounding the label only fixed the one shape the new test happened to assert. The real bound is on the input: descriptions longer than 64 KB have their tail left unscanned, and the truncation is LOGGED rather than passed over in silence. That fixes every hostile shape at once. Both are mutation-verified — restoring the tight bound fails the long-label test, removing the cap fails the URL-shape test. Two timing assertions are replaced with assertions on the observable work (the truncation warning). A wall-clock budget inside a parallel suite measures CPU contention as much as the code: both passed alone and failed under full-suite load, which would have been a flaky gate rather than a guard.
fd0834a to
4367dc5
Compare
dadf00f to
1a3f6bf
Compare
scottschreckengaust
left a comment
There was a problem hiding this comment.
Verdict: Approve with nits
Approve with nits. S4 lands the Linear issue-context surface (attachment fetch + screening, context probe, error classification, auth-health diagnostics) at a genuinely high engineering bar — fail-closed everywhere it matters, mutation-verified regression tests, and IAM grants scoped to the exact action×ARN pair each read needs. I found no blocking issue in the incremental diff. The items below are non-blocking.
Reviewed as a stacked PR: diff scoped to base carve/s3-linear-nomcp-attachments, not main. I did not re-flag anything introduced by S1–S3.
Governance
- Backing issues #247 and #299 both carry
approved(P0). Tracking epic #668 documents the 8-slice carve and why S4 stands alone. Gate satisfied per ADR-003. - Branch
carve/s4-linear-surface— the carve naming is a de-facto-waived deviation from(feat|fix|…)/<issue>-…, not a blocker. - CI is green on all four checks (build agentcore, secrets/deps scan, dead-code, PR title).
Vision alignment
Fits the fire-and-forget default; escalate by policy and bounded blast radius tenets. The whole slice is about turning previously-silent failure modes (dropped events on a dead Linear grant, unscreened attachment bytes, mis-classified infra faults reading as user errors) into diagnosable, attributable, fail-closed conditions. linear-attachments.ts fetches untrusted uploads through the same Bedrock Guardrail pipeline as every other attachment, with SSRF guards and a fail-closed contract — it widens the input surface but bounds it. No tenet is traded; ADR-016 (deterministic Linear, no MCP) is the cited basis for admission-time hydration.
Blocking issues
None.
Non-blocking suggestions / nits
-
isPrivateIpmisses hex-form IPv4-mapped IPv6 (cdk/src/handlers/shared/linear-attachments.ts, ~L520-540). The mapped-address regex/(?:::ffff:|::)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})$/only catches the dotted-quad textual form.dns.lookupcan return the metadata endpoint as the hextet form::ffff:a9fe:a9fe(169.254.169.254), which this regex does not match and the fe80–febf link-local check does not cover (its first hextet is0000). This is defense-in-depth only — the host is allowlisted touploads.linear.app, so reaching it requires Linear's own DNS to resolve internal, andredirect: 'error'closes the redirect vector — but if the SSRF guard is meant to be authoritative, normalize v4-mapped v6 by parsing the low 32 bits rather than string-matching the dotted form. -
looksLikeBundlingBugregex includes a bare/pdfjs/ialternative (cdk/src/handlers/shared/attachment-screening.ts, ~L335). A genuinely corrupt PDF whose pdfjs parse error message merely contains the substring "pdfjs" would emit the LOUD "almost certainly a BUNDLING bug" operator diagnostic. Log-only, user-facing message stays generic, so impact is a possibly-misleading operator log line — consider tightening to the DOM-global / native-binding signatures (DOMMatrix|ImageData|Path2D|napi-rs\/canvas|Cannot find native binding) and dropping the barepdfjstoken. -
Dormant IAM grant references a not-yet-existing GSI. The new
dynamodb:Querygrant on${taskTable}/index/LinearIssueIndex(github-screenshot-integration.ts) is gated onprops.taskTable, whichstacks/agent.tsdoes not pass yet, andLinearIssueIndexis not on the realTaskTable(onlyJIRA_ISSUE_INDEXexists today). This is consistent with the carve's dormant-until-S8 strategy and the construct test builds its own table with the index, so it compiles and tests pass — just flagging that the grant is inert onmainuntil the activation slice adds both the prop wiring and the GSI. Confirm #668's coverage check catches the GSI addition in its owning slice. -
pdf-parsepinned to exact2.4.5(no caret) incdk/package.json. Deliberate per the PR description (v2 API coupling), and yarn.lock agrees. Fine, but a pinned transitive-heavy dep like pdfjs will silently miss patch security fixes — worth a renovate/dependabot note so it doesn't rot.
Documentation
- ADR-016 is the cited basis and already exists; no ADR needed for this additive slice.
- New user-facing CLI surface —
--verify-refresh,--decompose-allowed,--max-sub-issues,--max-parent-budget-usd,--no-force-consent, and the newlinear_workspace_authdoctor check — is not reflected indocs/guides/USER_GUIDE.md. Non-blocking for a dormant-arc slice (decompose flags don't do anything until S6/S8), but the auth-health doctor check and--verify-refreshARE live through the existing webhook path and would benefit from a USER_GUIDE mention. No Starlight mirror edits are needed since nodocs/guidessource changed in this PR.
Tests & CI
Strong. New/changed suites: linear-attachments.test.ts (+534), linear-auth-health.test.ts (+216), platform-doctor.test.ts (+128), plus attachment-screening, error-classifier, validation, screenshot-url, oauth-resolver, and the construct IAM-grant regression tests. Coverage hits the failure paths, not just happy paths: fail-closed on over-page/over-byte PDFs, unsupported types, magic-byte mismatch, SSRF, mid-batch S3 cleanup, revoked-marker installation-scoping race, refresh-persist-or-report-error. The two timing-based ReDoS assertions were correctly replaced with assertions on observable work (the truncation warning) to avoid a flaky wall-clock gate under parallel load. I independently reproduced the ReDoS bound: at the 64 KB SCAN_MAX_DESCRIPTION_CHARS cap the worst-case URL-shape scan is ~630 ms and bracket-flood ~1080 ms — comfortably under the 30 s webhook timeout.
Bootstrap synth-coverage: N/A. The diff adds only dynamodb:Query/GetItem IAM policy statements to an existing Lambda execution role — no new CloudFormation resource types (no new bucket/queue/secret/service). No cdk/src/bootstrap/* update required.
Test-perf (CDK synth): the new construct suite uses a single beforeAll App/Template.fromStack and does not re-enable bundling — compliant with #366.
Review agents run
- code-reviewer / correctness (
/code-reviewhigh, base-scoped): ran over the full incremental diff (8 finder angles + verify). No CONFIRMED correctness bugs survived; the SSRF hex-mapped gap and thepdfjslog-regex breadth surfaced as low-severity/defense-in-depth and are folded in as nits 1–2 above. - silent-failure-hunter (manual, error-handling scope): the diff is heavy on error handling; I traced every catch. Swallowed failures are all intentional and either logged loudly or converted to fail-closed rejections (S3 cleanup best-effort with 90-day lifecycle backstop;
onAuthorizationRevokedbest-effort so a marker write can't break token resolution; probeok:false→ attachment hydration fails CLOSED). No hidden-default failures found. - type-design-analyzer (manual, new types):
LinearAuthState/ProbeResult/RefreshVerifyResultdiscriminated unions and theSelectedUpload/LinearProbeAttachmentshapes are tight and make illegal states unrepresentable (e.g.expired_indeterminateis deliberately un-collapsible to healthy). No sharedhandlers/shared/types.tsAPI shape changed, so nocli/src/types.tssync needed. - comment-analyzer (manual): comments are unusually thorough and accurate against the code; the ABCA-### tracker-id scrubbing (→ "observed in practice") matches the diff. No misleading comments.
- pr-test-analyzer (manual): covered under Tests & CI — coverage is good, mutation-verified.
- /security-review: ran mentally over the IAM (scoped Query+GetItem, least-privilege), SSRF guard (DNS pre-resolve + private-IP reject +
redirect: 'error'), secrets (tokens read narrowly, never returned; rotated refresh token persisted-or-report-error), and input-gateway (fail-closed screening, magic-bytes authoritative over attacker-controlled label, bounded scan) changes. One residual defense-in-depth gap (nit 1); no blocking security issue. - Omitted: no dedicated agent for pure-docs or infra-only scope was needed beyond the above.
Human heuristics
- Proportionality — pass. Complexity matches the problem. The revoked-marker opt-in-until-grant-lands design and the probe
ok:falsefail-closed are proportionate responses to real observed outages, not speculative abstraction.linear-attachments.tsat 743 lines is essential surface (fetch/SSRF/magic-bytes/screen/upload/cleanup), not accreted. - Coherence — pass.
linear-attachments.tsis a deliberate, documented analog ofjira-attachments.ts(same select→fetch→screen→upload→record shape), reusing sharedattachment-screening/validationhelpers rather than re-implementing. Same concepts, same terms. - Clarity — pass. Names communicate intent; error handling surfaces failures loudly rather than hiding them behind plausible defaults. The
expired_indeterminatestate is a model of refusing to fake certainty. - Appropriateness — concern (minor). Integration is verified against a class-level
PDFParsemock, not the real pdfjs worker (the code comments are candid that ts-jest can't drive the ESM worker and real extraction is validated on live deploy). This is the AI001 pattern — mocks matching self-written expectations — mitigated by the honest disclosure and the live-deploy verification plan, but the v2 extraction contract is not exercised end-to-end in CI. Acceptable given the constraint; worth a live-deploy smoke check before S8 activation.
Stacked on #654 → #653 → #647. Review those first; this PR's diff is only its own slice.
What this adds
The Linear surface's issue-context gathering and operator tooling. None of it depends on the sub-issue orchestration arc — it's reachable today through the existing Linear webhook path.
Issue context.
linear-attachments.tsandlinear-issue-context-probe.tspull an issue's file attachments and recent comments into the task context, so the agent sees the screenshots and discussion a human reading the ticket would.Attachment screening.
attachment-screening.tsscreens fetched images and PDFs before they reach the model.Error classification.
error-classifier.tsseparates transient compute faults from genuine build failures, and reports whether the work was preserved when a run doesn't produce a pull request — so a user gets "the branch has your work, the PR just didn't open" instead of a generic failure.Auth health. An expired or revoked Linear authorization previously meant events were silently dropped.
linear-auth-health.ts+platform-doctor.tsmake it a diagnosable condition, surfaced through the CLI.One coupling worth flagging
The PDF path upgrades
pdf-parseto v2, and that's three inseparable changes: the source uses the v2PDFParseclass, which requires pinning the dependency and deleting the hand-written module declaration that described the v1 function export. Any two without the third fails to compile. They're together in this PR for that reason.The bundling-contract test that guards the dependency being externalized from Lambda bundles asserts on the integration constructs, so it lands with those in a later slice.
Verification
Full CDK suite 2598 passing, CLI 675 passing, both type-checks clean. Comments and test names explain their reasoning directly rather than citing internal tracker ids, and two CLI strings that printed an internal issue number to user-facing output are reworded.
Tracking
Slice S4 of the linear-vercel → main carve. Tracking issue: #668 (slice table, why it is sliced rather than merged, and review guidance).
Lands part of #247 (parent/sub-issue orchestration) and #299 (auto-decomposition). Deliberately not
Closes— no single slice closes either; they close when S8 activates the arc.Stacked on #654 — review that first; this PR's diff is against its branch.