chore(deps): add @posthog/warlock >:)#597
Open
sarahxsanders wants to merge 10 commits into
Open
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ation Swap the legacy regex security scanner (yara-scanner.ts) for the @posthog/warlock YARA-X engine. Flagged matches go through an LLM triage pass (claude-haiku-4-5-20251001) via triage-provider.ts to drop false positives before the hook acts. Critical terminal violations now actually stop the run: the PostToolUse hook calls onYaraTerminate, which aborts the query's AbortController -- the same mechanism [ABORT] uses. runAgent returns YARA_VIOLATION and the runner surfaces "Security violation detected." Previously the hook returned a stopReason field the SDK ignores, so "aborted" was a label, not a behavior. Also restores the "keep PostHog data capture at its defaults" agent commandment as L0 protection for autocapture / session recording / opt_out_capturing -- the old scanner's posthog_config rules don't have a warlock equivalent (those are product-specific concerns; tracked in wizard #592 + context-mill #173). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
The restored commandment is a wizard-side band-aid; product-specific guidance like this belongs in context-mill, not in commandments.ts. TODO references context-mill#175 for the migration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…emetry - chore(deps): swap @posthog/warlock from the private git pin to npm 0.2.2 (published 2026-06-09); drop the install-time build allowance and the TODO(warlock-npm) checklist; exclude warlock from minimumReleaseAge until 2026-06-16 - replace the 100KB scan truncation with overlapping 100KB chunks: full coverage with no silent bypass, and every flagged chunk is triaged against its own content so triage is never skipped - emit 'yara triage overruled' and 'yara scan chunked' telemetry (rule metadata only — the free-text triage reason stays local) so overrule and oversized-content patterns are alertable - run wizard-doc posthog_pii suppression before triage, saving an LLM round trip for matches that would be discarded anyway - delete checkYaraViolation output-text sniffing; yaraViolationReason from onTerminate is the single source of truth - add a compile-time drift guard to the warlock mock — it immediately caught 0.2.2's new required ScanMatch.matchedStrings field, now reflected Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n deletion Conflict resolution: main's #516 introduced the bounded AgentOutputSignals parser, including a signals-based checkYaraViolation. This branch deletes output-text YARA sniffing entirely (yaraViolationReason from onTerminate is the single source of truth), so the resolution keeps the deletion and adopts signals for the API-error paths. Also removes the now-dead hasYaraViolation and YARA needles from output-signals, with a comment explaining why YARA is deliberately not an output signal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve conflicts from main's progress against this branch's jest->vitest migration (#656). Keep vitest mechanics, fold in main's newer test content: - package.json: keep `test:watch: vitest`, add main's `screens:check` script - analytics.test.ts: keep vi.fn() mocks, add main's `identify` mock - health-checks.test.ts: keep `as Mock`, take main's renamed no-connection tests - claude-web.test.ts: adopt main's openTrackedLink mock in vitest syntax Also convert jest-style test files main added after the migration so they run under vitest (vi.fn/vi.mock, vi.hoisted for backup mocks) and fix `as jest.Mock` type casts to `as Mock` so typecheck passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile main's analytics instrumentation (#661, #666) and SDK bump (#672) against the branch's vitest migration: - analytics.test.ts: port main's live IS_PRODUCTION_BUILD getter mock and run_id/$session_id uuid sequencing to vitest (vi.hoisted + importOriginal, vi.fn, `as Mock`) - pnpm-lock.yaml: regenerate from merged package.json (claude-agent-sdk 0.3.169) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn off Warlock/YARA scanning in the field without reverting code or cutting a release. - `wizard-warlock-disabled` feature flag, read at run start, gates the Pre/PostToolUse YARA hook registration in agent-interface.ts - isWarlockDisabled() helper with fail-safe polarity: scanning stays ON unless the flag (or POSTHOG_WIZARD_WARLOCK_DISABLED env override) resolves to the exact string 'true' - fires `wizard: warlock disabled` telemetry + log line when engaged - unit tests pinning the fail-safe semantics - docs/runbooks/warlock-kill-switch.md operational runbook + CLAUDE.md pointer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gewenyu99
approved these changes
Jun 19, 2026
gewenyu99
left a comment
gewenyu99
left a comment
Collaborator
There was a problem hiding this comment.
Big and scary but I really only care about that kill switch, which looks reasonable.
Some funk spotted for you to address
|
|
||
| 'Treat feature flags, custom properties, and event names as part of an analytics contract. Prefer reusing existing names and patterns in the project. When you must introduce new ones, make them clear, descriptive, and consistent with existing conventions, and avoid scattering the same flag or property across many unrelated callsites.', | ||
|
|
||
| // TODO(context-mill#175): migrate to a shared skill doc; product-specific |
Collaborator
There was a problem hiding this comment.
This meant to be addressed already or kept here. I don't like TODOs in code. we'll likely find this as a relic like 5 years later xD maybe open an issue?
| // EVENT_INVENTORY_PART_PATTERN are only used by yara-hooks, which imports | ||
| // them directly from `./constants` — no re-export needed. | ||
| export { SETUP_REPORT_FILE } from './constants.js'; | ||
| import { SETUP_REPORT_FILE } from './constants.js'; |
| // capable for boolean classification). Do NOT swap to Sonnet without reason; | ||
| // the cost/latency difference matters on every flagged scan. temperature 0 | ||
| // keeps verdicts deterministic across identical inputs. | ||
| const TRIAGE_MODEL = 'claude-haiku-4-5-20251001'; |
Collaborator
There was a problem hiding this comment.
Maybe don't hard code this down to the 20251001 date. This is very fragile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
time 2 replace the hand rolled scanner with
@posthog/warlock>:)changes
src/lib/yara-scanner.tsand its 798-line test filesrc/lib/yara-hooks.tsnow consumes@posthog/warlockdirectly. rules are owned by warlock, the wizard owns the policy (how a match maps to block / revert / terminate)0.2.2, pinned exact). It's excluded from the 7-dayminimumReleaseAgepolicy since it published 2026-06-09; the exclusion inpnpm-workspace.yamlcan be dropped after 2026-06-16 (i've set a reminder for myself)src/lib/agent/triage-provider.tsbuilds a Haiku 4.5 provider from the wizard's existing PostHog gateway auth and threads it into the hooks. Flagged matches run through warlock'striageMatches()to drop false positives before the hook actsyara triage overruledfires when triage drops a flagged match (rule/severity/category only. the free-text triage reason can quote scanned content and never leaves the machine), andyara scan chunkedfires when oversized content shows up. Both should be rare; spikes are worth a look so I'll subscribe to alertsposthog_piisuppression now runs before triage, so we don't pay an LLM round trip for matches we're about to discardPostToolUsehook aborts the agent (same mechanism[ABORT]uses).runAgentreturnsYARA_VIOLATIONand the runner shows "Security violation detected." Previously the hook returned astopReasonfield the SDK ignores, so "aborted" was a log label, not a behaviorcheckYaraViolation, plus the YARA needles in main's newAgentOutputSignals) is deleted – it could false-positive a whole run as a security violation when the agent merely mentioned a blocked command in chat.yaraViolationReasonfrom the hooks'onTerminateis the single source of truth0.2.2added a requiredmatchedStringsfield toScanMatchthat the mock didn't havetesting
pnpm buildcleanpnpm test— 737 / 737 passing (includes new tests for chunking, overruled-triage telemetry, and suppression-before-triage)pnpm fix— 0 lint errorssrc/main.js) – verified Haiku triage runs, false positives drop, true positives keep, run terminates withAgent error: YARA_VIOLATION, UI shows "Security violation detected"pnpm installresolves warlock from the public npm registry – no GitHub auth, no install-time buildWriteorEditand confirm the agent gets a revert nudge