Skip to content

Commit 187c111

Browse files
dcramerclaude
andcommitted
feat: Add notification system for scheduled triggers
Replace the single-tracking-issue approach with a provider-based notification layer. Scheduled findings now route through configurable providers (GitHub Issues, Slack) with suppression rules for false positive management. - Add GitHub Issues provider with two-tier dedup (hash + semantic via Haiku) - Add Slack provider with Block Kit webhook posting - Add YAML-based suppression system (skill + glob paths + title matching) - Add notification dispatcher to orchestrate suppression and provider execution - Remove createOrUpdateIssue and issue-renderer (replaced by providers) - Remove issueTitle from schedule config schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent f3880d3 commit 187c111

24 files changed

Lines changed: 1710 additions & 341 deletions

specs/notifications.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Notification System for Scheduled Triggers
2+
3+
## Problem
4+
5+
Warden's scheduled triggers analyze the full codebase on a cron. Today, findings go to a single tracking issue per skill that gets overwritten each run. There's no way to:
6+
- Get notified in Slack when new findings appear
7+
- Mark findings as false positives so they don't recur
8+
- Track individual findings across runs
9+
10+
## Goals
11+
12+
1. Per-finding GitHub issues with semantic dedup across runs
13+
2. Slack webhook notifications for new findings
14+
3. Suppression file for false positives (rule-based pre-filtering)
15+
4. Replace the single-tracking-issue approach with a provider-based notification layer
16+
17+
## Configuration
18+
19+
### `warden.toml`
20+
21+
```toml
22+
[[notifications]]
23+
type = "github-issues"
24+
labels = ["warden"]
25+
26+
[[notifications]]
27+
type = "slack"
28+
webhookUrl = "$SLACK_WEBHOOK_URL"
29+
```
30+
31+
Top-level `[[notifications]]` array. Each provider receives all non-suppressed findings independently. Environment variables expanded at runtime via `$VAR` syntax.
32+
33+
### Migration
34+
35+
The `[[notifications]]` section replaces the existing `schedule.issueTitle` tracking-issue approach. The `createOrUpdateIssue` code path and `schedule.issueTitle` config are removed. `schedule.createFixPR` and `schedule.fixBranchPrefix` remain (fix PRs are orthogonal to notifications).
36+
37+
## Suppression File
38+
39+
Located at `.agents/warden/suppressions.yaml`:
40+
41+
```yaml
42+
suppressions:
43+
- skill: "security-audit"
44+
paths: ["src/legacy/**"]
45+
reason: "Legacy code, not worth fixing"
46+
47+
- skill: "security-audit"
48+
paths: ["src/admin/query.ts"]
49+
title: "SQL injection"
50+
reason: "Uses parameterized queries, false positive"
51+
```
52+
53+
Rules match on:
54+
- **skill** (required): Exact skill name
55+
- **paths** (required): Glob patterns matched against finding location path
56+
- **title** (optional): Substring match against finding title
57+
- **reason** (required): Human-readable justification
58+
59+
Loaded once per workflow run, applied before any provider receives findings.
60+
61+
## Provider Interface
62+
63+
```typescript
64+
interface NotificationProvider {
65+
readonly name: string;
66+
notify(context: NotificationContext): Promise<NotificationResult>;
67+
}
68+
69+
interface NotificationContext {
70+
findings: Finding[];
71+
reports: SkillReport[];
72+
repository: { owner: string; name: string };
73+
commitSha: string;
74+
}
75+
76+
interface NotificationResult {
77+
provider: string;
78+
sent: number;
79+
skipped: number;
80+
errors: string[];
81+
}
82+
```
83+
84+
## Provider Flow
85+
86+
```
87+
Skill Report -> Apply Suppressions -> All Providers (each gets same findings)
88+
|-- github-issues (semantic dedup, creates/skips per finding)
89+
|-- slack (sends all findings it receives)
90+
```
91+
92+
## GitHub Issues Provider
93+
94+
- Creates one issue per unique finding
95+
- Dedup: two-tier
96+
1. Hash match via `<!-- warden:SHA256 -->` marker in issue body (cheap, catches identical text)
97+
2. Semantic match via Haiku for same-file findings with no hash match (handles LLM variation)
98+
- Also checks closed issues with `warden:false-positive` label (treated as suppressed)
99+
- Issue title: `[Warden] {finding.title}`
100+
- Labels: configurable base labels + `warden:{skillName}`
101+
- Body: severity, description, location with code link, suggested fix, hash marker
102+
- Config: `labels` (string array, default `["warden"]`)
103+
104+
## Slack Provider
105+
106+
- Posts to incoming webhook URL using Block Kit formatting
107+
- Sends all non-suppressed findings it receives
108+
- Message: repo, commit, severity summary, up to 10 findings with details
109+
- Skips notification if findings array is empty
110+
- Config: `webhookUrl` (string, supports `$ENV_VAR` expansion)
111+
112+
## Schedule Workflow Integration
113+
114+
In `src/action/workflow/schedule.ts`, the `createOrUpdateIssue` call is replaced with:
115+
116+
```typescript
117+
const dispatcher = new NotificationDispatcher(providers, suppressions);
118+
const result = await dispatcher.dispatch({
119+
findings: report.findings,
120+
reports: [report],
121+
repository: { owner, name: repo },
122+
commitSha: headSha,
123+
skillName: resolved.name,
124+
});
125+
```
126+
127+
## Future Work
128+
129+
- Persistent storage for "previously seen" finding tracking across all providers
130+
- `autoClose` config flag on GitHub Issues provider (close issues when findings disappear)
131+
- CLI command for managing suppressions (`warden suppress add`)
132+
- PR trigger integration (generic enough, but PR comments already have sophisticated dedup)

src/action/workflow/__fixtures__/schedule-title/warden.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,4 @@ paths = ["src/**/*.ts"]
88
type = "schedule"
99

1010
[skills.triggers.schedule]
11-
issueTitle = "Custom Issue Title"
11+
createFixPR = false

src/action/workflow/schedule.test.ts

Lines changed: 23 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -52,12 +52,24 @@ vi.mock('../../event/schedule-context.js', () => ({
5252
buildScheduleEventContext: vi.fn(),
5353
}));
5454

55-
// Mock GitHub issue/PR creation
55+
// Mock GitHub PR creation (createOrUpdateIssue removed; notifications handle issues now)
5656
vi.mock('../../output/github-issues.js', () => ({
57-
createOrUpdateIssue: vi.fn(),
5857
createFixPR: vi.fn(),
5958
}));
6059

60+
// Mock suppressions loader
61+
vi.mock('../../suppressions/loader.js', () => ({
62+
loadSuppressions: vi.fn(() => []),
63+
}));
64+
65+
// Mock notification system
66+
vi.mock('../../notifications/index.js', () => ({
67+
NotificationDispatcher: vi.fn().mockImplementation(() => ({
68+
dispatch: vi.fn(() => Promise.resolve({ suppressed: 0, results: [] })),
69+
})),
70+
buildProviders: vi.fn(() => []),
71+
}));
72+
6173
// Mock skill loader — filesystem reads; keep clearSkillsCache real
6274
vi.mock('../../skills/loader.js', async () => {
6375
const actual = await vi.importActual('../../skills/loader.js');
@@ -76,7 +88,7 @@ vi.mock('../../skills/loader.js', async () => {
7688
// Import after mocks
7789
import { runSkill } from '../../sdk/runner.js';
7890
import { buildScheduleEventContext } from '../../event/schedule-context.js';
79-
import { createOrUpdateIssue, createFixPR } from '../../output/github-issues.js';
91+
import { createFixPR } from '../../output/github-issues.js';
8092
import { resolveSkillAsync } from '../../skills/loader.js';
8193
import { setFailed } from './base.js';
8294
import { runScheduleWorkflow } from './schedule.js';
@@ -85,7 +97,6 @@ import { clearSkillsCache } from '../../skills/loader.js';
8597
// Type the mocks
8698
const mockRunSkill = vi.mocked(runSkill);
8799
const mockBuildContext = vi.mocked(buildScheduleEventContext);
88-
const mockCreateOrUpdateIssue = vi.mocked(createOrUpdateIssue);
89100
const mockCreateFixPR = vi.mocked(createFixPR);
90101
const mockResolveSkillAsync = vi.mocked(resolveSkillAsync);
91102
const mockSetFailed = vi.mocked(setFailed);
@@ -195,11 +206,6 @@ describe('runScheduleWorkflow', () => {
195206
// Default mock: context with files, no findings
196207
mockBuildContext.mockResolvedValue(createScheduleContext());
197208
mockRunSkill.mockResolvedValue(createSkillReport());
198-
mockCreateOrUpdateIssue.mockResolvedValue({
199-
issueNumber: 1,
200-
issueUrl: 'https://github.com/test-owner/test-repo/issues/1',
201-
created: true,
202-
});
203209
mockResolveSkillAsync.mockResolvedValue({
204210
name: 'test-skill',
205211
description: 'Test skill',
@@ -227,7 +233,6 @@ describe('runScheduleWorkflow', () => {
227233
await runScheduleWorkflow(mockOctokit, createDefaultInputs(), PR_ONLY_FIXTURES);
228234

229235
expect(mockRunSkill).not.toHaveBeenCalled();
230-
expect(mockCreateOrUpdateIssue).not.toHaveBeenCalled();
231236
});
232237

233238
it('fails when GITHUB_REPOSITORY is not set', async () => {
@@ -270,33 +275,22 @@ describe('runScheduleWorkflow', () => {
270275
// ---------------------------------------------------------------------------
271276

272277
describe('happy path', () => {
273-
it('runs skill and creates issue when findings exist', async () => {
278+
it('runs skill when findings exist', async () => {
274279
const finding = createFinding({ severity: 'high' });
275280
const report = createSkillReport({ findings: [finding] });
276281
mockRunSkill.mockResolvedValue(report);
277282

278283
await runScheduleWorkflow(mockOctokit, createDefaultInputs(), SCHEDULE_FIXTURES);
279284

280285
expect(mockRunSkill).toHaveBeenCalledTimes(1);
281-
expect(mockCreateOrUpdateIssue).toHaveBeenCalledWith(
282-
mockOctokit,
283-
'test-owner',
284-
'test-repo',
285-
[report],
286-
expect.objectContaining({
287-
title: 'Warden: test-skill',
288-
commitSha: 'abc123',
289-
})
290-
);
291286
});
292287

293-
it('creates issue even when no findings', async () => {
288+
it('runs skill even when no findings', async () => {
294289
mockRunSkill.mockResolvedValue(createSkillReport({ findings: [] }));
295290

296291
await runScheduleWorkflow(mockOctokit, createDefaultInputs(), SCHEDULE_FIXTURES);
297292

298293
expect(mockRunSkill).toHaveBeenCalledTimes(1);
299-
expect(mockCreateOrUpdateIssue).toHaveBeenCalledTimes(1);
300294
});
301295

302296
it('skips skill run when no files match trigger', async () => {
@@ -319,15 +313,14 @@ describe('runScheduleWorkflow', () => {
319313
await runScheduleWorkflow(mockOctokit, createDefaultInputs(), SCHEDULE_FIXTURES);
320314

321315
expect(mockRunSkill).not.toHaveBeenCalled();
322-
expect(mockCreateOrUpdateIssue).not.toHaveBeenCalled();
323316
});
324317
});
325318

326319
// ---------------------------------------------------------------------------
327-
// Issue & PR Creation
320+
// Fix PR Creation
328321
// ---------------------------------------------------------------------------
329322

330-
describe('issue and PR creation', () => {
323+
describe('fix PR creation', () => {
331324
it('creates fix PR when schedule.createFixPR is enabled', async () => {
332325
const finding = createFinding({
333326
suggestedFix: { description: 'Fix it', diff: '--- a\n+++ b\n' },
@@ -369,25 +362,17 @@ describe('runScheduleWorkflow', () => {
369362
expect(mockCreateFixPR).not.toHaveBeenCalled();
370363
});
371364

372-
it('uses custom issue title from schedule config', async () => {
373-
const report = createSkillReport({ findings: [] });
374-
mockRunSkill.mockResolvedValue(report);
365+
it('does not create fix PR for schedule-title fixture', async () => {
366+
const finding = createFinding();
367+
mockRunSkill.mockResolvedValue(createSkillReport({ findings: [finding] }));
375368

376369
await runScheduleWorkflow(
377370
mockOctokit,
378371
createDefaultInputs(),
379372
SCHEDULE_TITLE_FIXTURES
380373
);
381374

382-
expect(mockCreateOrUpdateIssue).toHaveBeenCalledWith(
383-
mockOctokit,
384-
'test-owner',
385-
'test-repo',
386-
[report],
387-
expect.objectContaining({
388-
title: 'Custom Issue Title',
389-
})
390-
);
375+
expect(mockCreateFixPR).not.toHaveBeenCalled();
391376
});
392377
});
393378

src/action/workflow/schedule.ts

Lines changed: 26 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,13 @@
77
import { dirname, join } from 'node:path';
88
import type { Octokit } from '@octokit/rest';
99
import { loadWardenConfig, resolveSkillConfigs } from '../../config/loader.js';
10-
import type { ScheduleConfig } from '../../config/schema.js';
1110
import { buildScheduleEventContext } from '../../event/schedule-context.js';
1211
import { runSkill } from '../../sdk/runner.js';
13-
import { createOrUpdateIssue, createFixPR } from '../../output/github-issues.js';
12+
import { createFixPR } from '../../output/github-issues.js';
1413
import { shouldFail, countFindingsAtOrAbove, countSeverity } from '../../triggers/matcher.js';
1514
import { resolveSkillAsync } from '../../skills/loader.js';
15+
import { loadSuppressions } from '../../suppressions/loader.js';
16+
import { NotificationDispatcher, buildProviders } from '../../notifications/index.js';
1617
import type { SkillReport } from '../../types/index.js';
1718
import type { ActionInputs } from '../inputs.js';
1819
import {
@@ -70,6 +71,13 @@ export async function runScheduleWorkflow(
7071

7172
const defaultBranch = await getDefaultBranchFromAPI(octokit, owner, repo);
7273

74+
// Load suppressions and build notification providers
75+
const suppressions = loadSuppressions(repoPath);
76+
const providers = config.notifications
77+
? buildProviders({ configs: config.notifications, octokit, apiKey: inputs.anthropicApiKey })
78+
: [];
79+
const dispatcher = new NotificationDispatcher(providers, suppressions);
80+
7381
logGroup('Processing schedule triggers');
7482
for (const trigger of scheduleTriggers) {
7583
console.log(`- ${trigger.name}: ${trigger.skill}`);
@@ -127,24 +135,27 @@ export async function runScheduleWorkflow(
127135
allReports.push(report);
128136
totalFindings += report.findings.length;
129137

130-
// Create/update issue with findings
131-
const scheduleConfig: Partial<ScheduleConfig> = resolved.schedule ?? {};
132-
const issueTitle = scheduleConfig.issueTitle ?? `Warden: ${resolved.name}`;
133-
134-
const issueResult = await createOrUpdateIssue(octokit, owner, repo, [report], {
135-
title: issueTitle,
136-
commitSha: headSha,
137-
});
138+
// Dispatch notifications for findings
139+
if (providers.length > 0) {
140+
const dispatchResult = await dispatcher.dispatch({
141+
findings: report.findings,
142+
reports: [report],
143+
repository: { owner, name: repo },
144+
commitSha: headSha,
145+
skillName: resolved.name,
146+
});
138147

139-
if (issueResult) {
140-
console.log(`${issueResult.created ? 'Created' : 'Updated'} issue #${issueResult.issueNumber}`);
141-
console.log(`Issue URL: ${issueResult.issueUrl}`);
148+
for (const r of dispatchResult.results) {
149+
if (r.sent > 0) {
150+
console.log(`${r.provider}: ${r.sent} notification${r.sent === 1 ? '' : 's'} sent`);
151+
}
152+
}
142153
}
143154

144155
// Create fix PR if enabled and there are fixable findings
145-
if (scheduleConfig.createFixPR) {
156+
if (resolved.schedule?.createFixPR) {
146157
const fixResult = await createFixPR(octokit, owner, repo, report.findings, {
147-
branchPrefix: scheduleConfig.fixBranchPrefix ?? 'warden-fix',
158+
branchPrefix: resolved.schedule.fixBranchPrefix ?? 'warden-fix',
148159
baseBranch: defaultBranch,
149160
baseSha: headSha,
150161
repoPath,

0 commit comments

Comments
 (0)