Skip to content

Commit 69f1dfd

Browse files
os-zhuangclaude
andauthored
fix(webhooks): materialize stack-declared webhooks into the dispatcher (#3461) (#3489)
The spec `WebhookSchema` authoring surface (`defineStack({ webhooks })` / `defineWebhook()`, `object` / `isActive`) was disconnected from the runtime dispatcher, which fans out off `sys_webhook` DATA rows (`object_name` / `active`) written only by hand through the object's CRUD UI. Nothing turned a declared webhook into a dispatchable row, so authoring `webhooks:` on a stack was a silent no-op (ADR-0078) — the showcase itself shipped one that did nothing. - `bootstrapDeclaredWebhooks` reads declared `webhook` metadata from the ObjectQL registry (where manifest decomposition already parks `stack.webhooks`), validates each through `WebhookSchema.parse()` (the spec schema finally gets a real consumer), and materializes it into a `sys_webhook` row: `object → object_name`, `isActive → active`, full envelope → `definition_json`. Runs on the DATA ENGINE alone, before the auto-enqueuer's first refresh — NOT gated behind the realtime/messaging dispatch prerequisites (else a realtime-less deployment reproduces the silent no-op). - Seed-not-clobber provenance (mirrors sys_sharing_rule #2909): `sys_webhook` gains `managed_by` / `customized`. Declared webhooks re-seed as `managed_by: 'package'`; a row an admin created (`admin`) or edited (`customized`, stamped by a beforeUpdate hook) is never overwritten. - showcase: require the `webhooks` + `realtime` capabilities (so the dispatcher actually mounts) and ship the demo webhook inactive (placeholder endpoint). - Fix the stale `SysWebhookDelivery` import in the i18n extract config (dead since delivery moved to service-messaging). Connector `webhooks` remain not-yet-enforced (#3197). Registering `webhook` as a metadata type + GOVERNED liveness enrollment is a tracked follow-up. Verified: 9 new unit tests (mapping / idempotency / seed-not-clobber / invalid / end-to-end dispatch), red-proofed; and a real showcase boot — declared webhook materializes into a sys_webhook row, same-DB reboot stays idempotent, and an admin's customized edit survives redeploy. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent a6c35a2 commit 69f1dfd

10 files changed

Lines changed: 742 additions & 21 deletions

File tree

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
"@objectstack/plugin-webhooks": minor
3+
"@objectstack/spec": patch
4+
---
5+
6+
fix(webhooks): materialize stack-declared webhooks into the dispatcher (#3461)
7+
8+
A webhook authored declaratively — `defineStack({ webhooks })` / `defineWebhook()`,
9+
validated against the spec `WebhookSchema` — was a **silent no-op**. The runtime
10+
dispatcher (`AutoEnqueuer`) fans out off `sys_webhook` DATA rows (`object_name` /
11+
`active`), which until now were only ever written by hand through the object's
12+
CRUD UI. Nothing turned a declared webhook (`object` / `isActive`) into a
13+
dispatchable row, so authoring `webhooks:` on a stack produced `webhook` metadata
14+
that never fired (ADR-0078). The showcase app itself shipped a `webhooks:` entry
15+
that did nothing.
16+
17+
`@objectstack/plugin-webhooks` now bridges the two on boot:
18+
19+
- **`bootstrapDeclaredWebhooks`** reads declared `webhook` metadata from the
20+
ObjectQL registry (where the manifest decomposition already parks
21+
`stack.webhooks`), validates each through `WebhookSchema.parse()` — the spec
22+
schema finally has a real consumer — and materializes it into a `sys_webhook`
23+
row, mapping `object → object_name`, `isActive → active`, and stashing the full
24+
envelope (headers / secret / retry / timeout) in `definition_json`. The
25+
auto-enqueuer's first cache refresh then picks the row up and dispatches it.
26+
- **Seed-not-clobber provenance** (mirrors `sys_sharing_rule`, #2909): `sys_webhook`
27+
gains `managed_by` / `customized` columns. Declared webhooks re-seed every boot
28+
as `managed_by: 'package'`, but a row an admin created (`managed_by: 'admin'`) or
29+
edited in Setup (`customized: true`, stamped by a `beforeUpdate` hook) is never
30+
overwritten — a deactivated noisy webhook survives redeploys.
31+
32+
Connector-declared `webhooks` remain not-yet-enforced (that is a separate seam,
33+
#3197). Registering `webhook` as a first-class metadata type + enrolling it in the
34+
liveness `GOVERNED` set is a tracked follow-up.
35+
36+
Migration: none required. Existing hand-authored `sys_webhook` rows default to
37+
`managed_by: 'admin'` and are never touched by the seeder. Anyone who authored
38+
`webhooks:` on a stack expecting it to fire will find it now does — review those
39+
declarations (especially `url` / `isActive`) before upgrading.

examples/app-showcase/objectstack.config.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ export default defineStack({
100100
// blueprint flow to auto-create a writable "app package" home
101101
// (ADR-0033 zero-package app building) and the Studio package
102102
// selector to list DB packages.
103-
requires: ['ui', 'automation', 'approvals', 'messaging', 'triggers', 'job', 'marketplace'],
103+
requires: ['ui', 'automation', 'approvals', 'messaging', 'triggers', 'job', 'marketplace', 'webhooks', 'realtime'],
104104

105105
// Concrete connectors for the `connector_action` node. The baseline engine
106106
// ships the dispatch node + an empty registry; these plugins populate it.
Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,29 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

3+
import { defineWebhook } from '@objectstack/spec/automation';
4+
35
/**
46
* Outbound webhook — fans out task changes to an external endpoint with a
5-
* retry policy. Validated as part of `defineStack({ webhooks })`.
7+
* retry policy. Authored via `defineStack({ webhooks })`; on boot the webhooks
8+
* plugin materializes this into a `sys_webhook` row (#3461) that the
9+
* auto-enqueuer dispatches off.
10+
*
11+
* Shipped INACTIVE on purpose: `hooks.example` is a placeholder endpoint, so an
12+
* active subscription would emit a failed HTTP delivery on every task change.
13+
* The demo flow is to open Setup → Integrations → Webhooks and flip this row to
14+
* Active (pointing it at a real endpoint) — that admin edit is remembered
15+
* (`customized: true`) and survives redeploys.
616
*/
7-
export const TaskChangedWebhook = {
17+
export const TaskChangedWebhook = defineWebhook({
818
name: 'showcase_task_changed',
919
label: 'Task Changed → External',
1020
object: 'showcase_task',
11-
triggers: ['create', 'update', 'delete'] as ('create' | 'update' | 'delete')[],
21+
triggers: ['create', 'update', 'delete'],
1222
url: 'https://hooks.example/showcase/task',
13-
method: 'POST' as const,
14-
retryPolicy: { maxRetries: 3, backoffStrategy: 'exponential' as const, initialDelayMs: 1000, maxDelayMs: 30000 },
15-
isActive: true,
16-
description: 'Sends task lifecycle events to an external system.',
17-
};
23+
method: 'POST',
24+
retryPolicy: { maxRetries: 3, backoffStrategy: 'exponential', initialDelayMs: 1000, maxDelayMs: 30000 },
25+
isActive: false,
26+
description: 'Sends task lifecycle events to an external system. Activate in Setup and point at a real endpoint.',
27+
});
1828

1929
export const allWebhooks = [TaskChangedWebhook];

packages/plugins/plugin-webhooks/scripts/i18n-extract.config.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,17 @@
1414

1515
import { defineStack } from '@objectstack/spec';
1616
import { SysWebhook } from '../src/sys-webhook.object.js';
17-
import { SysWebhookDelivery } from '../src/sys-webhook-delivery.object.js';
17+
// NOTE: sys_webhook_delivery moved to @objectstack/service-messaging
18+
// (sys_http_delivery, ADR-0018 M3) — this plugin no longer owns a delivery
19+
// object, so it is not extracted here.
1820
import { enObjects } from '../src/translations/en.objects.generated.js';
1921
import { zhCNObjects } from '../src/translations/zh-CN.objects.generated.js';
2022
import { jaJPObjects } from '../src/translations/ja-JP.objects.generated.js';
2123
import { esESObjects } from '../src/translations/es-ES.objects.generated.js';
2224

2325
export default defineStack({
2426
name: 'plugin-webhooks-i18n-extract',
25-
objects: [SysWebhook, SysWebhookDelivery] as any,
27+
objects: [SysWebhook] as any,
2628
translations: [
2729
{ en: { objects: enObjects } },
2830
{ 'zh-CN': { objects: zhCNObjects } },
Lines changed: 283 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,283 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* bootstrapDeclaredWebhooks — the ingestion bridge that closes #3461.
5+
*
6+
* Verifies that stack/connector-declared `webhook` metadata (spec shape:
7+
* `object` / `isActive`) is materialized into `sys_webhook` data rows
8+
* (`object_name` / `active` / `definition_json`), idempotently and without
9+
* clobbering admin edits — and that the dispatcher then sees those rows.
10+
*/
11+
12+
import { describe, expect, it, vi } from 'vitest';
13+
import { AutoEnqueuer, type HttpEnqueueFn } from './auto-enqueuer.js';
14+
import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js';
15+
import { bindWebhookProvenanceStamp } from './webhook-provenance.js';
16+
17+
// ---------------------------------------------------------------------------
18+
// Fakes
19+
// ---------------------------------------------------------------------------
20+
21+
interface HookEntry {
22+
event: string;
23+
handler: (ctx: any) => any;
24+
object?: string;
25+
packageId?: string;
26+
}
27+
28+
/**
29+
* A small engine fake that mirrors the real ObjectQL surface the bridge and
30+
* provenance hook touch: `find({ filter | where })`, `insert`, update-by-id (the
31+
* patch carries `id`, no `where`), a `_registry.listItems(type)` for declared
32+
* metadata, and `beforeUpdate` hooks that run inside `update()`.
33+
*/
34+
class FakeEngine {
35+
rows: Record<string, any[]> = {};
36+
private hooks: HookEntry[] = [];
37+
private declared: Record<string, any[]> = {};
38+
39+
constructor(seed?: { rows?: Record<string, any[]>; declared?: Record<string, any[]> }) {
40+
if (seed?.rows) this.rows = JSON.parse(JSON.stringify(seed.rows));
41+
if (seed?.declared) this.declared = JSON.parse(JSON.stringify(seed.declared));
42+
}
43+
44+
// Declared-metadata registry (where manifest decomposition parks stack.webhooks).
45+
get _registry() {
46+
return {
47+
listItems: (type: string) => (this.declared[type] ?? []).map((content) => ({ content })),
48+
};
49+
}
50+
51+
private matches(row: any, cond?: Record<string, any>): boolean {
52+
if (!cond) return true;
53+
return Object.entries(cond).every(([k, v]) => row[k] === v);
54+
}
55+
56+
async find(name: string, q?: any): Promise<any[]> {
57+
const all = this.rows[name] ?? [];
58+
const cond = q?.filter ?? q?.where;
59+
const out = all.filter((r) => this.matches(r, cond));
60+
return typeof q?.limit === 'number' ? out.slice(0, q.limit) : out;
61+
}
62+
async findOne(name: string, q?: any): Promise<any> {
63+
return (await this.find(name, q))[0] ?? null;
64+
}
65+
async insert(name: string, data: any): Promise<any> {
66+
const arr = (this.rows[name] = this.rows[name] ?? []);
67+
arr.push({ ...data });
68+
return data;
69+
}
70+
async update(name: string, data: any, opts?: any): Promise<any> {
71+
// Run beforeUpdate hooks (the provenance stamp lives here).
72+
const id = data?.id ?? opts?.where?.id;
73+
const ctx = { input: { id, data }, session: opts?.context };
74+
for (const h of this.hooks) {
75+
if (h.event === 'beforeUpdate' && (!h.object || h.object === name)) {
76+
await h.handler(ctx);
77+
}
78+
}
79+
const arr = this.rows[name] ?? [];
80+
const cond = opts?.where ?? (id ? { id } : undefined);
81+
for (const r of arr) {
82+
if (this.matches(r, cond)) Object.assign(r, data);
83+
}
84+
return { affected: 0 };
85+
}
86+
async delete(): Promise<any> {
87+
return { affected: 0 };
88+
}
89+
async count(name: string): Promise<number> {
90+
return (this.rows[name] ?? []).length;
91+
}
92+
async aggregate(): Promise<any[]> {
93+
return [];
94+
}
95+
96+
registerHook(event: string, handler: (ctx: any) => any, options?: Record<string, any>): void {
97+
this.hooks.push({ event, handler, object: options?.object, packageId: options?.packageId });
98+
}
99+
unregisterHooksByPackage(packageId: string): number {
100+
const before = this.hooks.length;
101+
this.hooks = this.hooks.filter((h) => h.packageId !== packageId);
102+
return before - this.hooks.length;
103+
}
104+
}
105+
106+
class FakeRealtime {
107+
private subs = new Map<string, { handler: any; opts?: any }>();
108+
private n = 0;
109+
async publish(event: any): Promise<void> {
110+
for (const sub of this.subs.values()) {
111+
const o = sub.opts ?? {};
112+
if (o.object && event.object !== o.object) continue;
113+
await sub.handler(event);
114+
}
115+
}
116+
async subscribe(_channel: string, handler: any, opts?: any): Promise<string> {
117+
const id = `s-${++this.n}`;
118+
this.subs.set(id, { handler, opts });
119+
return id;
120+
}
121+
async unsubscribe(id: string): Promise<void> {
122+
this.subs.delete(id);
123+
}
124+
}
125+
126+
const ADMIN_CTX = { isSystem: false, positions: [], permissions: [] };
127+
128+
function declaredWebhook(over: Record<string, any> = {}): any {
129+
return {
130+
name: 'task_changed',
131+
label: 'Task Changed',
132+
object: 'showcase_task',
133+
triggers: ['create', 'update'],
134+
url: 'https://hooks.example/task',
135+
method: 'POST',
136+
isActive: true,
137+
...over,
138+
};
139+
}
140+
141+
async function flush() {
142+
await new Promise((r) => setTimeout(r, 0));
143+
}
144+
145+
// ---------------------------------------------------------------------------
146+
// Tests
147+
// ---------------------------------------------------------------------------
148+
149+
describe('bootstrapDeclaredWebhooks', () => {
150+
it('materializes a declared webhook into a sys_webhook row (object→object_name, isActive→active)', async () => {
151+
const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } });
152+
const res = await bootstrapDeclaredWebhooks(engine as any, null);
153+
154+
expect(res).toEqual({ seeded: 1, skipped: 0 });
155+
const rows = engine.rows['sys_webhook'];
156+
expect(rows).toHaveLength(1);
157+
const row = rows[0];
158+
expect(row.name).toBe('task_changed');
159+
expect(row.object_name).toBe('showcase_task'); // object → object_name
160+
expect(row.active).toBe(true); // isActive → active
161+
expect(row.method).toBe('post'); // lowercased to match the select options
162+
expect(row.managed_by).toBe('package');
163+
expect(row.customized).toBe(false);
164+
// Full validated envelope stashed for the enqueuer's advanced-config read.
165+
const defn = JSON.parse(row.definition_json);
166+
expect(defn.object).toBe('showcase_task');
167+
expect(defn.timeoutMs).toBe(30000); // default filled by WebhookSchema.parse
168+
});
169+
170+
it('maps isActive:false → active:false so a placeholder webhook ships inactive', async () => {
171+
const engine = new FakeEngine({ declared: { webhook: [declaredWebhook({ isActive: false })] } });
172+
await bootstrapDeclaredWebhooks(engine as any, null);
173+
expect(engine.rows['sys_webhook'][0].active).toBe(false);
174+
});
175+
176+
it('is idempotent — a second boot updates in place, never duplicates', async () => {
177+
const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } });
178+
await bootstrapDeclaredWebhooks(engine as any, null);
179+
await bootstrapDeclaredWebhooks(engine as any, null);
180+
expect(engine.rows['sys_webhook']).toHaveLength(1);
181+
});
182+
183+
it('propagates a declared change to a pristine (non-customized) row', async () => {
184+
const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } });
185+
await bootstrapDeclaredWebhooks(engine as any, null);
186+
187+
engine['declared'].webhook = [declaredWebhook({ url: 'https://hooks.example/task-v2' })];
188+
await bootstrapDeclaredWebhooks(engine as any, null);
189+
190+
expect(engine.rows['sys_webhook']).toHaveLength(1);
191+
expect(engine.rows['sys_webhook'][0].url).toBe('https://hooks.example/task-v2');
192+
});
193+
194+
it('seed-not-clobber: an admin edit (customized) survives the next boot', async () => {
195+
const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } });
196+
bindWebhookProvenanceStamp(engine as any);
197+
await bootstrapDeclaredWebhooks(engine as any, null);
198+
199+
// Admin deactivates the noisy webhook through the CRUD door (non-system).
200+
const id = engine.rows['sys_webhook'][0].id;
201+
await engine.update('sys_webhook', { id, active: false }, { context: ADMIN_CTX });
202+
expect(engine.rows['sys_webhook'][0].customized).toBe(true); // hook stamped it
203+
204+
// Redeploy re-runs the seeder — the declared row is still active:true, but
205+
// the admin's active:false must win.
206+
await bootstrapDeclaredWebhooks(engine as any, null);
207+
expect(engine.rows['sys_webhook'][0].active).toBe(false);
208+
});
209+
210+
it('never overwrites an admin-authored row that collides by name', async () => {
211+
const engine = new FakeEngine({
212+
rows: {
213+
sys_webhook: [
214+
{ id: 'admin-1', name: 'task_changed', url: 'https://admin.example', active: true, managed_by: 'admin', customized: false },
215+
],
216+
},
217+
declared: { webhook: [declaredWebhook()] },
218+
});
219+
const res = await bootstrapDeclaredWebhooks(engine as any, null);
220+
expect(res).toEqual({ seeded: 0, skipped: 1 });
221+
expect(engine.rows['sys_webhook']).toHaveLength(1);
222+
expect(engine.rows['sys_webhook'][0].url).toBe('https://admin.example'); // untouched
223+
});
224+
225+
it('skips an invalid declared webhook (bad URL) with a warning, without crashing boot', async () => {
226+
const warn = vi.fn();
227+
const engine = new FakeEngine({
228+
declared: { webhook: [declaredWebhook({ name: 'good' }), declaredWebhook({ name: 'bad', url: 'not-a-url' })] },
229+
});
230+
const res = await bootstrapDeclaredWebhooks(engine as any, null, { warn });
231+
232+
expect(res.seeded).toBe(1); // the good one still lands
233+
expect(res.skipped).toBe(1);
234+
expect(engine.rows['sys_webhook'].map((r) => r.name)).toEqual(['good']);
235+
expect(warn).toHaveBeenCalledWith(expect.stringContaining('failed validation'), expect.objectContaining({ name: 'bad' }));
236+
});
237+
238+
it('is a no-op when nothing is declared', async () => {
239+
const engine = new FakeEngine();
240+
const res = await bootstrapDeclaredWebhooks(engine as any, null);
241+
expect(res).toEqual({ seeded: 0, skipped: 0 });
242+
expect(engine.rows['sys_webhook']).toBeUndefined();
243+
});
244+
245+
it('end-to-end: a declared webhook, once materialized, dispatches on a matching data event', async () => {
246+
const engine = new FakeEngine({
247+
declared: {
248+
webhook: [
249+
declaredWebhook({
250+
triggers: ['create'],
251+
secret: 'shh',
252+
headers: { 'X-Env': 'prod' },
253+
}),
254+
],
255+
},
256+
});
257+
await bootstrapDeclaredWebhooks(engine as any, null);
258+
259+
const realtime = new FakeRealtime();
260+
const calls: any[] = [];
261+
const enqueue: HttpEnqueueFn = async (input) => {
262+
calls.push(input);
263+
return 'id';
264+
};
265+
const ae = new AutoEnqueuer(engine as any, realtime as any, enqueue, { refreshIntervalMs: 0 });
266+
await ae.start();
267+
268+
await realtime.publish({
269+
type: 'data.record.created',
270+
object: 'showcase_task',
271+
payload: { recordId: 't-1' },
272+
timestamp: '2026-05-24T00:00:00.000Z',
273+
});
274+
await flush();
275+
276+
expect(calls).toHaveLength(1);
277+
expect(calls[0].url).toBe('https://hooks.example/task');
278+
// headers + secret came from the definition_json envelope the bridge wrote.
279+
expect(calls[0].signingSecret).toBe('shh');
280+
expect(calls[0].headers).toEqual({ 'X-Env': 'prod' });
281+
await ae.stop();
282+
});
283+
});

0 commit comments

Comments
 (0)