feat: storyboard-driven compliance routing#445
Conversation
Migrate comply() from track-based to storyboard-based routing. Storyboards are now the routing mechanism; tracks become a reporting layer derived from storyboard results. - Add PLATFORM_STORYBOARDS mapping (15 platform types → storyboard IDs) - Add storyboards option to ComplyOptions (priority: storyboards > platform_type > tracks > all) - Add extractScenariosFromStoryboard() and filterToKnownScenarios() - Remove TRACK_DEFINITIONS, TRACK_RELEVANCE, and legacy scenario runner - Remove dead runTrackStoryboards/getApplicableTracks functions - Deduplicate TRACK_LABELS (single source in storyboard-tracks.ts) - Fix skipped steps incorrectly marked as failed in track mapping - Add audience_sync to display_ad_server, dsp, audio_platform - Add media_buy_non_guaranteed to search_platform, ai_platform Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Run Prettier on all changed files - Fix state-machine-scenarios test that checked removed TRACK_DEFINITIONS - Add storyboard-specific comply_scenario names to KNOWN_SCENARIOS - Fix filterToKnownScenarios return type to string[] (not all names are TestScenario) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bokelley
left a comment
There was a problem hiding this comment.
The migration looks right — removing TRACK_DEFINITIONS, TRACK_RELEVANCE, and the legacy runAgentTest import is the correct cleanup. Storyboards own routing now. Two issues to flag:
Skipped step mapping regression
- passed: stepResult.skipped ? false : stepResult.passed,
+ passed: stepResult.skipped ? true : stepResult.passed,This reverts the fix from #441 (#440). The #441 change was specifically to prevent skipped steps from being counted as passed, which was masking tool discovery mismatches — all tracks showed "pass" when every step was actually skipped. Changing this back to true reintroduces that masking. Was this intentional? If the concern is that skipped steps shouldn't count as failures either, the fix should be to exclude them from the pass/fail accounting in computeTrackStatus() rather than counting them as passes.
Phantom scenario names in KNOWN_SCENARIOS
account_setup, audience_sync, governance_setup, media_buy_flow are listed as known scenarios. In adcp#1985 we explicitly removed these phantom names from all storyboard YAMLs and replaced them with real TestScenario values (full_sales_flow, create_media_buy, media_buy_lifecycle, reporting_flow, creative_lifecycle, etc.). If the client keeps phantom names as "known," filterToKnownScenarios() won't filter them out, which defeats the validation purpose.
These phantom names existed in the original storyboard migration (#424) before the comply_scenario annotations were fixed. They should be removed from KNOWN_SCENARIOS.
Storyboard divergence: two sets of storyboards
Right now there are two storyboard sets:
adcprepo:docs/storyboards/*.yaml(21 storyboards — the protocol-level definitions)adcp-clientrepo:storyboards/*.yaml(bundled with the npm package)
These have diverged — the adcp repo has storyboards the client doesn't (social_platform, si_session, brand_rights, property_governance, content_standards) and different comply_scenario annotations. The client has storyboards the adcp repo doesn't (schema_validation, behavioral_analysis, deterministic_testing).
The adcp repo storyboards should be the single source of truth. The client should either:
- Import them at build time — copy from adcp repo during the @adcp/client build/release process
- Fetch them at runtime — load from a well-known URL (e.g., the published docs site)
- Publish as a shared package —
@adcp/storyboardsthat both repos depend on
Option 1 is simplest for now. The client build step could pull the latest storyboard YAMLs from the adcp repo's docs/storyboards/ directory. This ensures one source of truth and prevents drift.
The PLATFORM_STORYBOARDS mapping also needs to converge — adcp#1985 maps to capability_discovery as first storyboard for every platform; this PR maps to schema_validation. The right answer depends on which storyboards exist in the canonical set.
- Revert skipped steps back to passed:false (restoring #441 fix). Skipped steps must not count as passed or computeTrackStatus() masks tool discovery mismatches. - Remove phantom comply_scenario names (account_setup, audience_sync, governance_setup, media_buy_flow) from KNOWN_SCENARIOS. These are storyboard YAML artifacts, not real TestScenario values. filterToKnownScenarios() should filter them out, not legitimize them. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace client storyboard copies with canonical definitions from adcontextprotocol/adcp (PR #2002 + #2003). 26 canonical protocol storyboards + deterministic_testing (client overlay). Added (8 new from adcp): - capability_discovery, social_platform, creative_lifecycle - campaign_governance_conditions, campaign_governance_delivery, campaign_governance_denied - content_standards (replaces governance_content_standards) - property_governance (replaces governance_property_lists) Removed (renamed in adcp): - governance_content_standards.yaml → content_standards.yaml - governance_property_lists.yaml → property_governance.yaml Updated (18 synced with canonical versions): - All existing storyboards now match adcp repo definitions - track, required_tools, platform_types fields from adcp PLATFORM_STORYBOARDS now matches the canonical adcp mapping exactly: - capability_discovery leads every platform type - creative_library maps to creative_lifecycle (not creative_template) - All sales platforms include media_buy_state_machine and error_compliance Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pulls merged canonical version which adds audio_platform to platform_types. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Run sync-schemas + generate-types to pick up upstream schema changes. Fix removed SignalPricingOption/SignalPricing re-exports (renamed to PricingOption). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PricingOption → VendorPricingOption rename in upstream schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
26 storyboards now reflected in llms.txt and TYPE-SUMMARY.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
Migrates
comply()from track-based to storyboard-based routing, completing the client-side of adcp#1985.PLATFORM_STORYBOARDS— maps all 15 platform types to curated storyboard ID setsstoryboardsoption onComplyOptions— priority:storyboards>platform_type>tracks> allextractScenariosFromStoryboard()andfilterToKnownScenarios()— scenario extraction and validation helpersTRACK_DEFINITIONS,TRACK_RELEVANCE, andrunAgentTestimport — storyboardtrackandrequired_toolsfields now drive applicabilityrunTrackStoryboards/getApplicableTracksfunctionsTRACK_LABELSto single source of truthaudience_synctodisplay_ad_server,dsp,audio_platformplatform mappingsmedia_buy_non_guaranteedtosearch_platform,ai_platformplatform mappingsTest plan
npx tsc --noEmit— clean (0 errors)npm run build— cleanas anywarnings)🤖 Generated with Claude Code