Skip to content

feat: storyboard-driven compliance routing#445

Merged
bokelley merged 10 commits intomainfrom
bokelley/storyboard-client
Apr 8, 2026
Merged

feat: storyboard-driven compliance routing#445
bokelley merged 10 commits intomainfrom
bokelley/storyboard-client

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented Apr 8, 2026

Summary

Migrates comply() from track-based to storyboard-based routing, completing the client-side of adcp#1985.

  • PLATFORM_STORYBOARDS — maps all 15 platform types to curated storyboard ID sets
  • storyboards option on ComplyOptions — priority: storyboards > platform_type > tracks > all
  • extractScenariosFromStoryboard() and filterToKnownScenarios() — scenario extraction and validation helpers
  • Removes legacy TRACK_DEFINITIONS, TRACK_RELEVANCE, and runAgentTest import — storyboard track and required_tools fields now drive applicability
  • Tracks become a reporting/grouping layer derived from storyboard results, not a routing mechanism
  • Fixes skipped steps incorrectly marked as failed in track result mapping
  • Removes dead runTrackStoryboards/getApplicableTracks functions
  • Deduplicates TRACK_LABELS to single source of truth
  • Adds audience_sync to display_ad_server, dsp, audio_platform platform mappings
  • Adds media_buy_non_guaranteed to search_platform, ai_platform platform mappings

Test plan

  • npx tsc --noEmit — clean (0 errors)
  • npm run build — clean
  • 18 test suites pass (50 existing compliance + 19 new platform-storyboards tests), 0 failures
  • ESLint: 0 errors (3 pre-existing as any warnings)
  • Changeset included for minor version bump

🤖 Generated with Claude Code

bokelley and others added 2 commits April 8, 2026 12:27
Migrate comply() from track-based to storyboard-based routing.
Storyboards are now the routing mechanism; tracks become a
reporting layer derived from storyboard results.

- Add PLATFORM_STORYBOARDS mapping (15 platform types → storyboard IDs)
- Add storyboards option to ComplyOptions (priority: storyboards > platform_type > tracks > all)
- Add extractScenariosFromStoryboard() and filterToKnownScenarios()
- Remove TRACK_DEFINITIONS, TRACK_RELEVANCE, and legacy scenario runner
- Remove dead runTrackStoryboards/getApplicableTracks functions
- Deduplicate TRACK_LABELS (single source in storyboard-tracks.ts)
- Fix skipped steps incorrectly marked as failed in track mapping
- Add audience_sync to display_ad_server, dsp, audio_platform
- Add media_buy_non_guaranteed to search_platform, ai_platform

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Run Prettier on all changed files
- Fix state-machine-scenarios test that checked removed TRACK_DEFINITIONS
- Add storyboard-specific comply_scenario names to KNOWN_SCENARIOS
- Fix filterToKnownScenarios return type to string[] (not all names are TestScenario)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor Author

@bokelley bokelley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The migration looks right — removing TRACK_DEFINITIONS, TRACK_RELEVANCE, and the legacy runAgentTest import is the correct cleanup. Storyboards own routing now. Two issues to flag:

Skipped step mapping regression

-    passed: stepResult.skipped ? false : stepResult.passed,
+    passed: stepResult.skipped ? true : stepResult.passed,

This reverts the fix from #441 (#440). The #441 change was specifically to prevent skipped steps from being counted as passed, which was masking tool discovery mismatches — all tracks showed "pass" when every step was actually skipped. Changing this back to true reintroduces that masking. Was this intentional? If the concern is that skipped steps shouldn't count as failures either, the fix should be to exclude them from the pass/fail accounting in computeTrackStatus() rather than counting them as passes.

Phantom scenario names in KNOWN_SCENARIOS

account_setup, audience_sync, governance_setup, media_buy_flow are listed as known scenarios. In adcp#1985 we explicitly removed these phantom names from all storyboard YAMLs and replaced them with real TestScenario values (full_sales_flow, create_media_buy, media_buy_lifecycle, reporting_flow, creative_lifecycle, etc.). If the client keeps phantom names as "known," filterToKnownScenarios() won't filter them out, which defeats the validation purpose.

These phantom names existed in the original storyboard migration (#424) before the comply_scenario annotations were fixed. They should be removed from KNOWN_SCENARIOS.

Storyboard divergence: two sets of storyboards

Right now there are two storyboard sets:

  • adcp repo: docs/storyboards/*.yaml (21 storyboards — the protocol-level definitions)
  • adcp-client repo: storyboards/*.yaml (bundled with the npm package)

These have diverged — the adcp repo has storyboards the client doesn't (social_platform, si_session, brand_rights, property_governance, content_standards) and different comply_scenario annotations. The client has storyboards the adcp repo doesn't (schema_validation, behavioral_analysis, deterministic_testing).

The adcp repo storyboards should be the single source of truth. The client should either:

  1. Import them at build time — copy from adcp repo during the @adcp/client build/release process
  2. Fetch them at runtime — load from a well-known URL (e.g., the published docs site)
  3. Publish as a shared package@adcp/storyboards that both repos depend on

Option 1 is simplest for now. The client build step could pull the latest storyboard YAMLs from the adcp repo's docs/storyboards/ directory. This ensures one source of truth and prevents drift.

The PLATFORM_STORYBOARDS mapping also needs to converge — adcp#1985 maps to capability_discovery as first storyboard for every platform; this PR maps to schema_validation. The right answer depends on which storyboards exist in the canonical set.

bokelley and others added 8 commits April 8, 2026 12:55
- Revert skipped steps back to passed:false (restoring #441 fix).
  Skipped steps must not count as passed or computeTrackStatus()
  masks tool discovery mismatches.
- Remove phantom comply_scenario names (account_setup, audience_sync,
  governance_setup, media_buy_flow) from KNOWN_SCENARIOS. These are
  storyboard YAML artifacts, not real TestScenario values.
  filterToKnownScenarios() should filter them out, not legitimize them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace client storyboard copies with canonical definitions from
adcontextprotocol/adcp (PR #2002 + #2003).

26 canonical protocol storyboards + deterministic_testing (client overlay).

Added (8 new from adcp):
- capability_discovery, social_platform, creative_lifecycle
- campaign_governance_conditions, campaign_governance_delivery,
  campaign_governance_denied
- content_standards (replaces governance_content_standards)
- property_governance (replaces governance_property_lists)

Removed (renamed in adcp):
- governance_content_standards.yaml → content_standards.yaml
- governance_property_lists.yaml → property_governance.yaml

Updated (18 synced with canonical versions):
- All existing storyboards now match adcp repo definitions
- track, required_tools, platform_types fields from adcp

PLATFORM_STORYBOARDS now matches the canonical adcp mapping exactly:
- capability_discovery leads every platform type
- creative_library maps to creative_lifecycle (not creative_template)
- All sales platforms include media_buy_state_machine and error_compliance

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pulls merged canonical version which adds audio_platform
to platform_types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Run sync-schemas + generate-types to pick up upstream schema changes.
Fix removed SignalPricingOption/SignalPricing re-exports (renamed to PricingOption).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PricingOption → VendorPricingOption rename in upstream schemas.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
26 storyboards now reflected in llms.txt and TYPE-SUMMARY.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bokelley bokelley merged commit 6dc5ad0 into main Apr 8, 2026
13 checks passed
@bokelley bokelley deleted the bokelley/storyboard-client branch April 8, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant