Skip to content

Tighten ast-grep rules in CI - #2753

Open
monadoid wants to merge 4 commits into
mainfrom
protocol-pipeline-checks
Open

Tighten ast-grep rules in CI#2753
monadoid wants to merge 4 commits into
mainfrom
protocol-pipeline-checks

Conversation

@monadoid

@monadoid monadoid commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Before: a developer could add, rename, or remove a protocol field without every SDK constructing or consuming it.

    • Now: source-derived tests compare every registered request and result field with the TypeScript, Python, and Go wrappers.
  • Before: SDK-only options could drift between TypeScript Zod schemas, Python declarations and runtime models, and Go structs.

    • Now: cross-language parity tests derive and compare those fields automatically.
  • Before: public creation and browser options could be added without updating the reference and configuration docs.

    • Now: Stagehand.create fields are checked structurally, and browser and logging fields must appear in the guides.
  • Before: a registered operation could be omitted from its receiving endpoint or wired in the wrong direction.

    • Now: every operation must have exactly one receiving endpoint and matching SDK boundaries.
  • Before: the protocol-schema lint checked obsolete paramsSchema and resultSchema properties.

    • Now: it checks the actual params and result registry entries.

Validation


Summary by cubic

Tightens cross-language protocol pipeline checks and documentation completeness to prevent SDK drift. Enforces per-field construction/consumption, single-endpoint routing with the extension, and canonical schema references.

  • Field pipeline: sdk-field-pipeline.test.ts requires every request field to be constructed or forwarded and every result field to be consumed across TypeScript, Python, and Go; allows rest-spread adapters and exempts page.screenshot.type.

  • SDK-only schema parity: sdk-client-schema-parity.test.ts derives field names from TS Zod schemas and compares them with Python declarations/runtime models and Go structs; Stagehand.create fields must match all reference tabs; all exported SDK-owned schemas are either compared or explicitly classified.

  • Routing: sdk-parity.test.ts ensures each protocol operation is handled by exactly one receiving endpoint (extension or SDK), aligns outbound/inbound boundaries across languages, and matches SDK outbound/inbound sets with the extension router.

  • Docs and references: every public RPC-backed TypeScript object discovered from source must have a reference page; browser and logging fields must appear in the configuration guides; Stagehand.create fields are documented structurally per language tab.

  • Linting: no-loose-json-schema-in-protocol-operations now requires params/result entries in StagehandMethods/StagehandNotifications to reference canonical schemas.

  • CI: runs parity tests via pnpm test:unit.

  • Required migration: replace any remaining paramsSchema/resultSchema with params/result that reference the canonical schemas.

Written for commit 4931836. Summary will update on new commits.

Review in cubic

@mintlify

mintlify Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
stagehand 🟢 Ready View Preview Aug 17, 2026, 1:29 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@changeset-bot

changeset-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 4931836

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@monadoid monadoid changed the title Add protocol pipeline checks Tighten ast-grep rules in CI Aug 17, 2026
@monadoid
monadoid marked this pull request as ready for review August 17, 2026 13:51
@monadoid
monadoid requested a review from a team as a code owner August 17, 2026 13:51

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 7 files

Architecture diagram
sequenceDiagram
    participant CI as CI Pipeline
    participant Unit as Unit Tests (vitest)
    participant Protocol as Protocol Schema (stagehand.v4.json)
    participant Registry as Schema Registry (TS)
    participant TS as TypeScript SDK
    participant PY as Python SDK
    participant GO as Go SDK
    participant Ext as Extension rpcRouter
    participant Docs as Documentation MDX

    Note over CI,Unit: NEW: Cross-language protocol validation pipeline
    
    CI->>Unit: pnpm test:unit
    Unit->>Protocol: Read protocol document
    Unit->>Registry: Read StagehandMethods registry
    
    Note over Unit,TS: Field Pipeline Verification
    Unit->>TS: Extract public RPC calls (source AST)
    Unit->>PY: Extract public RPC calls (source AST)
    Unit->>GO: Extract public RPC calls (source AST)
    TS-->>Unit: Request/result field usage
    PY-->>Unit: Request/result field usage
    GO-->>Unit: Request/result field usage
    Unit->>Unit: Verify every request field constructed & forwarded
    Unit->>Unit: Verify every result field consumed & returned

    Note over Unit,Ext: Routing Parity Verification
    Unit->>Ext: Find request.method route switch
    Ext-->>Unit: Extension inbound operations list
    Unit->>TS: Find outbound .send() calls
    Unit->>PY: Find outbound .send() calls
    Unit->>GO: Find outbound .call() calls
    TS-->>Unit: Outbound operation set
    PY-->>Unit: Outbound operation set
    GO-->>Unit: Outbound operation set
    
    alt Operations handled by both endpoints
        Unit->>Unit: Fail - duplicate routing detected
    else Every operation has exactly one receiver
        Unit->>Unit: Pass - extension and SDK boundaries match
    end

    Note over Unit,Docs: Schema Parity Verification
    Unit->>TS: Read Zod client schemas (clientSchemas.ts)
    TS-->>Unit: Canonical field names
    
    Unit->>PY: Parse TypedDict declarations (client_types.py)
    PY-->>Unit: Public input fields
    Unit->>PY: Parse Pydantic models (client_models.py)
    PY-->>Unit: Runtime model fields
    
    alt Python input != runtime model
        Unit->>Unit: Fail - declaration drift detected
    end
    
    Unit->>GO: Parse struct definitions (client_options.go, browser_factories.go)
    GO-->>Unit: Struct field names
    
    alt Go struct fields != TypeScript Zod fields
        Unit->>Unit: Fail - cross-language schema drift detected
    end

    Note over Unit,Docs: Documentation Completeness
    Unit->>Docs: Scan Stagehand.create reference sections
    Unit->>Docs: Scan browser configuration sections
    Docs-->>Unit: Documented ParamField entries
    
    alt Stagehand.create fields missing from docs
        Unit->>Unit: Fail - reference documentation incomplete
    end
    
    alt Browser/extension options undocumented
        Unit->>Unit: Fail - configuration guide incomplete
    end

    Note over Unit,CI: Lint Rule Update
    Unit->>Unit: Verify no-loose-json-schema rule targets params/result
    alt Operation uses loose schema instead of canonical registry
        Unit->>Unit: Fail - lint error reported
    else Operation references canonical $defs schema
        Unit->>Unit: Pass - schema usage valid
    end

    CI-->>CI: Pipeline completes with gating results
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/docs/v4/reference/stagehand.mdx Outdated
Comment thread packages/docs/v4/configuration/browser.mdx Outdated
Comment thread rules/ast-grep/sdk-parity.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated
Comment thread rules/ast-grep/sdk-field-pipeline.test.ts Outdated

@akeimach akeimach left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants