Skip to content

Add deterministic ARIA drift classifier, evidence materialization, and drift-aware graph/impact#11

Open
danielbdyer wants to merge 1 commit into
mainfrom
codex/add-drift-classifier-module-and-reporting
Open

Add deterministic ARIA drift classifier, evidence materialization, and drift-aware graph/impact#11
danielbdyer wants to merge 1 commit into
mainfrom
codex/add-drift-classifier-module-and-reporting

Conversation

@danielbdyer
Copy link
Copy Markdown
Owner

@danielbdyer danielbdyer commented Mar 7, 2026

Motivation

  • Provide a pure, deterministic way to detect and classify UI ARIA snapshot drift so evidence can be recorded and traced as first-class provenance.
  • Surface richer snapshot diagnostics to runtime and connect drift observations into the derived dependency/provenance graph so impact queries can answer which scenarios are affected by a given drift class.
  • Keep classification, serialization, and fingerprints deterministic to support cache/manifesting and stable downstream tooling.

Description

  • Added a pure domain classifier in lib/domain/drift.ts that compares baseline vs current ARIA snapshots and emits typed deltas (role-change, accessible-name-change, structural-move-order-change, addition-removal) plus deterministic fingerprints via stableStringify/sha256 and normalized snapshot handling.
  • Extended runtime snapshot handling so expectAriaSnapshot now computes and returns a DriftClassification result while keeping the existing assertion semantics (changes in lib/runtime/aria.ts and lib/runtime/program.ts).
  • Implemented an application flow materializeDriftEvidence in lib/application/drift.ts and wired a new CLI command drift so drift reports are materialized under .tesseract/evidence/drift/*.json with provenance and report fingerprints.
  • Extended graph build and derived-graph wiring to ingest evidence artifacts and attach drift metadata to evidence nodes, and to link evidence to snapshots, screens, surfaces, elements, and scenarios (lib/application/graph.ts, lib/domain/derived-graph.ts).
  • Added graph-query/impact integration so impact can answer "what scenarios are affected by this drift class?" via findScenarioIdsByDriftClass and impact --drift-class (lib/domain/graph-query.ts, lib/application/impact.ts, bin/tesseract.ts).
  • Added deterministic unit tests for drift categories, structural-move detection, and serialization stability (tests/domain.spec.ts).

Testing

  • Built the project with npm run build which completed successfully.
  • Ran the targeted test suite with npm test -- tests/domain.spec.ts tests/runtime-errors.spec.ts and all tests passed (15 passed, 0 failed) after iterative fixes.
  • Verified npm test traces and test outputs during development while addressing failing expectations until the suite was green.

Codex Task

danielbdyer pushed a commit that referenced this pull request Apr 25, 2026
@claude
step-11.Z11g.refactor-U: ManifestCone<Apex, Base> typed witness
a98b209 
Move U from Agent A's audit (#11) — names the categorical
"manifest-as-cone-apex" pattern that v2-direction.md §1 claims
("the manifest is the contract") but which the codebase
realized only ad-hoc.

The conceit: every manifest verb fans out into ~5 parallel
projections (probe fixture, rung-2 classifier, rung-3
classifier, runtime handler, MCP tool). Adding a verb is a
**single obligation** doctrinally but a **5-place update** in
practice; nothing typed tied them together.

**The witness** (product/domain/manifest/manifest-cone.ts):

  export type ManifestBaseProjection =
    | 'probe-fixture'
    | 'verb-classifier'
    | 'rung3-classifier'
    | 'runtime-handler'
    | 'mcp-tool';

  export interface ManifestCone {
    readonly apex: Manifest;
    readonly baseNames: readonly ManifestBaseProjection[];
    readonly projection<T>(name, lookup): readonly { verb, projection: T | null }[];
    readonly missingProjections<T>(name, lookup): readonly VerbEntry[];
  }

  manifestCone(manifest) → ManifestCone

The `projection<T>(name, lookup)` method is the per-verb
pull-back: given a base-projection name + a lookup function
`(VerbEntry) => T | null`, returns one entry per manifest
verb showing whether that projection exists.

`missingProjections<T>(name, lookup)` is the convenience —
returns only the verbs lacking the projection. Future
architecture-law tests can use it: "every verb has a probe
fixture" becomes:

  const missing = cone.missingProjections('probe-fixture',
    findFixtureForVerb);
  expect(missing).toEqual([]);

This commit only adds the abstraction. Adoption in arch-law
tests is a follow-up — one law per base projection, asserting
the empty-missing invariant. Each adoption replaces a bespoke
"iterate manifest verbs and grep for fixture" loop with a
one-line cone query.

**Tests**: 4071 passing / 10 skipped (was 4065 post-I+L).
+6 from manifest-cone.laws.spec.ts:
  - L-Apex-Preserved
  - L-BaseNames-Closed
  - L-Projection-Order-Stable
  - L-Missing-Empty-When-Total
  - L-Missing-All-When-Empty
  - partial-projection mixed case

Build ok; manifest drift clean.

https://claude.ai/code/session_01RyCaLypF4TrZqMjuqKpKM6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

codex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@danielbdyer