Skip to content

Make INDEX/routing-surface consistency a CI-checked invariant, not an authoring discipline #592

Description

@SUaDtL

Summary

The one-line surface summaries the orchestrator routes by (skills INDEX.md, agents INDEX.md, routing-table.md) are documentation, and documentation drifts from the bodies it describes. Today nothing mechanical detects that drift; the failure mode is silent misrouting. Add a CI check that fails when a surface entry and its body disagree.

Evidence

  • Documentation-drift literature at scale: code and comments co-evolve in ~90% of change events, but the doc update frequently lands in later commits or never (Wen et al., ICPC 2019, https://dl.acm.org/doi/abs/10.1109/ICPC.2019.00019; outdated-docs detection, https://arxiv.org/abs/2307.04291). A 23-skill / 28-agent index is the same artifact class.
  • The cost is amplified here because the index is not passive documentation: it is the routing surface. "Skill bodies load on routing only" means a stale one-liner can send the orchestrator to the wrong skill, and the model consulting routing-table.md inherits the miss (NoLiMa shows association from sparse descriptions is exactly where models are weakest, https://arxiv.org/abs/2502.05167).
  • This was flagged as the clearest open gap in the 2026-08-03 context-architecture research review (dossier in the arbiterforge-site repo, .codearbiter/research/2026-08-03-dossier-4-context-architecture.md): the pattern's known failure modes are retrieval-miss and index staleness, and staleness currently has no in-place mitigation.

Proposed checks (in ascending strength; pick during implementation)

  1. Inventory parity: every skill/agent on disk has exactly one INDEX row and every INDEX row has a body; routing-table.md routes only to things that exist. Pure structural check, zero false positives; do this first.
  2. Ownership-claim parity: each INDEX row's "routed to by" / "owns" claims cross-checked against the body's frontmatter or a declared owns:/routed-by: field, so the claim is generated or verified rather than hand-maintained.
  3. Derived one-liners: generate INDEX rows from skill frontmatter (description field) at build time, the same way multi-host surfaces are already generated from core/. Drift becomes impossible rather than detected.
  4. Optional freshness tripwire: fail CI when a SKILL.md changes without its INDEX row or routing-table entry being touched in the same PR (suppressible with an explicit marker for genuinely no-op body edits).

Acceptance criteria

  • Check level chosen and recorded (1-4 above); rationale noted
  • CI job fails on a seeded drift fixture (mutated one-liner, orphan row, missing row, dangling route)
  • Runs across all generated host surfaces (ca, ca-codex, ca-pi) so parity holds everywhere
  • skill-author routing-integration phase updated to reference the mechanical check

Origin

2026-08-03 research sweep for the arbiterforge.com "Why hard gates" essay; sibling of #591 (STE-inspired normative authoring) and the blocking-vs-reminder hook classification issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions