Skip to content

fix(validate,coverage): required-backlink matches inverse-name convention + honours alternate-backlinks#351

Merged
avrabe merged 2 commits into
mainfrom
fix/issue-349-required-backlink-inverse
May 31, 2026
Merged

fix(validate,coverage): required-backlink matches inverse-name convention + honours alternate-backlinks#351
avrabe merged 2 commits into
mainfrom
fix/issue-349-required-backlink-inverse

Conversation

@avrabe
Copy link
Copy Markdown
Contributor

@avrabe avrabe commented May 31, 2026

Closes #349.

The bug

Backlink.link_type stores the forward link-type name (e.g. supports) while several built-in schemas — notably schemas/safety-case.yaml — write required-backlink using the inverse name (e.g. supported-by). The strict equality match at rivet-core/src/validate.rs:906 and rivet-core/src/coverage.rs:200 therefore never matched for any inverse-name rule.

Observable effect on the loom safety case reported in the issue:

ERROR: [SG-1] Every safety goal must be supported by evidence (solution) or decomposed into sub-goals ...

…fired for every safety goal even though every one of them was correctly supported (SOL-1 supports → SG-1, etc.). dev.yaml's required-backlink: satisfies happens to use the forward name and worked; the bug surfaced only on schemas following the inverse-name convention.

The fix

Both call sites now accept either spelling:

bl.link_type == name || bl.inverse_type.as_deref() == Some(name)

Same fix path additionally evaluates rule.alternate_backlinksvalidate.rs previously ignored that field outright, so a safety goal satisfied only via an alternate (e.g. decomposed-by from a strategy instead of supported-by) still erroneously fired the rule. coverage.rs had the same gap. The match helper is now closed over the backlinks, called once for the primary and once per alternate; the rule passes when any shape matches.

Acceptance criteria from the issue

The issue lists a primary fix, a secondary bug, and four frictionless-ness suggestions. This PR addresses the two binding items; the frictionless suggestions are intentionally split out as a follow-up (per "one PR per issue, keep PRs focused" — they touch the CLI's rivet get rendering and require a new rivet why command, which is a separate feature).

Issue item Status Where
Match against the backlink's inverse name in validate.rs done validate.rs:902–941
Match against the backlink's inverse name in coverage.rs done coverage.rs:194–225
Evaluate rule.alternate_backlinks in validate.rs (secondary bug) done validate.rs:931–935
Evaluate rule.alternate_backlinks in coverage.rs done coverage.rs:222–225
rivet get shows inbound backlinks follow-up filed as a separate issue would be more appropriate
required-backlink failure message lists found backlinks follow-up the diagnostic-message redesign cuts across all rules, not this fix
rivet why <id> <rule> explain mode follow-up new CLI surface, separate feature
Per-rule failures generally explainable follow-up overlaps with REQ-124 agent-actionable remediation work

Tests

Four regression tests pin the fix — two per engine:

  • validate::tests::required_backlink_inverse_name_is_satisfied_by_forward_link — reproduces the loom GSN scenario exactly (rule names supported-by, artifact has forward supports); without the fix the rule fires, with the fix it does not.
  • validate::tests::validate_honours_alternate_backlinks — goal satisfied only via the alternate (decomposed-by from a strategy) must not fire the rule.
  • coverage::tests::required_backlink_matches_inverse_link_type_name — same scenario from the coverage engine's perspective; without the fix covered = 0, with the fix covered = 1.
  • coverage::tests::coverage_honours_alternate_backlinks — alternate-satisfied goal counts as covered, not uncovered.

Full cargo test -p rivet-core --lib1086 passed, 0 failed.
rivet validate on the rivet repo itself → PASS unchanged.

End-to-end verification

Reproduced and cleared the loom-style scenario on a minimal safety-case project (SG-1 supported by SOL-1; SG-2 decomposed by STRAT-1; SG-3 with neither):

ERROR: [SG-3] Every safety goal must be supported by evidence ...
Result: FAIL (1 errors, ...)

Only the genuinely-unsupported SG-3 errors — SG-1 (primary backlink path) and SG-2 (alternate-backlink path) both pass, which matches the user's expectation in the bug report.

Files touched

  • rivet-core/src/validate.rs — comparison fix + alternate_backlinks evaluation + 2 regression tests
  • rivet-core/src/coverage.rs — comparison fix + alternate_backlinks evaluation + 2 regression tests
  • CHANGELOG.md — Unreleased entry

Commit trailers: Fixes: REQ-004, Refs: #349.


Generated by Claude Code

…tion + honours alternate-backlinks (#349)

`Backlink.link_type` stores the *forward* link-type name (e.g. `supports`)
while schemas like `safety-case.yaml` write `required-backlink` as the
*inverse* name (e.g. `supported-by`). The strict equality match at
`validate.rs:906` and `coverage.rs:200` therefore never matched for any
inverse-name rule — `goal-has-support` fired for every safety goal even
when correctly supported by a solution, and coverage counted every such
goal as uncovered. `dev.yaml`'s `required-backlink: satisfies` happens
to use the forward name and worked; the bug surfaced only on schemas
following the inverse-name convention.

Match now accepts either spelling at both call sites:
`bl.link_type == name || bl.inverse_type.as_deref() == Some(name)`.
Same fix path additionally evaluates `rule.alternate_backlinks` —
`validate.rs` previously ignored that field outright, so a safety goal
satisfied only via an alternate (e.g. `decomposed-by` from a strategy
instead of `supported-by`) still erroneously fired the rule.

Four regression tests pin the behaviour (two per engine): one for the
inverse-name match, one for alternate-backlinks. Reproduces and clears
the loom safety-case scenario from the bug report.

Fixes: REQ-004
Refs: #349
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 31, 2026

📐 Rivet artifact delta

No artifact changes in this PR. Code-only changes (renderer, CLI wiring, tests) don't touch the artifact graph.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite Current: 015037a Previous: cb670c8 Ratio
link_graph_build/10000 37966689 ns/iter (± 3420341) 31257613 ns/iter (± 2082341) 1.21

This comment was automatically generated by workflow using github-action-benchmark.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 31, 2026

Codecov Report

❌ Patch coverage is 99.13793% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rivet-core/src/validate.rs 98.23% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

# Conflicts:
#	CHANGELOG.md
@avrabe avrabe merged commit c5fe0e7 into main May 31, 2026
16 of 18 checks passed
@avrabe avrabe deleted the fix/issue-349-required-backlink-inverse branch May 31, 2026 08:22
avrabe added a commit that referenced this pull request May 31, 2026
…363)

Mark-implemented sweep — acceptance met by merged code:
- REQ-105 (HTML export self-containment, v0.14.0)  draft    -> implemented
- REQ-110/111 (coverage relabel, #348)             draft    -> implemented
- REQ-124 (diagnostic remediation, v0.14.0)        approved -> implemented

Also re-files REQ-131 (the #349 required-backlink fix, status implemented):
its artifact was dropped when #351 was reworked to add alternate-backlink
support, so the code shipped on main without the requirement. Restores the
traceability that issue comments on #349/#350 reference.

rivet validate PASS.

Refs: REQ-105, REQ-110, REQ-111, REQ-124, REQ-131

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

validate/coverage required-backlink rules never match — compares Backlink.link_type (forward) against the inverse name

2 participants