actions: surface genuine oasdiff errors as PR annotations#130
Merged
Conversation
Previously a failed oasdiff invocation only printed to stderr (visible in the raw log, not on the Checks tab) unless it was the disallowed-external-ref case. Each entrypoint now promotes a genuine failure to a ::error:: GitHub annotation so it shows on the PR. Gated on exit code >=2, not "!= 0": exit 0 is success and exit 1 is the intended fail-on / changes-found result (not an error), so a normal "breaking changes found" run is never mislabeled as an error. The disallowed-external-ref remedy (exit 123) is layered on top. Adds test-error-annotation.yaml: drives the validate (standalone) and changelog (early-exit) entrypoints with a stubbed oasdiff at exit 102/123/1 and asserts an annotation is emitted for genuine errors and the remedy for 123, but not for the fail-on exit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on test Previously only validate + changelog were exercised, and the exit-0 success path was untested. Now loops over all five entrypoints (breaking, changelog, diff, pr-comment, validate) across four cases each: - exit 0 (success) -> no ::error:: annotation - exit 102 (genuine load error) -> ::error:: annotation surfaced - exit 123 (disallowed external ref) -> allow-external-refs remedy - exit 1 (fail-on/changes) -> no ::error:: annotation pr-comment is driven with an empty oasdiff-token so its success path skips the network POST; its error cases early-exit before it. 20 assertions, all hermetic (stubbed oasdiff, no network). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #128. Makes a failed oasdiff invocation visible on the PR's Checks tab, not just buried in the raw step log.
What
Each entrypoint (
breaking,changelog,diff,pr-comment,validate) now promotes a genuine oasdiff failure to a GitHub::error::annotation (in addition to keeping the raw stderr in the log).Gated on exit code
>= 2, not!= 0:0= success1= the intended fail-on / changes-found result — not an error, so a normal "breaking changes found" run is never mislabeled>= 2= a genuine error (load/parse/etc.) → surfacedThe disallowed-external-ref remedy (exit
123, from #128) is layered on top, so that case gets both the raw message and the actionable "setallow-external-refs: true" hint.Test
New
test-error-annotation.yamldrives thevalidate(standalone shape) andchangelog(early-exit shape) entrypoints with a stubbed oasdiff at exit102/123/1, asserting:102→ a::error::annotation is emitted123→ theallow-external-refs: trueremedy is emitted1→ no annotation (fail-on is not an error)This is the only way to assert annotation presence/absence (it isn't visible via step
outcome/outputs). Verified locally: all six assertions pass.Intended to land before the
v0.0.51action release so that release carries the full error-UX change.