Skip to content

Collapse superseded benchmark comments rather than prefixing them - #202

Merged
ms609 merged 1 commit into
mainfrom
claude/benchmark-hide-outdated-comments
Aug 20, 2026
Merged

Collapse superseded benchmark comments rather than prefixing them#202
ms609 merged 1 commit into
mainfrom
claude/benchmark-hide-outdated-comments

Conversation

@ms609

@ms609 ms609 commented Aug 20, 2026

Copy link
Copy Markdown
Owner

GitHub can genuinely collapse a superseded comment — minimizeComment with
classifier: OUTDATED — rather than leaving it at full height behind a ⚠️
banner. The benchmark comment step now does that.

What changed

  • The listing moves from issues.listComments to GraphQL, because REST's
    comment payload omits isMinimized. Minimizing no longer edits the body, so
    that flag is the only remaining idempotency guard; without it every run would
    re-minimize the entire history.
  • The listing and each mutation warn rather than throw. This step exists to post
    the new result, and losing that to a hiccup while tidying up the old ones
    would be a poor trade.

Worth knowing

  • Comments already carrying the old ⚠️ banner keep it, and are not minimized,
    so the first run under this code collapses them banner and all — one slightly
    ugly comment per open PR, then it is clean.
  • Fork PRs still get a read-only token and the whole step no-ops there, exactly
    as before.
  • This job declares no permissions: block, so its token comes from the
    repository default. minimizeComment needs write access to the comment; if
    the default proves too narrow, the run log says so and the benchmark comment
    still posts.

No NEWS.md / DESCRIPTION bump: CI plumbing is not user-visible package
behaviour.

Companion PRs apply the same change to the other repos carrying this script
(ms609/Coreset, ms609/TreeTools, ms609/TreeDist). TreeSearch's
agent-benchmark.yml is dispatch-only and uploads artifacts without commenting,
so it needs nothing.


Agent work, filed under @ms609's own account because the ms609-agent account
is currently suspended — so it has not had the independent review that
agent-authored PRs normally get.

🤖 Generated with Claude Code

The step already tracked which of its own comments were stale; it just had
no way to act on it beyond prepending a warning banner, which left every
old result at full height. GitHub's `minimizeComment` mutation with
`classifier: OUTDATED` is the real thing -- the comment folds into the
collapsed "marked as outdated" state.

Listing moves to GraphQL out of necessity, not taste: REST's comment
payload omits `isMinimized`, and since minimizing no longer touches the
body, that flag is the only remaining idempotency guard. Without it every
run would re-minimize the whole history.

Neither half may throw. The step exists to post the new result, and losing
that to a hiccup in tidying up the old ones would be a poor trade -- so a
failed listing or a failed collapse warns and the comment still lands. The
likeliest such failure is a token without the scope `minimizeComment`
needs. This job declares no `permissions:` block, so it runs on the
repository default; if the warning fires, that is where to look.

Comments already carrying the old banner keep it, and will be collapsed
banner-and-all on the next run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ms609
ms609 merged commit 6beac44 into main Aug 20, 2026
10 checks passed
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.55%. Comparing base (a8fc0b7) to head (3a712cc).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #202   +/-   ##
=======================================
  Coverage   95.55%   95.55%           
=======================================
  Files          56       56           
  Lines        5045     5045           
=======================================
  Hits         4821     4821           
  Misses        224      224           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant