Skip to content

fix(perf)!: unify local and release performance workflows - #210

Merged
acgetchell merged 1 commit into
mainfrom
fix/performance-workflow
Aug 12, 2026
Merged

fix(perf)!: unify local and release performance workflows#210
acgetchell merged 1 commit into
mainfrom
fix/performance-workflow

Conversation

@acgetchell

@acgetchell acgetchell commented Aug 12, 2026

Copy link
Copy Markdown
Owner
  • Retain local Markdown, CSV, and provenance artifacts while allowing same-version comparisons of tracked changes.
  • Exclude untracked files explicitly and isolate the narrowed non-exact comparison bundle.
  • Promote distinct-release artifacts through performance-doc and keep performance-release atomic and rollback-capable.
  • Render one consolidated table per benchmark suite and reject invalid release publication states.

BREAKING CHANGE: rename performance-rerender to performance-doc and performance-local-vs-linalg to performance-local-non-exact, replace --rerender with --promote-artifacts, and remove performance-archive-published.

Summary by CodeRabbit

  • New Features

    • Added a dedicated performance-doc workflow for generating and promoting documentation-ready reports.
    • Added retained CSV, JSON, and provenance artifacts for repeatable performance comparisons.
    • Added support for local same-version comparisons and non-exact scratch reports.
    • Consolidated benchmark results into clearer suite-based tables with a shared Case column.
  • Documentation

    • Expanded benchmarking, performance, release, and contribution guidance.
    • Clarified artifact validation, promotion behavior, tag checks, and tracked-file handling.
  • Bug Fixes

    • Prevented invalid same-version release promotions and improved command validation.

- Retain local Markdown, CSV, and provenance artifacts while allowing same-version comparisons of tracked changes.
- Exclude untracked files explicitly and isolate the narrowed non-exact comparison bundle.
- Promote distinct-release artifacts through performance-doc and keep performance-release atomic and rollback-capable.
- Render one consolidated table per benchmark suite and reject invalid release publication states.

BREAKING CHANGE: rename performance-rerender to performance-doc and performance-local-vs-linalg to performance-local-non-exact, replace --rerender with --promote-artifacts, and remove performance-archive-published.
@acgetchell acgetchell self-assigned this Aug 12, 2026
@acgetchell
acgetchell merged commit 58cccfc into main Aug 12, 2026
12 of 13 checks passed
@acgetchell
acgetchell deleted the fix/performance-workflow branch August 12, 2026 20:43
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0d65ddfe-a705-4a36-af84-552dda1dd9f2

📥 Commits

Reviewing files that changed from the base of the PR and between a94e698 and c601c41.

📒 Files selected for processing (14)
  • CONTRIBUTING.md
  • docs/BENCHMARKING.md
  • docs/PERFORMANCE.md
  • docs/RELEASING.md
  • justfile
  • scripts/README.md
  • scripts/archive_performance.py
  • scripts/bench_compare.py
  • scripts/check_docs_version_sync.py
  • scripts/performance_artifacts.py
  • scripts/tests/test_archive_performance.py
  • scripts/tests/test_bench_compare.py
  • scripts/tests/test_check_docs_version_sync.py
  • scripts/tests/test_performance_artifacts.py

📝 Walkthrough

Walkthrough

The PR updates performance workflows to retain validated CSV and provenance artifacts, support separate local and documentation-only commands, validate release identifiers, promote reports without rerunning benchmarks, and render consolidated suite-level comparison tables.

Changes

Performance artifact workflow

Layer / File(s) Summary
Workflow commands and documentation
CONTRIBUTING.md, docs/BENCHMARKING.md, docs/PERFORMANCE.md, docs/RELEASING.md, justfile, scripts/README.md, scripts/check_docs_version_sync.py
The workflows use performance-local-non-exact and performance-doc. Documentation describes retained CSV and provenance artifacts, tracked-file handling, validation, and promotion.
Artifact generation and promotion
scripts/archive_performance.py, scripts/performance_artifacts.py, scripts/tests/test_archive_performance.py, scripts/tests/test_performance_artifacts.py
Generation writes validated artifacts and Markdown. Promotion reloads artifacts without rerunning benchmarks. Same-version local comparisons are supported, while same-version promotion is rejected.
Suite-level comparison rendering
scripts/bench_compare.py, scripts/tests/test_bench_compare.py
Snapshot and comparison output now uses one table per suite with a shared Case column. Tests validate the consolidated format and retained-artifact workflow text.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant archive_performance
  participant performance_artifacts
  participant bench_compare
  participant ReleaseArchive
  CLI->>archive_performance: Generate comparison
  archive_performance->>performance_artifacts: Export CSV and provenance JSON
  archive_performance->>performance_artifacts: Reload and validate artifacts
  performance_artifacts->>bench_compare: Provide comparison inputs
  bench_compare-->>archive_performance: Render Markdown report
  CLI->>archive_performance: Promote retained artifacts
  archive_performance->>ReleaseArchive: Archive promoted report
Loading

Possibly related issues

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/performance-workflow

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.88%. Comparing base (a94e698) to head (c601c41).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #210   +/-   ##
=======================================
  Coverage   97.88%   97.88%           
=======================================
  Files           8        8           
  Lines        5007     5007           
=======================================
  Hits         4901     4901           
  Misses        106      106           
Flag Coverage Δ
unittests 97.88% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant