Skip to content

Add hyperfine benchmarks for CLI prove command#296

Closed
gabrielbosio wants to merge 21 commits into
mainfrom
feat/bench-prove
Closed

Add hyperfine benchmarks for CLI prove command#296
gabrielbosio wants to merge 21 commits into
mainfrom
feat/bench-prove

Conversation

@gabrielbosio

@gabrielbosio gabrielbosio commented Feb 6, 2026

Copy link
Copy Markdown
Collaborator

Prove benchmarking infrastructure mirroring the existing execute benchmarks. Local script (scripts/bench_prove.sh), CI workflow triggered by bench-prove label or manual dispatch, and bench-prove Makefile target. Only comments on the PR when a regression >5% is detected.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from greptile-apps Bot Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 6, 2026
@gabrielbosio gabrielbosio added the bench-prove Run prove benchmarks on this PR label Feb 6, 2026
@gabrielbosio gabrielbosio added bench-prove Run prove benchmarks on this PR and removed bench-prove Run prove benchmarks on this PR labels Feb 6, 2026
@gabrielbosio gabrielbosio added bench-prove Run prove benchmarks on this PR and removed bench-prove Run prove benchmarks on this PR labels Feb 6, 2026
@gabrielbosio gabrielbosio removed the bench-prove Run prove benchmarks on this PR label Feb 6, 2026
@gabrielbosio gabrielbosio added the bench-prove Run prove benchmarks on this PR label Feb 6, 2026
@gabrielbosio gabrielbosio added bench-prove Run prove benchmarks on this PR and removed bench-prove Run prove benchmarks on this PR labels Feb 6, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from greptile-apps Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from claude Bot Feb 9, 2026
@yetanotherco yetanotherco deleted a comment from github-actions Bot Feb 9, 2026
@github-actions

github-actions Bot commented Feb 9, 2026

Copy link
Copy Markdown

Benchmark Results for unmodified programs 🚀

Command Mean [ms] Min [ms] Max [ms] Relative
base binary_search 82.5 ± 12.4 63.2 92.6 1.25 ± 0.21
head binary_search 66.2 ± 4.4 62.7 75.0 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base bitwise_ops 63.2 ± 0.6 62.4 64.0 1.00
head bitwise_ops 66.7 ± 3.9 62.2 71.6 1.05 ± 0.06
Command Mean [ms] Min [ms] Max [ms] Relative
base fibonacci_26 66.8 ± 1.8 65.9 71.9 1.00
head fibonacci_26 69.4 ± 2.7 66.3 73.1 1.04 ± 0.05
Command Mean [ms] Min [ms] Max [ms] Relative
base hashmap 141.1 ± 1.2 139.2 142.6 1.00
head hashmap 142.2 ± 1.6 139.8 144.5 1.01 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base keccak 137.0 ± 3.4 132.9 142.6 1.00
head keccak 139.2 ± 2.7 135.9 142.4 1.02 ± 0.03
Command Mean [ms] Min [ms] Max [ms] Relative
base matrix_multiply 71.2 ± 1.1 69.3 73.0 1.01 ± 0.03
head matrix_multiply 70.5 ± 2.0 67.9 73.3 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base modular_exp 64.3 ± 2.9 62.2 69.7 1.00 ± 0.06
head modular_exp 64.1 ± 2.4 62.2 69.5 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base quicksort 67.9 ± 2.1 66.3 71.5 1.00
head quicksort 70.0 ± 3.3 66.5 76.0 1.03 ± 0.06
Command Mean [ms] Min [ms] Max [ms] Relative
base sieve 71.3 ± 1.7 69.8 75.7 1.00
head sieve 72.5 ± 3.2 69.6 77.3 1.02 ± 0.05
Command Mean [ms] Min [ms] Max [ms] Relative
base sum_array 81.0 ± 2.6 77.9 84.7 1.01 ± 0.04
head sum_array 79.9 ± 2.3 77.2 83.6 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base syscall_commit 98.0 ± 0.7 97.3 98.9 1.00
head syscall_commit 99.0 ± 3.2 97.3 107.8 1.01 ± 0.03

@MauroToscano

Copy link
Copy Markdown
Contributor

Superseeded by: #332

@gabrielbosio gabrielbosio deleted the feat/bench-prove branch April 20, 2026 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench-prove Run prove benchmarks on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants