Skip to content

Commit a18ffa9

Browse files
Add CI profiling workflow
Run the profiling tool on every PR with 3 runs per benchmark for statistical significance. Posts a text summary to the GitHub step summary and uploads flamegraph SVGs as downloadable artifacts. Uses the --auto benchmarks (linked_list, array_ops, structs) which exercise pointer analysis, array encoding, and struct handling respectively. Solver time is excluded to focus on CBMC's own code. Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
1 parent c6fe667 commit a18ffa9

1 file changed

Lines changed: 104 additions & 0 deletions

File tree

.github/workflows/profiling.yaml

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# CBMC Profiling Workflow
2+
#
3+
# Runs the CBMC profiling tool on every PR to detect performance changes
4+
# in the pre-solver pipeline (parsing, type-checking, goto conversion,
5+
# instrumentation, symbolic execution, and propositional encoding).
6+
#
7+
# What it does:
8+
# 1. Builds CBMC in Release mode (fast, symbols not stripped)
9+
# 2. Runs 3 built-in benchmarks (linked_list, array_ops, structs) 3 times
10+
# each for statistical significance
11+
# 3. Generates flamegraphs and an aggregated hotspot analysis
12+
# 4. Posts the text summary to the GitHub step summary
13+
# 5. Uploads flamegraph SVGs and results as downloadable artifacts
14+
#
15+
# The benchmarks exercise different CBMC subsystems:
16+
# - linked_list: pointer analysis + symbolic execution
17+
# - array_ops: array flattening + propositional encoding
18+
# - structs: type checking + goto conversion
19+
#
20+
# Solver time is excluded (--dimacs --outfile /dev/null) so that results
21+
# reflect only CBMC's own code, not the SAT solver.
22+
#
23+
# To run locally:
24+
# scripts/profile_cbmc.py --auto --runs 3
25+
#
26+
# For more thorough profiling with source locations:
27+
# cmake -S . -Bbuild-debug -DCMAKE_BUILD_TYPE=RelWithDebInfo -DWITH_JBMC=OFF
28+
# cmake --build build-debug --target cbmc -j$(nproc)
29+
# scripts/profile_cbmc.py --auto-large --auto-csmith --runs 3 \
30+
# --debug-binary build-debug/bin/cbmc
31+
32+
name: Profiling
33+
on:
34+
pull_request:
35+
branches: [ develop ]
36+
37+
jobs:
38+
profile:
39+
runs-on: ubuntu-24.04
40+
steps:
41+
- name: Check out repository
42+
uses: actions/checkout@v6
43+
with:
44+
submodules: recursive
45+
fetch-depth: 0
46+
47+
- name: Install dependencies
48+
env:
49+
DEBIAN_FRONTEND: noninteractive
50+
run: |
51+
sudo apt-get update
52+
sudo apt-get install --no-install-recommends -yq \
53+
cmake ninja-build gcc g++ flex bison ccache \
54+
linux-tools-common linux-tools-$(uname -r) || true
55+
# perf requires perf_event_paranoid=-1 for call graph recording
56+
sudo sysctl kernel.perf_event_paranoid=-1
57+
58+
- name: Prepare ccache
59+
uses: actions/cache/restore@v5
60+
with:
61+
path: .ccache
62+
key: ${{ runner.os }}-24.04-Release-profile-${{ github.sha }}
63+
restore-keys: |
64+
${{ runner.os }}-24.04-Release-profile
65+
${{ runner.os }}-24.04-Release
66+
- run: echo "CCACHE_DIR=$PWD/.ccache" >> $GITHUB_ENV
67+
68+
- name: Build CBMC (Release)
69+
run: |
70+
cmake -S . -Bbuild -G Ninja -DWITH_JBMC=OFF \
71+
-DCMAKE_BUILD_TYPE=Release \
72+
-DCMAKE_C_COMPILER=/usr/bin/gcc \
73+
-DCMAKE_CXX_COMPILER=/usr/bin/g++
74+
touch build/src/{ansi-c,cpp}/library-check.stamp
75+
ninja -C build cbmc -j$(nproc)
76+
77+
# Run each benchmark 3 times for statistical significance.
78+
# The summary reports mean ± stddev for timings.
79+
- name: Run profiling (3 runs per benchmark)
80+
run: |
81+
python3 scripts/profile_cbmc.py --auto --runs 3 --timeout 120 \
82+
--output-dir profile-results
83+
84+
- name: Post summary
85+
if: always()
86+
run: |
87+
if [ -f profile-results/summary.txt ]; then
88+
echo '```' >> "$GITHUB_STEP_SUMMARY"
89+
cat profile-results/summary.txt >> "$GITHUB_STEP_SUMMARY"
90+
echo '```' >> "$GITHUB_STEP_SUMMARY"
91+
fi
92+
93+
# Flamegraphs are interactive SVGs — download and open in a browser.
94+
- name: Upload artifacts
95+
if: always()
96+
uses: actions/upload-artifact@v4
97+
with:
98+
name: profile-results
99+
path: |
100+
profile-results/*/flamegraph.svg
101+
profile-results/aggregated.svg
102+
profile-results/summary.txt
103+
profile-results/results.json
104+
retention-days: 14

0 commit comments

Comments
 (0)