-
Notifications
You must be signed in to change notification settings - Fork 27
Optimize statevector simulator preprocessing #316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Kenny-Heitritter
wants to merge
35
commits into
main
Choose a base branch
from
kenny/evolved-sv-preprocess-no-cache
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
4f481f4
dev
ryanhill1 798d194
1 & 2 qubit gate handling
ryanhill1 2683290
remove class attributes
ryanhill1 7bc59c2
Merge branch 'main' into sv
ryanhill1 ea3b1ff
add result class
ryanhill1 13e76ae
check-in
ryanhill1 1135970
check-in updates2
ryanhill1 a274785
add test cases
ryanhill1 5881c97
get swap gate a little closer - still not right tho
ryanhill1 064b922
Merge branch 'main' into sv
ryanhill1 e71fe3d
Optimize statevector simulator: gate fusion, native gate preprocessin…
ryanhill1 b7bbe46
Optimize statevector preprocessing with evolved no-cache path
Kenny-Heitritter b84a23e
Merge branch 'main' into kenny/evolved-sv-preprocess-no-cache
Kenny-Heitritter 0cfd83a
Add evolved no-cache statevector benchmark artifacts
Kenny-Heitritter 5afa323
Regenerate statevector benchmark plots
Kenny-Heitritter 9af158f
Rename bench_random_evolved_no_cache.png to bench_random_evolved.png
Kenny-Heitritter e7879ce
Rename bench_qft_evolved_no_cache.png to bench_qft_evolved.png
Kenny-Heitritter f07ebcd
Rename bench_evolved_no_cache.json to bench_evolved.json
Kenny-Heitritter 1d56076
Fix CI for statevector simulator PR
Kenny-Heitritter 734ec88
fix: repair CI for statevector simulator PR
ryanhill1 bc49a3d
build: use portable, compiler-aware compile flags
ryanhill1 b78c0e2
fix: make macOS wheels portable by disabling OpenMP there
ryanhill1 2b03019
fix: make OpenMP a compile-time option so macOS builds serial
ryanhill1 762dd67
Fix crz fast path in statevector simulator; add qiskit-free tests
ryanhill1 1caac11
Merge branch 'main' into kenny/evolved-sv-preprocess-no-cache
ryanhill1 9b6d3ef
Merge remote-tracking branch 'origin/main' into kenny/evolved-sv-prep…
ryanhill1 14e2b28
fix(simulator): resolve qubits by register, stop silently dropping st…
ryanhill1 5cf1f32
fix(build): validate kernel inputs, make OpenMP flag-only, stop shipp…
ryanhill1 c21394b
chore(ci,docs,benchmarks): coverage path, tox benchmark filter, runna…
ryanhill1 3c3a2ba
fix(review): phase-invariant statevector comparisons (CodeRabbit)
ryanhill1 6f06324
fix(lint): satisfy pylint in CI's installed-package environment
ryanhill1 dfb2b03
fix(review): validate --repeats, quote pytest path, setup.py doc nitp…
ryanhill1 a39d9a4
docs(benchmarks): annotate and docstring bench_simulator helpers (Cod…
ryanhill1 456e91c
fix(review): address Argus findings in sv_sim kernel dispatch and ben…
ryanhill1 3fc9c01
bench: regenerate bench_qft.png for the corrected QFT circuit
ryanhill1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # Benchmarks | ||
|
|
||
| Scripts and reference results for the PyQASM statevector simulator | ||
| (`pyqasm.simulator.Simulator`). | ||
|
|
||
| ## Contents | ||
|
|
||
| | File | What it is | | ||
| | --- | --- | | ||
| | `bench_simulator.py` | Benchmarks PyQASM against Qiskit Aer, Cirq, and PennyLane Lightning on shared random and QFT circuits, with cross-simulator correctness checks. | | ||
| | `plot_benchmarks.py` | Runs the same benchmarks and renders log-scale comparison plots (`bench_random.png`, `bench_qft.png`). | | ||
| | `profile_simulator.py` | Profiles the PyQASM simulator itself: cProfile, section timing (preprocess vs simulation vs post-processing), or per-kernel timing. | | ||
| | `bench_random.png`, `bench_qft.png` | Committed comparison plots produced by `plot_benchmarks.py`. | | ||
| | `bench_evolved.json`, `bench_qft_evolved.png`, `bench_random_evolved.png` | Historical before/after results for the evolved preprocessing change (see provenance note below). | | ||
|
|
||
| ## Requirements | ||
|
|
||
| All scripts need only `pyqasm` (installed from this repo) for the PyQASM | ||
| paths. Additional dependencies: | ||
|
|
||
| - `bench_simulator.py`: optionally `qiskit` + `qiskit-aer`, `cirq`, and | ||
| `pennylane` (Lightning) for the comparison columns, and `tabulate` for the | ||
| summary table. Every one of these is optional — missing simulators are | ||
| reported as `skipped: <name> not installed` and the script runs the | ||
| remaining backends (worst case, PyQASM only). | ||
| - `plot_benchmarks.py`: `matplotlib`, plus the same optional simulators as | ||
| `bench_simulator.py`. Plots include only the installed backends. | ||
| - `profile_simulator.py`: no extra dependencies. | ||
|
|
||
| None of these are declared as a package extra; install them ad hoc, e.g. | ||
| `pip install qiskit qiskit-aer cirq pennylane tabulate matplotlib`. | ||
|
|
||
| ## Running | ||
|
|
||
| From the repository root: | ||
|
|
||
| ```bash | ||
| # Full benchmark table (up to 22 qubits; minutes of runtime) | ||
| python benchmarks/bench_simulator.py | ||
|
|
||
| # Smoke test: small circuits, 2 repeats | ||
| python benchmarks/bench_simulator.py --quick | ||
|
|
||
| # Regenerate bench_random.png / bench_qft.png in benchmarks/ | ||
| python benchmarks/plot_benchmarks.py | ||
|
|
||
| # Quick plot run, writing PNGs somewhere else | ||
| python benchmarks/plot_benchmarks.py --quick --out-dir /tmp/bench-plots | ||
|
|
||
| # Profile the simulator (modes: section | cprofile | kernel) | ||
| python benchmarks/profile_simulator.py --mode section --qubits 16 --depth 200 | ||
| python benchmarks/profile_simulator.py --mode cprofile --qubits 12 --depth 100 | ||
| python benchmarks/profile_simulator.py --mode kernel --qubits 12 --depth 100 | ||
| ``` | ||
|
|
||
| Timing covers simulation only; circuit construction/transpilation is done in | ||
| each backend's `prepare_*` step and excluded from the measured time. Reported | ||
| numbers are the median of `N_REPEATS` runs (default 5; `--quick` uses 2, | ||
| `--repeats N` overrides). | ||
|
|
||
| The two committed plots were measured on different machines, so compare | ||
| backends within a plot, never timings across the two: `bench_qft.png` is from | ||
| an Apple M1 Pro (serial build — macOS disables OpenMP by default) and | ||
| `bench_random.png` predates it. Each plot is internally consistent: every | ||
| backend in it was measured in one run on one host. | ||
|
|
||
| ## Provenance of the `*_evolved` files | ||
|
|
||
| `bench_evolved.json`, `bench_qft_evolved.png`, and `bench_random_evolved.png` | ||
| were produced during development of the evolved no-cache preprocessing change | ||
| (PR #316). They compare two states of this branch: | ||
|
|
||
| - **baseline**: the branch state *before* the evolved-preprocessing commit | ||
| (the previous statevector preprocessing implementation); | ||
| - **candidate**: the branch with the evolved no-cache preprocessing. | ||
|
|
||
| They are retained for historical reference only. They are **not** regenerated | ||
| by any script in this tree: the baseline implementation no longer exists in | ||
| the working tree, so the before/after comparison cannot be reproduced from | ||
| the current sources alone. To reproduce something comparable, check out the | ||
| commit preceding the evolved-preprocessing commit and benchmark both | ||
| revisions with `bench_simulator.py`. Current-code performance can always be | ||
| re-measured with `bench_simulator.py` / `plot_benchmarks.py`, which | ||
| regenerate `bench_random.png` and `bench_qft.png`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,183 @@ | ||
| { | ||
| "benchmark": "statevector preprocessing no-cache comparison", | ||
| "baseline": "PyQASM sv branch before evolved preprocessing commit", | ||
| "candidate": "PyQASM sv branch with evolved no-cache preprocessing", | ||
| "n_repeats": 5, | ||
| "pyqasm_num_threads": 1, | ||
| "notes": [ | ||
| "No QASM string memoization is included in the candidate.", | ||
| "Both series use pre-unrolled QasmModule inputs, matching the branch benchmark convention." | ||
| ], | ||
| "series": { | ||
| "random": [ | ||
| { | ||
| "num_qubits": 4, | ||
| "depth": 20, | ||
| "pyqasm_sv_ms": 0.06092991679906845, | ||
| "evolved_no_cache_ms": 0.06546999793499708, | ||
| "speedup": 0.9306540204807044, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 6, | ||
| "depth": 40, | ||
| "pyqasm_sv_ms": 0.1580890966579318, | ||
| "evolved_no_cache_ms": 0.1151899341493845, | ||
| "speedup": 1.372421104546456, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 8, | ||
| "depth": 60, | ||
| "pyqasm_sv_ms": 0.16204908024519682, | ||
| "evolved_no_cache_ms": 0.14194007962942123, | ||
| "speedup": 1.1416724625509327, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 10, | ||
| "depth": 80, | ||
| "pyqasm_sv_ms": 0.24963996838778257, | ||
| "evolved_no_cache_ms": 0.21547905635088682, | ||
| "speedup": 1.1585347208002805, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 12, | ||
| "depth": 100, | ||
| "pyqasm_sv_ms": 0.458328053355217, | ||
| "evolved_no_cache_ms": 0.41990901809185743, | ||
| "speedup": 1.0914937131808757, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 14, | ||
| "depth": 150, | ||
| "pyqasm_sv_ms": 1.5235739992931485, | ||
| "evolved_no_cache_ms": 1.3638049131259322, | ||
| "speedup": 1.117149516495739, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 16, | ||
| "depth": 200, | ||
| "pyqasm_sv_ms": 7.1852849796414375, | ||
| "evolved_no_cache_ms": 5.937180016189814, | ||
| "speedup": 1.210218480835721, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 18, | ||
| "depth": 250, | ||
| "pyqasm_sv_ms": 35.15114798210561, | ||
| "evolved_no_cache_ms": 28.548280941322446, | ||
| "speedup": 1.231287728124666, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 20, | ||
| "depth": 300, | ||
| "pyqasm_sv_ms": 162.53185598179698, | ||
| "evolved_no_cache_ms": 126.72401999589056, | ||
| "speedup": 1.282565499319447, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 22, | ||
| "depth": 400, | ||
| "pyqasm_sv_ms": 1289.0826460206881, | ||
| "evolved_no_cache_ms": 1231.460615992546, | ||
| "speedup": 1.0467916141854843, | ||
| "correct": true | ||
| } | ||
| ], | ||
| "qft": [ | ||
| { | ||
| "num_qubits": 4, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 0.20883907563984394, | ||
| "evolved_no_cache_ms": 0.16430008690804243, | ||
| "speedup": 1.2710831720784765, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 6, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 0.4983479157090187, | ||
| "evolved_no_cache_ms": 0.315018929541111, | ||
| "speedup": 1.5819618091997316, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 8, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 0.9089269442483783, | ||
| "evolved_no_cache_ms": 0.5902479169890285, | ||
| "speedup": 1.5399070764789728, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 10, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 1.5004950109869242, | ||
| "evolved_no_cache_ms": 1.0477059986442327, | ||
| "speedup": 1.4321718238977499, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 12, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 2.5123610394075513, | ||
| "evolved_no_cache_ms": 1.782613922841847, | ||
| "speedup": 1.409369133279482, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 14, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 5.448751035146415, | ||
| "evolved_no_cache_ms": 4.14173596072942, | ||
| "speedup": 1.3155718005226993, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 16, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 18.029507948085666, | ||
| "evolved_no_cache_ms": 15.524596092291176, | ||
| "speedup": 1.1613511772482323, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 18, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 77.3717820411548, | ||
| "evolved_no_cache_ms": 68.92851099837571, | ||
| "speedup": 1.1224931587885028, | ||
| "correct": true | ||
| }, | ||
| { | ||
| "num_qubits": 20, | ||
| "depth": "QFT", | ||
| "pyqasm_sv_ms": 382.6392919290811, | ||
| "evolved_no_cache_ms": 349.38782698009163, | ||
| "speedup": 1.0951706452866319, | ||
| "correct": true | ||
| } | ||
| ] | ||
| }, | ||
| "summary": { | ||
| "random": { | ||
| "geomean_speedup": 1.1521318658620598, | ||
| "min_speedup": 0.9306540204807044, | ||
| "max_speedup": 1.372421104546456, | ||
| "all_correct": true | ||
| }, | ||
| "qft": { | ||
| "geomean_speedup": 1.3147632402156637, | ||
| "min_speedup": 1.0951706452866319, | ||
| "max_speedup": 1.5819618091997316, | ||
| "all_correct": true | ||
| } | ||
| } | ||
| } |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.