Skip to content

benches: add mlxQ exact-statevector baseline (Apple silicon) to the third-party Pauli benchmark - #183

Open
BoltzmannEntropy wants to merge 1 commit into
Algorithmiq:mainfrom
BoltzmannEntropy:bench/mlxq-statevector-baseline
Open

benches: add mlxQ exact-statevector baseline (Apple silicon) to the third-party Pauli benchmark#183
BoltzmannEntropy wants to merge 1 commit into
Algorithmiq:mainfrom
BoltzmannEntropy:bench/mlxq-statevector-baseline

Conversation

@BoltzmannEntropy

Copy link
Copy Markdown

What this adds

An exact statevector baseline for the third-party Pauli benchmark: mlxQ, an MLX-based simulator for Apple-silicon GPUs, wired in as a mlxq backend following the existing per-backend pattern in benches/third_party/pauli_prop/backends.py (lazy imports inside the function, one Trotter step + expectation value per timed point, same timer bracket).

Pauli propagation's natural competitor at small-to-medium sizes is exact simulation, and none of the current backends provides that reference. Wherever the 2^n statevector fits in memory its per-step cost is independent of operator growth, so this backend locates the lattice size below which propagation is not worth running — and, symmetrically, makes the size where propagation wins unarguable.

Changes:

  • backends.py: run_mlxq() + registration (MEMORY_METRICS, LABELS, THREAD_VARS, new APPLE_BACKENDS group).
  • run_model.py / run_one.py: dispatch and --backends/--backend choices.
  • run_one.py: _num_threads now falls back to os.cpu_count()os.sched_getaffinity does not exist on macOS, so run_one.py/run_scaling.py previously could not run there for any backend.
  • benches/third_party/README.md: install/usage note.

Measured (your committed TFIM scenario, resized)

Apple M1 Max (32 GB), macOS 26.3, monoprop 0.8.0 PyPI wheel, medians over ≥3 dedicated-process runs, per-point timing identical to _run_steps:

Grid Qubits mlxQ (Metal) monoprop Faster
4x4 16 0.027 s 1.795 s mlxQ ~66x
4x5 20 0.062 s 2.302 s mlxQ ~37x
5x5 25 0.807 s 2.770 s mlxQ ~3.4x
4x7 28 49.8 s 2.121 s monoprop ~23x
6x6 36 ~0.5 TB, infeasible 3.604 s monoprop only

Full report with raw per-run data, the kicked-Ising counterpart, and fairness caveats: https://github.com/BoltzmannEntropy/Qupertino/tree/main/bench/monoprop_compare.

Correctness

Cross-validated against monoprop at 3x3/3x4 via run_model.py: expectation values agree to ~2.6e-5 (your lower_atol=1e-6 truncation plus mlxQ's complex64). Independently, both engines match a dense-matrix expm reference (monoprop to ~1e-14, mlxQ to ~1e-6).

Notes for maintainers

  • Dependencies: mlxq is intentionally not added to the uv project — MLX wheels only exist for macOS/arm64 and the current pyproject.toml pins CUDA packages that can't resolve there anyway. Happy to add it behind a sys_platform == "darwin" marker instead if you prefer lockfile-managed installs.
  • Docs aside (can file as a separate issue): while driving the native API we found ExpGate's parameter p acts as exp(−i·p·P) — back-propagating Z through an X gate with parameter p returns cos(2p) — while the README quick-example comments read as exp(−i·θ/2·P). The library is self-consistent; a clarifying line in the docstring would save independent users a debugging cycle.
  • Formatting checked with ruff format/ruff check (the one I001 on backends.py imports fires on the untouched file as well outside the prek context).
  • CLA: acknowledged, ready to sign the Individual CLA when the bot prompts.

…mark

Adds an exact (non-propagation) baseline to benches/third_party/pauli_prop:
mlxQ, an MLX statevector simulator for Apple-silicon GPUs. Wherever the 2^n
statevector fits, its per-step cost is independent of operator growth, so it
locates the lattice size below which Pauli propagation is not worth running
(~25-28 qubits against monoprop on an M1 Max, 32 GB).

- backends.py: run_mlxq() following the existing per-backend pattern (lazy
  imports, one Trotter step + expectation per timed point); registered in
  MEMORY_METRICS/LABELS/THREAD_VARS and a new APPLE_BACKENDS group.
- run_model.py / run_one.py: dispatch + choices for the new group. Also make
  _num_threads portable to macOS (os.sched_getaffinity does not exist there).
- README: install and usage note; the backend is deliberately not in the uv
  project's dependencies since MLX wheels only exist for Apple silicon.

Cross-validated against monoprop at 3x3/3x4: expectation values agree to
~2.6e-5 (monoprop's lower_atol=1e-6 truncation plus mlxQ's complex64).
@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.44%. Comparing base (3ca8c4e) to head (0ff4e8b).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #183      +/-   ##
==========================================
- Coverage   87.56%   87.44%   -0.12%     
==========================================
  Files          59       59              
  Lines        4239     4239              
  Branches     1497     1497              
==========================================
- Hits         3712     3707       -5     
- Misses        163      164       +1     
- Partials      364      368       +4     
Flag Coverage Δ
cpp 85.88% <ø> (-0.15%) ⬇️
python 95.31% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant