Evaluate a feature-flagged Rust accelerator build by wbarnha · Pull Request #749 · faust-streaming/faust

wbarnha · 2026-08-04T11:24:28Z

Assess whether faust should grow an optional, USE_RUST-gated Rust extension alongside the existing Cython ones, and record the result.

Recommendation: do not add a Rust build axis. That was the original conclusion and the additional work below reinforces it, though for a sharper reason than first recorded.

Original evaluation

A working USE_RUST build was prototyped (setuptools-rust + a PyO3 port of the Cython HoppingWindow) and benchmarked:

Rust beats Cython only where a call does real work per crossing: ranges() faster, but current()/stale()/earliest() are not, because the PyO3 call boundary is thicker than Cython's.
The remaining Cython modules (streams, conductor) are async dispatch code whose bodies are calls back into Python — the workload where that boundary cost is worst.
Every batch-shaped part of faust is already served by someone else's Rust: orjson, ciso8601, rocksdict (PyO3).

The document also records the integration details a future PR would need (MANIFEST.in misses *.rs, [build-system] requires cannot be conditional, RustExtension(optional=True) for graceful degradation, a worker-banner row), the CI cost, and the concrete triggers that would change the answer.

Added since: the trigger fired, and Rust still lost

§6 said the answer would change if "a batch-shaped hot path appears in faust's own code". One has — first_consecutive_run, which Consumer._new_offset calls per partition per commit to scan the sorted acked-offset list. At a busy partition that is one boundary crossing followed by 600k iterations, blocking the event loop.

It was ported to Rust four ways. At 100k offsets, against Cython's 549 µs:

implementation	time	vs cython
python	2775 µs	0.20x
cython (ships today)	549 µs	1.00x
rust, idiomatic PyO3, abi3	1793 µs	0.31x
rust, idiomatic PyO3, no abi3	1184 µs	0.46x
rust, raw `pyo3::ffi`, abi3	831 µs	0.66x
rust, raw ffi + CPython macros, no abi3	648 µs	0.85x
rust, same scan over a native `Vec<i64>`	83 µs	6.8x

Rust at its best is still 15% slower, and that best case is unsafe raw-pointer code that has given up both memory safety and abi3. The last row explains the rest: the loop is batch-shaped, but every element is a PyObject, so each iteration is three C-API calls Rust pays for identically. Handed the same data natively, the identical scan is 6.8x faster than Cython.

So "batch-shaped" was the wrong test. §6's trigger is rewritten around whether per-element work is native or Python-object-bound, with a practical check: count the C-API calls per element.

Two corollaries: abi3 forbids the fastest variant (PyList_GET_ITEM is not in the limited API), so abi3 and performance are in direct tension; and §3.1's HoppingWindow port is now on-branch and re-measured, doing better than the original off-branch one (mostly lto + codegen-units = 1).

Added since: CPU and memory across Python, PyPy, Cython and Rust

The measurements above compare accelerators inside CPython, on time only. extra/tools/bench_accel_matrix.py adds PyPy — the runtime faust already tests, whose CI leg runs pure Python with USE_CYTHON: 'false' — and memory. Every cell runs in a fresh subprocess.

CPU, first_consecutive_run:

offsets	python	pypy	cython	rust
10 000	270.5 µs	59.4 µs	54.0 µs	63.7 µs
100 000	2729.8 µs	1307.7 µs	555.5 µs	684.0 µs
600 000	16843.6 µs	6158.7 µs	3416.7 µs	3965.8 µs

CPU, HoppingWindow.ranges per call:

case	python	pypy	cython	rust
`ranges(ts)`	1435–1482 ns	162–198 ns	732–744 ns	311–317 ns

On ranges — the one case where Rust clearly beat Cython, and so the strongest reason to consider it — PyPy running the pure-Python code beats Rust, at roughly 4x Cython. It has no call boundary to pay, so it JITs ranges together with its caller. PyPy does not win the offset scan, where allocation and GC traffic dominate.

Memory, peak RSS of a fresh process (MiB):

offsets	impl	after import	input list	peak
100 000	python	16.5	3.9	22.7
100 000	pypy	112.7	1.1	135.8
100 000	cython	16.5	3.9	22.7
100 000	rust	16.7	3.9	23.0
600 000	python	16.5	23.0	53.2
600 000	pypy	112.8	5.0	244.8
600 000	cython	16.5	22.9	53.2
600 000	rust	16.6	23.0	53.4

On CPython the accelerator is invisible — all three land within 0.3 MiB at every size, because all are dominated by the Python objects the data is made of. Memory supports no case either way. PyPy trades a 112 MiB interpreter footprint for a 4.6x more compact list of ints, but peaks far higher on GC headroom.

Caveats stated in the document: PyPy here is Python 3.9 against CPython 3.11, and it runs the pure-Python implementations because cpyext makes C extensions slower on PyPy than the Python they replace — which is why the CI leg disables Cython in the first place.

What is shipped here

Documentation, a prototype crate and three benchmark scripts. No build changes: the crate is not wired into setup.py, because PEP 518 has no conditional build-requires and wiring it would make setuptools-rust a mandatory build dependency for everyone. faust/_rust/ has no __init__.py, so find_packages() cannot pull it into the wheel. Build it with extra/tools/build_rust_accel.sh when re-checking the numbers.

This branch also carries the three Cython accelerators from #751, since first_consecutive_run is what §3.3 measures against.

🤖 Generated with Claude Code

https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

Assess whether faust should grow an optional, USE_RUST-gated Rust extension alongside the existing Cython ones, and record the result. A working USE_RUST build was prototyped against a clone of the repo (setuptools-rust + a PyO3 port of the Cython HoppingWindow) and benchmarked three ways. Findings: - Rust beats Cython only where a call does real work per crossing: ranges() 1.7x faster, but current()/stale()/earliest() are 13-17% *slower*, because the PyO3 call boundary costs 79ns against Cython's 59ns. - The two remaining Cython modules (streams, conductor) are async dispatch code whose bodies are calls back into Python -- the workload where that boundary cost is worst. - Every batch-shaped part of faust is already served by someone else's Rust: orjson, ciso8601, rocksdict (PyO3). So the recommendation is not to add a Rust build axis now. The document also records the integration details a future PR would need (MANIFEST.in misses *.rs, [build-system] requires cannot be conditional, RustExtension(optional=True) for graceful degradation, a worker-banner row), the CI cost (the pytest matrix doubles to 20 legs), and the concrete triggers that would change the answer. Ship the benchmark used, so the numbers can be re-checked on other hardware; it skips any implementation it cannot import. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9i6CXVTRGNRwEvzzCSP1B

codecov · 2026-08-04T11:26:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.06%. Comparing base (803c7a4) to head (588d1ac).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #749   +/-   ##
=======================================
  Coverage   96.06%   96.06%           
=======================================
  Files         103      103           
  Lines       11072    11094   +22     
  Branches     1191     1193    +2     
=======================================
+ Hits        10636    10658   +22     
  Misses        345      345           
  Partials       91       91

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Faust already ships optional Cython implementations of the window types, the stream iterator and the topic conductor. This extends that to three more hot paths, found by profiling the per-message and per-commit work. Each follows the existing pattern: the pure-Python implementation stays and is used whenever the extension could not be built or NO_CYTHON is set, so this is purely additive. faust/utils/_cython/functional.pyx first_consecutive_run(), a dedicated helper for the one thing Consumer._new_offset actually needs from consecutive_numbers(): the first run. The groupby() version builds a tuple, calls a Python key function and creates a group generator per acked offset, and runs once per assigned partition on every commit. Because it blocks the event loop, the win shows up as latency: for 100k un-committed offsets, 2.71ms -> 0.55ms. faust/transport/_cython/scheduler.pyx A C-level cursor replacing the round-robin generator returned by DefaultSchedulingStrategy.records_iterator, which also takes over TopicBuffer's inner per-partition generator. This runs once per record fetched from the broker: ~300-490ns -> ~50-70ns. faust/sensors/_cython/base.pyx A cdef base class carrying the four sensor hooks that fire on every message; SensorDelegate subclasses it and keeps its ~20 occasional hooks in plain Python. on_stream_event_in, which builds a dict per event, goes 374ns -> 182ns. Behaviour is preserved rather than approximated. Both scheduler implementations re-read the topic index and each topic's buffer map at every pass and pop drained entries from them, so mutating either mid-iteration gives the same result; a TopicBuffer subclass is still driven through next(); map_from_records and records_iterator remain overridable. first_consecutive_run stops consuming as soon as the run ends, so a shared iterator is left where the pure-Python version leaves it, and it re-reads the list length while scanning because a custom __sub__ can resize the list underneath it. The sensor set is read live rather than snapshotted, so direct mutation behaves identically. Tests run every new code path against both implementations, including a randomised differential test over 200 scheduler topologies. The full suite passes with and without NO_CYTHON. extra/tools/benchmark_cython.py times each accelerator against its pure-Python counterpart in the same interpreter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

docs/proposals/rust-acceleration.md said the recommendation would change if "a batch-shaped hot path appears in Faust's own code". One has: first_consecutive_run, which Consumer._new_offset calls once per assigned partition per commit to scan the sorted list of acked offsets. At a busy partition that is one boundary crossing followed by 600k iterations, and it blocks the event loop while it runs. So it was ported to Rust and measured. Rust lost, and the reason sharpens the whole evaluation. At 100k offsets, against the Cython that ships today (549us): python 2775us 0.20x rust, idiomatic PyO3, abi3 1793us 0.31x rust, idiomatic PyO3, no abi3 1184us 0.46x rust, raw pyo3::ffi, abi3 831us 0.66x rust, raw pyo3::ffi + CPython macros 648us 0.85x rust, same scan over a native Vec<i64> 83us 6.8x Rust at its best is still 15% slower, and that best case is unsafe raw-pointer code that has given up both memory safety and abi3 -- the two reasons to prefer Rust in the first place. The last row explains the rest: the loop is batch-shaped, but every element is a PyObject, so each iteration is three C-API calls that Rust makes at the same price Cython does. Handed the same data natively, the identical scan is 6.8x faster than Cython. Being batch-shaped was therefore the wrong test. Section 6's trigger is rewritten: what matters is whether the per-element work is native or Python-object-bound, and the practical check before porting anything is to count the C-API calls per element. Two smaller corrections fall out of it: - abi3 and performance are in direct tension. The limited API does not expose PyList_GET_ITEM, so the fastest variant cannot be compiled in an abi3 build: best-with-abi3 is 0.66x, best-without is 0.85x. - Section 3.1's HoppingWindow port is now on-branch and re-measured. It does better than the original off-branch one (mostly lto plus codegen-units = 1): ranges is 2.26x rather than 1.71x, and the three scalar methods are a wash rather than a 0.83-0.87x loss. The conclusion is unchanged. The crate is not wired into setup.py, for the reason the document already gives: PEP 518 has no conditional build-requires, so wiring it would make setuptools-rust a mandatory build dependency for everyone. It builds via extra/tools/build_rust_accel.sh instead, and faust/_rust/ has no __init__.py so find_packages() cannot pull it into the wheel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

The measurements so far compared accelerators inside CPython and only on time. That left out the runtime faust already supports and tests -- PyPy, whose CI leg runs the pure-Python code with USE_CYTHON: 'false' -- and left out memory entirely. extra/tools/bench_accel_matrix.py adds both. Every cell runs in a fresh subprocess, so no implementation's allocations can leak into another's reading, and the PyPy rows are produced by shelling out to pypy3. The PyPy column changes the argument. On HoppingWindow.ranges -- the one case in this whole evaluation where Rust clearly beat Cython, and so the strongest reason to consider it -- PyPy running the pure-Python version is faster than Rust: 162-198 ns against 311 ns, roughly 4x Cython. It has no call boundary to pay, so it JITs ranges together with its caller while both native extensions pay a crossing per call. PyPy does not win the offset scan (0.46-0.90x), where allocation and GC traffic dominate. Memory turns out to support no case at all: on CPython, python, cython and rust land within 0.3 MiB of each other at every size, because all three are dominated by the Python objects the data is made of. PyPy trades a 112 MiB interpreter footprint (6.8x CPython) for a 4.6x more compact list of ints, but peaks far higher (244.8 vs 53.2 MiB at 600k) on GC headroom. Section 3.4 records the tables and the caveats: PyPy here is Python 3.9 against CPython 3.11, and it runs the pure-Python implementations because cpyext makes C extensions slower on PyPy than the Python they replace -- which is why the CI leg disables Cython in the first place. Sections 4 and 6 are updated where the PyPy result changes a verdict. Also corrects the windows figures to ranges rather than point values: PyPy's JIT is the one column with real run-to-run spread. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

The copartitioned-assignor property tests run on PyPy since the PyPy skips were removed. They carried a fixed 4000ms Hypothesis deadline, but PyPy's JIT warmup makes a single example's timing vary by over a second, so runs intermittently exceed it and Hypothesis reports DeadlineExceeded / FlakyFailure -- failing the (non-required) PyPy leg and turning master CI red even though the assignment produced is valid. These tests assert assignment correctness, not performance, so set deadline=None (as the DeadlineExceeded message itself recommends). The property assertions are unchanged and the job timeout still guards against a real hang. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHPL4VFWQRQPpjR1gXSKyL

Follows the "faster code via static typing" guidance: declare return types on cdef functions, and type parameters and locals rather than leaving them implicit. The substantive change is on the two _start_pass methods, which had no declared return type. They are now `cdef int ... except -1`. Both call into Python (dict.pop, .items(), the _TopicCursor constructor) and so can raise, and the except clause states that rather than depending on a compiler default -- Cython 3 propagates from a bare cdef, but Cython 0.x swallowed unless an except clause was given, and legacy_implicit_noexcept restores that behaviour. Verified both ways: an exception raised inside _start_pass now propagates identically through the Cython and pure-Python iterators. Also types the len() results as Py_ssize_t instead of letting them box into Python ints, and types the two sensor-delegate parameters that were bare. Note this is a correctness and clarity change, not a speed one: the scheduler benchmark is unmoved at 5.8-7.7x over pure Python, because the loops were already fully typed. Parameters that face callers are deliberately left as `object` -- typing them to a builtin would add an exact-type runtime check, which is what made an OrderedDict fail against a `cdef dict` local earlier in this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

Reported in review of #751. _start_pass skipped rebuilding its snapshot when the mapping's length was unchanged since the last pass. That check cannot tell "unchanged" from "swapped": replacing index["foo"] with a new TopicBuffer under the same key leaves the length identical, so the shortcut was taken, the stale cursor kept being used, and every record in the replacement was silently dropped. The `cursor.source is not buffer` check that exists to catch exactly this lived inside the rebuild, which the shortcut skipped. Reproduced against the pure-Python iterator, which re-reads the mapping every pass and so gets it right: python: [1, 10, 11] cython: [1, 2] <- 10 and 11 never delivered The same flaw was present one level down in _TopicCursor, where an iterator swapped in under an existing TP left the partition cursor stale. The report only named the topic level; both are fixed. Both shortcuts are replaced by _snapshot_is_current(), which compares the cached snapshot against the live mapping entry by entry, by identity. It walks the mapping but allocates nothing, which is what still makes reusing the snapshot worthwhile. This costs real throughput, and the PR body is updated rather than left overstating it: the scheduler was 5.7-7.7x over pure Python and is now 2.2-2.8x. A pass must consult the live mapping to be correct, and a single-partition topic reaches a pass boundary on every record. An unconditional rebuild was measured too, at 1.6-2.1x, so the identity check is worth keeping over simply rebuilding. Adds regression tests for both levels, run against both implementations. Swapping in a topic under a *different* name is deliberately not covered: the pure-Python generator iterates the live dict, so CPython raises "dictionary keys changed during iteration" -- undefined behaviour in Python itself rather than a guarantee to pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019oX4oGCQGgvPJHabjwfaBk

claude and others added 8 commits August 4, 2026 12:19

Merge Cython accelerators into the Rust evaluation branch

8eb39f9

Merge master into claude/faust-cython-rewrites-vwltqs

f31e9a2

Merge master into the Rust evaluation branch

5b53d71

Merge the PyPy hypothesis-deadline fix from the Cython branch

4f99a2d

wbarnha mentioned this pull request Aug 4, 2026

Add three Cython accelerators: offset commit, record scheduler, sensor fan-out #751

Open

claude added 4 commits August 4, 2026 13:43

Merge the Cython static-typing tightening

ecf86fd

Merge the scheduler buffer-replacement fix

588d1ac

wbarnha mentioned this pull request Aug 4, 2026

Disable the Hypothesis deadline on the copartitioned-assignor property tests #752

Open

Navigation Menu

Uh oh!

Evaluate a feature-flagged Rust accelerator build - #749

Evaluate a feature-flagged Rust accelerator build#749
wbarnha wants to merge 13 commits into
masterfrom
claude/rust-faust-feature-flags-haoq0k

wbarnha commented Aug 4, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

wbarnha commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Original evaluation

Added since: the trigger fired, and Rust still lost

Added since: CPU and memory across Python, PyPy, Cython and Rust

What is shipped here

Uh oh!

codecov Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wbarnha commented Aug 4, 2026 •

edited

Loading

codecov Bot commented Aug 4, 2026 •

edited

Loading