Skip to content

perf(atomsplit): fuse o200k letter case-split + 32-wide ASCII classify#2201

Open
ArthurZucker wants to merge 2 commits into
feat/train_encode_splitfrom
perf/o200k-classify
Open

perf(atomsplit): fuse o200k letter case-split + 32-wide ASCII classify#2201
ArthurZucker wants to merge 2 commits into
feat/train_encode_splitfrom
perf/o200k-classify

Conversation

@ArthurZucker

@ArthurZucker ArthurZucker commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Two byte-exact perf changes to atomsplit, both isolated to the scalar o200k FSM and the NEON classify kernel. No behavioural change — o200k_parity and the classify SIMD-vs-scalar parity tests pass, 0 warnings.

1. o200k letter case-split: one pass instead of two

fsm_o200k walked every [\p{L}\p{M}]+ run twiceletter_end() to find the run end, then emit_o200k_letters/o200k_letter_match re-scanned the same bytes for the case split. Since the tags already carry exact case (UpperLetter/LowerLetter/caseless), the run end can be discovered inline during the case split:

  • fold letter_end into the phase-A [UC]* / phase-B [LC]+ scans (each stops at a non-member = run end);
  • unroll those scans 16 tags/chunk with get_unchecked, same shape as run_end (a 16-wide scan already skips cont spans wholesale — a 48-wide variant was measured and lost);
  • resolve the give-back last-caseless target by a backward hop only on the no-lowercase exit, so the forward scans stay pure skips (no per-char tracking).

FSM throughput (ns/byte, min-of-7, byte-exact ✓ vs onig):

lang before after Δ
English 2.163 1.602 −26%
French 1.936 1.437 −26%
Russian 1.243 0.761 −39%
Greek 1.316 0.824 −37%
Arabic 1.539 1.104 −28%
Thai 1.076 0.636 −41%
Chinese 1.154 0.759 −34%
Japanese 1.126 0.764 −32%
Korean 1.746 1.371 −21%

Every corpus now beats logos on the full (classify + fsm) pipeline; the previous Chinese/Thai ties are gone.

2. classify: 32-wide ASCII fast path

The NEON kernel loads 32 bytes to produce 16 tags (it needs the b1/b2 lookahead), so non-ASCII bytes were effectively loaded twice. The ASCII fast path now peeks 32 bytes at once — one bounds check + one vmaxvq reduction covers two 16-byte stores — done in place. The multibyte body is completely untouched (same code, same comments). Real text is ~half ASCII (spaces / digits / punctuation), so this fires constantly.

classify throughput (ns/byte, byte-exact ✓ vs classify_scalar):

lang before after Δ
English 0.153 0.085 +45%
French 0.252 0.218 +14%
Russian 0.384 0.344 +10%
Chinese / Korean / Thai ~neutral

No regression on non-ASCII scripts. (A full 32-wide multibyte driver was prototyped too — it gains Chinese +17% but regresses 2-byte scripts −8 to −14%, so only the clean ASCII-widening is kept.)

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker
ArthurZucker force-pushed the perf/o200k-classify branch from 5c18f87 to 4ca03be Compare July 15, 2026 16:58
o200k: fold the per-run `letter_end` scan into the case-split so each
`[\p{L}\p{M}]+` run is walked once instead of twice, unroll phase-A/B
16 tags per chunk like `run_end`, and resolve the give-back last-C by a
backward hop (no per-char tracking). Byte-exact (o200k_parity); FSM
-21% to -41% across corpora (Thai -41%, Russian -39%, Chinese -34%),
so every corpus now beats logos on the full pipeline.

classify: widen the ASCII fast path to 32 bytes (one bounds check + one
`vmaxvq` covers two 16-byte stores) in place; the multibyte path is
untouched. English classify +45%, French/Russian +10-14% (text is
~half ASCII), no multibyte regression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants