opt: tabulate the optimal parser's per-code price components

[matevz-kovacic]

Summary

ZSTD_getMatchPrice() and ZSTD_litLengthPrice() in lib/compress/zstd_opt.c re-derive each price from the raw frequency arrays on every call, although within a stretch group the statistics are frozen and the price is a pure function of the symbol code alone — 36 litLength codes, 53 matchLength codes, 32 offset codes. At level 19 the match-price loop runs once per (match, candidate length) pair, so that derivation is repeated hundreds of times per position for values that cannot have changed.

This patch stores the per-code half of each price and adds the running *SumBasePrice term at the point of use, and splits the match price so its offset half can be hoisted out of the length loop.

Compressed output is byte-identical at every level under both compilers on both test machines, so there is no ratio trade-off to weigh. Level-19 encode throughput rises +12.1% (gcc) / +6.7% (clang) on an AMD Ryzen 7 9700X and +9.0% / +6.4% on an Intel i5-13400, and retired instructions at level 19 fall by 30.2% and 21.4% respectively — the work removed is measured directly, not inferred from timing. Levels below 13 never enter this file and are unchanged, as the counters confirm.

Is the parse affected?

No, and this is checkable rather than a matter of judgement. Every transformation only regroups a sum of U32 terms:

term	before	after
offset half	oc*BM + (offSum - W(offFreq[oc]))	offCodeSumBasePrice + offCodePriceNoSum[oc]
length half	ML_bits[mc]*BM + (mlSum - W(mlFreq[mc])) + BM/5	matchLengthSumBasePrice + matchLengthPriceNoSum[mc]
litLength	LL_bits[lc]*BM + (llSum - W(llFreq[lc]))	litLengthSumBasePrice + litLengthPriceNoSum[lc]

U32 addition is associative, so each price is reproduced bit for bit, the DP compares identical prices, selects an identical path, and emits an identical byte stream. The BITCOST_MULTIPLIER/5 constant and the long-offset handicap are folded into the stored entries for the same reason.

Stored entries are deliberately allowed to wrap: each is a difference that is only meaningful once its *SumBasePrice is added back, and modular arithmetic restores the original value exactly.

The identity was checked exhaustively offline over all code pairs at both optLevels and 63 frequency tables including adversarial ones — 218,232 cases, all exact, including the 177,440 in which a stored entry wraps past 2^31.

Where the tables are maintained

The statistics these tables depend on are written at exactly two places in the library: ZSTD_rescaleFreqs() replaces all of them once per block, and ZSTD_updateStats() increments one entry of each per emitted sequence.

That is why the *SumBasePrice term is excluded from the stored value: a statistics update then invalidates exactly one entry per table, so ZSTD_updateStats() refreshes in place and a full rebuild is only needed in ZSTD_rescaleFreqs(). An earlier version that stored the whole price had to rebuild all 121 entries after every stretch group; clang compiled that rebuild to 1015 instructions and it consumed most of the gain (+1.9% instead of +4.7% under clang on the first test machine).

Generated code

Retired instructions and related counters at level 19, full Silesia, AMD Ryzen 7 9700X, medians of 3 runs. The bracketed figure is the run-to-run spread of the baseline, which is why these are quoted in preference to wall time — retired instructions is essentially deterministic for a deterministic workload.

counter	gcc 16.0.1	clang 22.1.8
instructions	−30.18% (spread 0.70%)	−21.41% (spread 0.71%)
L1-dcache-loads	−22.10% (0.55%)	−16.89% (0.86%)
cycles	−10.55% (1.24%)	−5.77% (1.14%)
backend stall slots	−11.26%	−10.88%
demand fills from DRAM	+2.99%	+1.50%
branch-misses	−0.50%	−0.21%

The unchanged DRAM-fill figure is the expected result and worth stating: this change is arithmetic only and does not alter memory behaviour.

Effect on compressed output: none

Compressed size is identical, as exact integers, in every cell tested — 4 levels × 2 compilers × 2 machines, all 12 Silesia files:

level	bytes, baseline and patched
1	73,229,468
3	66,137,723
9	59,081,628
19	52,891,946

Also identical through the CLI path (which streams, so the totals differ from the one-shot library figures above but likewise match exactly).

Effect on other levels: none

Levels below 13 never enter zstd_opt.c. The counters confirm it rather than assuming it — retired instructions at level 9 change by +0.00% (gcc) and −0.00% (clang), against a baseline spread of 0.01%.

Timing at levels 1/3/9 lands within ±0.6% with overlapping interquartile ranges on both machines, i.e. below what the measurement resolves. Decode is unaffected at every level.

Correctness

• fuzzer -i2000 -s1 and zstreamtest -i1500 -s1 pass under both compilers on both machines.
• Round-trip, forward-compatibility (an unmodified upstream binary decodes this build's output) and backward-compatibility (this build decodes unmodified upstream output) verified on all 12 Silesia files at levels 1/3/9/19 under both compilers on both machines.
• Compiles with zero warnings under the project's own warning set (-Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wmissing-prototypes -Wc++-compat) at DEBUGLEVEL 0 and 2, both compilers.

Benchmarks

Level-19 encode, full Silesia (211,938,580 B), single-threaded, pinned to one core, machine otherwise idle, baseline and candidate runs alternated, first repetition discarded, medians over 8 repetitions, with non-overlapping interquartile ranges required before a win is claimed.

machine	compiler	baseline	patched	change
Ryzen 7 9700X, Ubuntu 24.04	gcc 16.0.1	5.507 MB/s	6.172 MB/s	+12.07%
Ryzen 7 9700X, Ubuntu 24.04	clang 22.1.8	5.647 MB/s	6.028 MB/s	+6.73%
i5-13400, Windows/MinGW	gcc 16.1.0	3.294 MB/s	3.592 MB/s	+9.03%
i5-13400, Windows/MinGW	clang 22.1.8	3.414 MB/s	3.631 MB/s	+6.38%

Measurement floor: running the unmodified baseline against itself through the identical path gives a worst cell of 1.18% on the Ryzen, with 0 of 16 cells showing disjoint interquartile ranges. On the i5-13400 the same check gives 3.40%, which is why the Ryzen figures are the primary ones.

Notes

• optState_t grows by 484 bytes (three tables: 36 + 53 + 32 U32 entries), which grows ZSTD_MatchState_t and therefore the values ZSTD_estimateCCtxSize() and ZSTD_sizeof_CCtx() report. Every such size derives from sizeof, so the accounting stays self-consistent, but it is a visible change in reported context size. If you would prefer the tables in the cwksp alongside litFreq et al. for consistency, that is a small change; I kept them inline to avoid a pointer indirection in the hot path.
• Not tested: ARM, 32-bit, big-endian, dictionary-heavy workloads beyond what fuzzer covers, and levels other than 1/3/9/19.
• Not included: a follow-up change adding speculative prefetch to the binary-tree match finder. It is worth a further +4.9 points on the i5-13400, but on the Ryzen 9700X it is neutral to slightly negative under both compilers and introduces a small statistically detectable decode regression. The counters show why: it does cut demand fills from DRAM by ~23%, but on a part with a 32 MB L3 those fills were not on the critical path. Happy to share that data if useful.