wc: increase buffer size for Unicode counting paths#11276
wc: increase buffer size for Unicode counting paths#11276mattsu2020 wants to merge 1 commit intouutils:mainfrom
Conversation
|
GNU testsuite comparison: |
Merging this PR will degrade performance by 93.59%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | wc_words_large_line_count[100000] |
16.7 KB | 260.4 KB | -93.59% |
| ❌ | Memory | wc_words_synthetic[2000] |
16.7 KB | 260.4 KB | -93.59% |
| ❌ | Memory | wc_default_large_line_count[100000] |
16.7 KB | 260.4 KB | -93.59% |
Comparing mattsu2020:performance_analystic (d8a78c1) with main (36b5e59)2
Footnotes
-
48 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
main(88219aa) during the generation of this report, so 36b5e59 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Increased buffer size to 256KB for improved performance in word counting operations across all input types (stdin and files). This optimization reduces I/O overhead by processing larger chunks of data at once.
11c950b to
d8a78c1
Compare
|
GNU testsuite comparison: |
It uses a 256 KiB stack array, this PR uses |
Summary
The Unicode path in
wcwas still using the defaultBufReadercapacity of 8 KiB. That made-w,-L, and the default-lwcpath pay morefill_bufand UTF-8 decoding overhead than necessary.This PR switches the Unicode counting path to use a 256 KiB buffer, matching the fast path buffer size and reducing per-chunk overhead.
Changes
WORD_COUNT_BUF_SIZE = 256 * 1024insrc/uu/wc/src/countable.rsFile'sWordCountable::buffered()to useBufReader::with_capacity(...)StdinLock'sWordCountable::buffered()to useBufReader::with_capacity(...)Background
The
-c/-l/-mfast path already uses a 256 KiB buffer, but the Unicode path used by-w,-L, and related combinations goes throughBufReadDecoderand was still backed by the default 8 KiBBufReader.As a result, large inputs on the Unicode path were processed in much smaller chunks, adding avoidable fixed overhead.
Testing
cargo build -p uu_wc --releasecargo test -p uu_wccargo test --features wc test_wcThe
wcintegration tests passed (57 passed).