Skip to content

gh-150942: Speed up BytesIO.readlines() using reference-stealing append - #155171

Open
Taeknology wants to merge 4 commits into
python:mainfrom
Taeknology:perf/gh-150942-bytesio-readlines-final
Open

gh-150942: Speed up BytesIO.readlines() using reference-stealing append#155171
Taeknology wants to merge 4 commits into
python:mainfrom
Taeknology:perf/gh-150942-bytesio-readlines-final

Conversation

@Taeknology

@Taeknology Taeknology commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Use _PyList_AppendTakeRef() while building the private result list in
BytesIO.readlines().

The new bytes object is transferred directly to the result list, avoiding the
incref/decref pair performed by PyList_Append(). The result list is freshly
allocated, remains local for the complete loop, and is protected by the
BytesIO critical section until it is returned.

Partially fixes gh-150942.

Benchmark

Author-run pyperf comparisons used five BytesIO.readlines() workloads on a
free-threaded Clang 21 PGO+ThinLTO build. Two balanced baseline/patch comparison
pairs reported improvements of 8.64% to 36.18% across the workloads, with
19.27% and 20.43% equal-weight geometric-mean improvements. Earlier regular-GIL
PGO+LTO comparisons reported 7.64% and 8.68% geometric-mean improvements.

Several individual suites produced pyperf check stability warnings. The raw
benchmark script and result files are not attached to this PR, so these figures
are author-reported results and are not independently reproducible from the
public PR artifacts.

Verification

All required GitHub checks pass on the current head, including regular and
free-threaded builds and tests, sanitizer jobs, and generated-file checks.
git diff --check also passes for the complete two-file diff.

@Taeknology
Taeknology marked this pull request as ready for review August 4, 2026 15:14
Comment thread Misc/NEWS.d/next/Library/2026-08-04-19-50-52.gh-issue-150942.B8ytes.rst Outdated
@Taeknology
Taeknology requested a review from eendebakpt August 5, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve performance by using reference stealing methods

2 participants