stream: optimize webstreams `pipeTo` further by MattiasBuelens · Pull Request #62079 · nodejs/node

MattiasBuelens · 2026-03-02T21:06:20Z

I've started looking more closely at the pipeTo() implementation, and found some low-hanging fruit to further optimize it:

We don't have to await writer.ready every time, that's only needed if there is backpressure. So we check that first.
We don't have to go through writer.desiredSize to check if there is backpressure, we can read that directly from the internal state of the WritableStreamDefaultController.
readableStreamDefaultControllerCallPullIfNeeded already checks state === 'readable' && !closeRequested through readableStreamDefaultControllerCanCloseOrEnqueue, so we don't have to repeat that check.

This also fixes a few more edge cases:

Previously, we were using writer.desiredSize inside pipeTo(), which is part of the public WritableStreamDefaultWriter API, and step 15 of the spec explicitly forbids using the public API since it can be modified by user-land code. We now bypass this entirely using dest[kState].backpressure.
- There doesn't seem to be much WPT test coverage for this... I might add more tests to the upstream patched-global.any.js suite.
If the writable end becomes closed or errored in the middle of a batch of synchronous reads, we should stop immediately.

On my machine, this improves the webstreams/pipe-to.js benchmark by 3% to 7%:

Baseline @ 199daab

webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=512 n=500000: 1,639,426.502378644
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=512 n=500000: 1,672,850.078172284
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=512 n=500000: 1,684,703.098034221
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=512 n=500000: 1,658,174.6018059512
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=1024 n=500000: 1,684,320.5922340695
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=1024 n=500000: 1,663,948.3297453062
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=1024 n=500000: 1,651,671.1443471392
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=1024 n=500000: 1,630,100.2576862487
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=2048 n=500000: 1,662,200.8869503932
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=2048 n=500000: 1,638,533.5910855907
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=2048 n=500000: 1,671,184.435256644
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=2048 n=500000: 1,639,392.637815542
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=4096 n=500000: 1,644,874.8168020674
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=4096 n=500000: 1,702,132.8746198711
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=4096 n=500000: 1,608,151.915035545
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=4096 n=500000: 1,688,220.1736233155

This PR @ 7b31401

webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=512 n=500000: 1,768,254.040106831
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=512 n=500000: 1,717,563.2174905646
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=512 n=500000: 1,746,963.515365767
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=512 n=500000: 1,787,073.738236587
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=1024 n=500000: 1,756,757.2790359478
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=1024 n=500000: 1,704,454.421184323
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=1024 n=500000: 1,761,150.6372019118
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=1024 n=500000: 1,766,948.0354718352
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=2048 n=500000: 1,727,179.5537866165
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=2048 n=500000: 1,768,980.3632565797
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=2048 n=500000: 1,752,746.9927243474
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=2048 n=500000: 1,793,850.0362895865
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=4096 n=500000: 1,769,294.5999359516
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=4096 n=500000: 1,768,876.4768791925
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=4096 n=500000: 1,748,478.5613798152
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=4096 n=500000: 1,783,529.4621231847

Credok12

Ok

nodejs-github-bot · 2026-03-03T07:30:20Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71539/

codecov · 2026-03-03T08:34:13Z

Codecov Report

❌ Patch coverage is 75.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.64%. Comparing base (199daab) to head (7b31401).
⚠️ Report is 121 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/webstreams/readablestream.js	75.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #62079      +/-   ##
==========================================
- Coverage   89.75%   89.64%   -0.11%     
==========================================
  Files         674      676       +2     
  Lines      204886   206310    +1424     
  Branches    39376    39520     +144     
==========================================
+ Hits       183894   184951    +1057     
- Misses      13280    13480     +200     
- Partials     7712     7879     +167

Files with missing lines	Coverage Δ
lib/internal/webstreams/readablestream.js	`98.34% <75.00%> (-0.12%)`	⬇️

... and 166 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nodejs-github-bot · 2026-03-03T13:30:17Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71543/

MattiasBuelens · 2026-03-08T18:09:49Z

I finally figured out how to do a proper benchmark comparison. (Am I a real scientist now? 😄)

These are the raw results: webstreams.csv. node-benchmark-compare confirms a 6%~7% improvement:

                                                                       confidence improvement accuracy (*)   (**)  (***)
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=1024 n=500000        ***      6.53 %       ±0.82% ±1.10% ±1.43%
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=2048 n=500000        ***      7.43 %       ±0.79% ±1.05% ±1.37%
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=4096 n=500000        ***      6.59 %       ±0.86% ±1.15% ±1.50%
webstreams\pipe-to.js highWaterMarkW=1024 highWaterMarkR=512 n=500000         ***      6.20 %       ±1.06% ±1.41% ±1.84%
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=1024 n=500000        ***      7.41 %       ±1.05% ±1.40% ±1.82%
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=2048 n=500000        ***      7.78 %       ±0.75% ±1.00% ±1.31%
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=4096 n=500000        ***      6.53 %       ±0.73% ±0.97% ±1.26%
webstreams\pipe-to.js highWaterMarkW=2048 highWaterMarkR=512 n=500000         ***      7.36 %       ±0.74% ±0.99% ±1.29%
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=1024 n=500000        ***      7.18 %       ±0.76% ±1.01% ±1.31%
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=2048 n=500000        ***      7.18 %       ±0.85% ±1.13% ±1.47%
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=4096 n=500000        ***      7.65 %       ±0.74% ±0.98% ±1.28%
webstreams\pipe-to.js highWaterMarkW=4096 highWaterMarkR=512 n=500000         ***      6.82 %       ±0.78% ±1.03% ±1.35%
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=1024 n=500000         ***      7.13 %       ±0.72% ±0.96% ±1.25%
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=2048 n=500000         ***      7.35 %       ±0.72% ±0.96% ±1.24%
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=4096 n=500000         ***      7.15 %       ±1.01% ±1.35% ±1.76%
webstreams\pipe-to.js highWaterMarkW=512 highWaterMarkR=512 n=500000          ***      7.41 %       ±0.79% ±1.05% ±1.36%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 16 comparisons, you can thus expect the following amount of false-positive results:
  0.80 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.16 false positives, when considering a   1% risk acceptance (**, ***),
  0.02 false positives, when considering a 0.1% risk acceptance (***)

MattiasBuelens · 2026-03-08T18:11:21Z

cc @nodejs/web-standards for reviews

mcollina

lgtm

nodejs-github-bot · 2026-03-08T22:56:18Z

Commit Queue failed

- Loading data for nodejs/node/pull/62079 ✔ Done loading data for nodejs/node/pull/62079 ----------------------------------- PR info ------------------------------------ Title stream: optimize webstreams `pipeTo` further (#62079) ⚠ Could not retrieve the email or name of the PR author's from user's GitHub profile! Branch MattiasBuelens:webstreams-optimize-pipeto -> nodejs:main Labels needs-ci, web streams Commits 5 - stream: optimize webstreams pipeTo - fixup! do not use writer public API - fixup! optimize backpressure check - fixup! wait for ready only if there is backpressure - fixup! disturbed is already true at start of pipeTo Committers 1 - Mattias Buelens <mattias@buelens.com> PR-URL: https://github.com/nodejs/node/pull/62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> -------------------------------------------------------------------------------- ℹ This PR was created on Mon, 02 Mar 2026 21:06:20 GMT ✔ Approvals: 2 ✔ - James M Snell (@jasnell) (TSC): https://github.com/nodejs/node/pull/62079#pullrequestreview-3878880914 ✔ - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/62079#pullrequestreview-3912132009 ✔ Last GitHub CI successful ℹ Last Full PR CI on 2026-03-03T13:30:17Z: https://ci.nodejs.org/job/node-test-pull-request/71543/ - Querying data for job/node-test-pull-request/71543/ ✔ Build data downloaded ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ No git cherry-pick in progress ✔ No git am in progress ✔ No git rebase in progress -------------------------------------------------------------------------------- - Bringing origin/main up to date... From https://github.com/nodejs/node * branch main -> FETCH_HEAD ✔ origin/main is now up-to-date - Downloading patch for 62079 From https://github.com/nodejs/node * branch refs/pull/62079/merge -> FETCH_HEAD ✔ Fetched commits as f38a73962384..7b31401e6d9a -------------------------------------------------------------------------------- Auto-merging lib/internal/webstreams/readablestream.js [main bd9b38ce57] stream: optimize webstreams pipeTo Author: Mattias Buelens <mattias@buelens.com> Date: Sun Mar 1 13:33:20 2026 +0100 1 file changed, 1 insertion(+), 4 deletions(-) Auto-merging lib/internal/webstreams/readablestream.js [main 8f72309155] fixup! do not use writer public API Author: Mattias Buelens <mattias@buelens.com> Date: Sun Mar 1 13:35:28 2026 +0100 1 file changed, 2 insertions(+), 1 deletion(-) Auto-merging lib/internal/webstreams/readablestream.js [main 0124d30806] fixup! optimize backpressure check Author: Mattias Buelens <mattias@buelens.com> Date: Sun Mar 1 13:56:04 2026 +0100 1 file changed, 6 insertions(+), 7 deletions(-) Auto-merging lib/internal/webstreams/readablestream.js [main fc8a6fb91a] fixup! wait for ready only if there is backpressure Author: Mattias Buelens <mattias@buelens.com> Date: Mon Mar 2 20:58:14 2026 +0100 1 file changed, 5 insertions(+), 3 deletions(-) Auto-merging lib/internal/webstreams/readablestream.js [main 78f7f67869] fixup! disturbed is already true at start of pipeTo Author: Mattias Buelens <mattias@buelens.com> Date: Mon Mar 2 21:04:06 2026 +0100 1 file changed, 1 deletion(-) ✔ Patches applied There are 5 commits in the PR. Attempting autorebase. (node:451) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated. (Use `node --trace-deprecation ...` to show where the warning was created) Rebasing (2/10) Executing: git node land --amend --yes --------------------------------- New Message ---------------------------------- stream: optimize webstreams pipeTo

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 9b67015883] stream: optimize webstreams pipeTo
Author: Mattias Buelens <mattias@buelens.com>
Date: Sun Mar 1 13:33:20 2026 +0100
1 file changed, 1 insertion(+), 4 deletions(-)
Rebasing (3/10)
Rebasing (4/10)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
fixup! do not use writer public API

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 864973cb6c] fixup! do not use writer public API
Author: Mattias Buelens <mattias@buelens.com>
Date: Sun Mar 1 13:35:28 2026 +0100
1 file changed, 2 insertions(+), 1 deletion(-)
Rebasing (5/10)
Rebasing (6/10)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
fixup! optimize backpressure check

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 0abbb8a8d2] fixup! optimize backpressure check
Author: Mattias Buelens <mattias@buelens.com>
Date: Sun Mar 1 13:56:04 2026 +0100
1 file changed, 6 insertions(+), 7 deletions(-)
Rebasing (7/10)
Rebasing (8/10)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
fixup! wait for ready only if there is backpressure

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD ba81ca4e7d] fixup! wait for ready only if there is backpressure
Author: Mattias Buelens <mattias@buelens.com>
Date: Mon Mar 2 20:58:14 2026 +0100
1 file changed, 5 insertions(+), 3 deletions(-)
Rebasing (9/10)
Rebasing (10/10)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
fixup! disturbed is already true at start of pipeTo

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 0e2f60e018] fixup! disturbed is already true at start of pipeTo
Author: Mattias Buelens <mattias@buelens.com>
Date: Mon Mar 2 21:04:06 2026 +0100
1 file changed, 1 deletion(-)
Successfully rebased and updated refs/heads/main.

ℹ Add commit-queue-squash label to land the PR as one commit, or commit-queue-rebase to land as separate commits.

https://github.com/nodejs/node/actions/runs/22831749349

nodejs-github-bot · 2026-03-09T03:18:19Z

Landed in f82525e

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

MattiasBuelens added 5 commits March 1, 2026 13:33

stream: optimize webstreams pipeTo

dd496c2

fixup! do not use writer public API

3479d39

fixup! optimize backpressure check

b8f166c

fixup! wait for ready only if there is backpressure

911f895

fixup! disturbed is already true at start of pipeTo

7b31401

MattiasBuelens added the web streams label Mar 2, 2026

nodejs-github-bot added the needs-ci PRs that need a full CI run. label Mar 2, 2026

jasnell approved these changes Mar 2, 2026

View reviewed changes

Credok12 reviewed Mar 3, 2026

View reviewed changes

MattiasBuelens marked this pull request as ready for review March 3, 2026 07:12

MattiasBuelens added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 3, 2026

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 3, 2026

mcollina approved these changes Mar 8, 2026

View reviewed changes

mcollina added the commit-queue Add this label to land a pull request using GitHub Actions. label Mar 8, 2026

nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Mar 8, 2026

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Mar 9, 2026

nodejs-github-bot merged commit f82525e into nodejs:main Mar 9, 2026
112 of 113 checks passed

MattiasBuelens deleted the webstreams-optimize-pipeto branch March 9, 2026 03:22

aduh95 pushed a commit that referenced this pull request Mar 10, 2026

stream: optimize webstreams pipeTo

0609510

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

github-actions bot mentioned this pull request Mar 10, 2026

2026-03-11, Version 25.8.1 (Current) #62184

Merged

aduh95 pushed a commit that referenced this pull request Mar 10, 2026

stream: optimize webstreams pipeTo

0278461

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Uh oh!

Conversation

MattiasBuelens commented Mar 2, 2026

Uh oh!

Credok12 left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Mar 3, 2026

Uh oh!

codecov bot commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nodejs-github-bot commented Mar 3, 2026

Uh oh!

MattiasBuelens commented Mar 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattiasBuelens commented Mar 8, 2026

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Mar 8, 2026

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 0e2f60e018] fixup! disturbed is already true at start of pipeTo Author: Mattias Buelens <mattias@buelens.com> Date: Mon Mar 2 21:04:06 2026 +0100 1 file changed, 1 deletion(-) Successfully rebased and updated refs/heads/main.

Uh oh!

Uh oh!

nodejs-github-bot commented Mar 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Mar 3, 2026 •

edited

Loading

MattiasBuelens commented Mar 8, 2026 •

edited

Loading

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: #62079
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

[detached HEAD 0e2f60e018] fixup! disturbed is already true at start of pipeTo
Author: Mattias Buelens <mattias@buelens.com>
Date: Mon Mar 2 21:04:06 2026 +0100
1 file changed, 1 deletion(-)
Successfully rebased and updated refs/heads/main.