fix(ext/node): upgrade `node:stream` #28855

littledivy · 2025-04-11T18:02:10Z

This PR replaces the outdated _streams.mjs bundle with an auto-managed, file-by-file port of Node.js stream internals using the ext/node/update_node_stream.ts script.

TBD:

Get it to compile
All tests pass
- Assign all error classes in internal/errors.ts #28856
- Implement newReadableStreamFromStreamReadable for deno_web interop
- Fix internal/streams/duplexify lazy loading (can't be hoisted due to circular dependency)
Enable more tests
Performance (size diff? snapshot deserde?)

Performance

Binary size (aarch64-apple-darwin): 85396664 bytes (+140KB from main)

Startup time:

$ hyperfine 'target/release/deno run empty.js' './deno_main run empty.js'
Benchmark 1:
  Time (mean ± σ):      20.6 ms ±   0.8 ms    [User: 14.4 ms, System: 5.0 ms]
  Range (min … max):    19.6 ms …  22.9 ms    123 runs

Benchmark 2:
  Time (mean ± σ):      19.8 ms ±   0.6 ms    [User: 14.2 ms, System: 5.0 ms]
  Range (min … max):    19.4 ms …  22.3 ms    125 runs

ry · 2025-04-11T20:57:50Z

ext/node/update_node_stream.js

@dsherret because this involves code transpilation, it'd be good to get your eyes.

ry · 2025-04-13T12:59:01Z

ext/node/update_node_stream.js

+// This file is used to transform Node.js internal streams code to
+// Deno polyfills.


Can you expand this description? This file is only run when we want to upgrade the node streams code?

Can you use TypeScript instead?

Ported to TypeScript

ry · 2025-04-13T12:59:54Z

ext/node/update_node_stream.js

+// Workaround a bug in our formatter: "export default from;" does not work
+// correctly, so we rename it to something else and export.
+const renameForDefaultExport = ["from"];


can you open a dprint issue and link it here?

dprint/dprint-plugin-typescript#705

It looks like an swc syntax error.

it can parse it on the swc playground

It's an swc diagnostic that's not surfaced in the playground. I'll open an swc issue.

Looked into it and here's an example of the swc playground erroring: https://play.swc.rs/?version=1.11.20&code=H4sIAAAAAAAAA0vOzysuUUgrys9VsFWorrXmSq0oyC8qUUhJTUsszYHIWAMAg8l5qiUAAAA%3D&config=H4sIAAAAAAAAA1WPsQ6DMAxEd74Cee7aDp2rbv0IKzUoKCGRbSQQ4t8JkNCyxXfPd%2FFc1TV0YuBZz%2BmZhogsxOecFJl6xTEpQMajGLZR4VZcGmNgfVGDg9M3B59A5YFOoJNtt0EntEvL4YAit6R7rNxzHrgQhAqeNW9720z%2FPzLBRyaRK7ih2LeOrnVVrgQfvsNu5kt1inTUP%2BAHlbIzGKx8yuZ2WLWst%2B%2Fb9zUBAAA%3D

It's because of the export default from proposal.

swc-project/swc#10372 -- Using identifiers that are also contextual keywords is kind of asking to hit edge cases.

ry

LGTM - this is a massive improvement!

Ref #28836 This PR replaces the _stream.mjs bundle with a file-by-file port instead. A codemod transpiles Node.js internals to ESM. The codemod performs three tasks: translating CJS to ESM, remapping internal dependencies, and hoisting lazy requires as imports. The process is fully automated through the `update_node_stream.ts` script, simplifying future internal updates. The script checks out Node.js from a specific tag defined in the `tests/node_compat/runner`. Additionally, the update enables new tests in our Node test runner and adds features (like compose()) that were missing from the outdated bundle. ## Performance There is a 140KB+ binary size increase on aarch64-apple-darwin and nop startup time stays the same.

littledivy changed the title ~~fix(ext/node): upgrade node:streams~~ fix(ext/node): upgrade node:stream Apr 11, 2025

littledivy mentioned this pull request Apr 11, 2025

Assign all error classes in internal/errors.ts #28856

Closed

ry reviewed Apr 11, 2025

View reviewed changes

ext/node/update_node_stream.js Outdated

Copy link

Member

ry Apr 11, 2025 •

edited

Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dsherret because this involves code transpilation, it'd be good to get your eyes.

littledivy force-pushed the streams-nightmare branch 4 times, most recently from 0319ace to 82367f4 Compare April 13, 2025 03:49

littledivy added the ci-draft Run the CI on draft PRs. label Apr 13, 2025

littledivy force-pushed the streams-nightmare branch 2 times, most recently from 80fe5e8 to 82e758d Compare April 13, 2025 09:16

littledivy removed the ci-draft Run the CI on draft PRs. label Apr 13, 2025

littledivy marked this pull request as ready for review April 13, 2025 09:16

littledivy mentioned this pull request Apr 10, 2025

Work towards addressing node-compat stdio streaming bugs #28836

Open

8 tasks

ry reviewed Apr 13, 2025

View reviewed changes

littledivy force-pushed the streams-nightmare branch 2 times, most recently from 6aa984f to 257132a Compare April 13, 2025 17:18

Rewrite node:stream

b7b27cf

littledivy force-pushed the streams-nightmare branch from 257132a to b7b27cf Compare April 13, 2025 17:19

ry approved these changes Apr 14, 2025

View reviewed changes

littledivy merged commit 01b6da9 into denoland:main Apr 14, 2025
18 checks passed

bartlomieju mentioned this pull request Apr 15, 2025

CONNECTION_CLOSED error when using postgres since Deno 2.2.10 #28905

Closed

JasperVanEsveld mentioned this pull request Apr 27, 2025

fix(compile): panic when compiled with --no-terminal flag #28823

Merged

littledivy mentioned this pull request May 14, 2025

Unignore test cases in test-stream-duplex-from.js #21336

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(ext/node): upgrade `node:stream` #28855

fix(ext/node): upgrade `node:stream` #28855

Uh oh!

littledivy commented Apr 11, 2025 •

edited

Loading

Uh oh!

ry Apr 11, 2025 •

edited

Loading

Uh oh!

ry Apr 13, 2025

Uh oh!

littledivy Apr 13, 2025

Uh oh!

ry Apr 13, 2025

Uh oh!

littledivy Apr 13, 2025

Uh oh!

dsherret Apr 13, 2025

Uh oh!

littledivy Apr 13, 2025

Uh oh!

dsherret Apr 13, 2025

Uh oh!

dsherret Apr 13, 2025

Uh oh!

dsherret Apr 13, 2025 •

edited

Loading

Uh oh!

ry left a comment

Uh oh!

Uh oh!

Uh oh!

		// This file is used to transform Node.js internal streams code to
		// Deno polyfills.

fix(ext/node): upgrade node:stream #28855

fix(ext/node): upgrade node:stream #28855

Uh oh!

Conversation

littledivy commented Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance

Uh oh!

ry Apr 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsherret Apr 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fix(ext/node): upgrade `node:stream` #28855

fix(ext/node): upgrade `node:stream` #28855

littledivy commented Apr 11, 2025 •

edited

Loading

ry Apr 11, 2025 •

edited

Loading

dsherret Apr 13, 2025 •

edited

Loading