http: sendfile

[ronag]

Sending files is a common usage of the http api's. We could make a specialised path for this that is more optimised, e.g. we could totally skip all streams boilerplate and directly read buffers through the fs api and write to the response. Further steps could be to move the whole thing into native land. The recent ioring optimisation in libuv also adds some interesting possibilities.

This currently only contains a stub as a suggestion for anyone that wants to pick it up.

[nodejs-github-bot]

Review requested:

• @nodejs/http
• @nodejs/net

[debadree25]

read buffers through the fs api and write to the response

Is there any similar implementation or which apis to use could you guide? would like to give it a try!

[bnoordhuis]

Relevant issue: #41393 - but see the discussion for why it offers only limited returns beyond cutting node's self-imposed red tape.

[mcollina]

Ultimately I'm not opposed but this would need to match all the functionality of https://github.com/pillarjs/send.

[ronag]

Ultimately I'm not opposed but this would need to match all the functionality of https://github.com/pillarjs/send.

I disagree with this. It just needs to send the body. All the header stuff would be up to the user.

[mcollina]

Then we are not in agreement. Adding this to core without handling of all the headers does not improve the developer experience, as the user would need to do all of that by themselves.

[ronag]

Then we are not in agreement. Adding this to core without handling of all the headers does not improve the developer experience, as the user would need to do all of that by themselves.

For me, this is about performance, not functionality.

I don't see this as replacing something like pillarjs/send. I see it as a tool for pillarjs/send to make it faster, i.e. replace: pipeline(fs.createReadStream(path, { start, end }), res, cb) with res.sendFile(path, { start, end }, cb).

I see only the reason to add any other logic besides sending the body because it is unlikely to get a consensus as to what it should do and how. Also, doing it will take extra and unnecessary effort (pillarjs/send and the like already have implemented it). So, in the end, we end up with nothing when we could have provided the ecosystem with something that makes it faster for everyone. Personally I'm also worried that whatever we chose it should do would be in conflict with how we want it to work and would therefore not be usable for us and there is no other alternative to achieve similar performance.

[ronag]

That being said. If we don't see a path to make this significantly faster in the future, then it's pointless. I would defer to @bnoordhuis @jasnell @addaleax on that point. I hoped it could be possible to move the whole transfer between the file and socket handle into kernel space or something between that and doing it in javascript like we do now.

[cjihrig]

It looks like discussions don't link to issues, but this comment seems relevant to this issue.

[bnoordhuis]

Quick recap: openssl can, when the stars align, use zero-copy kTLS on Linux and FreeBSD to send files over TLS.

As to @mcollina's objection: node historically and philosophically has been firmly in the "mechanism over policy" camp. It's not necessary to support all the bells and whistles of send as long as you can layer that on top of the built-in mechanism.

[mcollina]

Ack @bnoordhuis.

I'm ok with adding this if it provides a significantly improved performance.