Error when trying to pipe large binary file

[Sov3rain]

Hi there,

I'm getting an error when trying to pipe a binary file (a 3D model in .glb). Here is the server error:

[ERROR] default - on uncaughtException Error [ERR_STREAM_WRITE_AFTER_END]: write after end
at new NodeError (node:internal/errors:405:5)
at ServerResponse.end (node:_http_outgoing:1017:15)
at PassThrough.<anonymous> (/app/dist/src/piping.js:315:40)
at PassThrough.emit (node:events:514:28)
at errorEventQueue (/app/node_modules/multiparty/index.js:645:18)
at handleError (/app/node_modules/multiparty/index.js:211:9)
at IncomingMessage.onReqAborted (/app/node_modules/multiparty/index.js:197:5)
at IncomingMessage.emit (node:events:514:28)
at IncomingMessage._destroy (node:_http_incoming:224:10)
at _destroy (node:internal/streams/destroy:109:10) {
code: 'ERR_STREAM_WRITE_AFTER_END'
}

The file is 47 MB.

What I'm doing to get there:

• Reading a file from disk
• Creating a blob from this file
• Putting the blob in a form data
• Sending the form data with a POST request

Let me know if you need more details.

Environment:
Deployed from the Dockerfile on Railway.app

[Sov3rain]

From what I understand from this, it seems like the response is closed too early. Is that correct?

EDIT: related to #261 ?

[nwtgck]

I have the same error, but I could not reproduce it 100% yet. I will continue to investigate.

[Sov3rain]

Yeah, same here, the issue is not 100% reproductible.

[nwtgck]

I've found a 100% reproducible case.

A sender uses HTTP/1.1 over TLS and a receiver uses plain HTTP/1.1.

# sender
curl -T- --http1.1 -k https://localhost:8443/mydata

# receiver
curl http://localhost:8080/mydata

Close the receiver by Ctrl-C during transferring and the server emits the following error:

[ERROR] default - on uncaughtException Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at write_ (node:_http_outgoing:911:11)
    at ServerResponse.write (node:_http_outgoing:864:15)
    at abortedListener (/app/dist/src/piping.js:234:44)
    at IncomingMessage.<anonymous> (/app/dist/src/piping.js:292:21)
    at IncomingMessage.emit (node:events:519:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

edit:
The following case also reproduced the issue.

• sender: HTTP/1.1 over TLS
• receiver: HTTP/1.1 over TLS

[nwtgck]

I assume your receiver closed for some reason and the server emitted the error. I'm still not sure why the server behaves differently in plain HTTP/1.1 and HTTP/1.1 over TLS.

[Sov3rain]

I'll be honest here, I've got no idea how to verify what protocol is used. I use a piping server exclusively on the frontend, to pass large 3D models from desktop to mobile and view them in a 3D viewer.

Nothing fancy, just fetch calls.

[nwtgck]

What are your HTTP clients? Google Chrome?

---

If you are intereseted in what protocol is used and the frontend is a browser, try the following steps.

Open a browser (like Chrome) and click F12 for DevTools.

Go to the Network tab as follows.

Reload the page.

Left click the bar with "Name", "Type", "Initiator"... and check "Protocol" as follows.

If "Protocol" is "h2", HTTP/2 is used.

If http/1.1 is shown, HTTP/1.1 is used.

[Sov3rain]

Thanks for the quick tutorial! I'll make sure to test it out when i'm able to somewhat repro the issue.

I'm using Brave browser, which is a Chromium based browser. Brave Shields disabled (adblockers and such).