Add ability to transform file stream before sending it #171
Conversation
|
I know there was some previous PR / issue on this, and I think it is desired. I think there are some issues to work out in this implementation, mainly that any alteration in one of those transform streams that alters the length will mean the Content-Length header will be wrong and cause a transport error. |
|
Ok, I will try to fix this issues |
|
Hi @dougwilson ! I am figuring out how to apply transformations on the file (like compressing or adding ID3 tag to mp3 file) and set proper Content-Length to the response. a) We need to apply a deterministic transformation (when we know exactly how it will affect the size) to set the right header before it's being sent Am I thinking correclty? |
dougwilson
force-pushed
the
master
branch
2 times, most recently
from
4f438e7
to
a17c053
Compare
|
I'm going to see if I can find the previous conversation(s) on this, which may help. You are thinking correctly, but of course the downside to reading in the entire file into memory is that it's very memory-intensive and becomes much easier to overflow the server's memory with lots of requests, exhausting the Node.js process memory until it just dies. That's the main reason we are using streams right now. We've had bug reports about the handling of files >2GB a few times, so there are definitely large file usage it seems. |
|
Hello @dougwilson ! Any update on this? |
|
No, sorry. |
|
The conversation I'm thinking of is in a previous PR for this feature: #69 |
I have the case, where I need to do some transformations on the file before writing it to the response.
The way I do it now is by passing the array of transformation streams with options.
Could it be useful for others? If yes, maybe there are better ways of doing this?