Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HttpFoundation] Fix memory limit problems in BinaryFileResponse #48972

Merged
merged 1 commit into from
Apr 17, 2023

Conversation

glady
Copy link
Contributor

@glady glady commented Jan 13, 2023

see #48692

Q A
Branch? 5.4
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #48692
License MIT
Doc PR -

@carsonbot
Copy link

Hey!

I see that this is your first PR. That is great! Welcome!

Symfony has a contribution guide which I suggest you to read.

In short:

  • Always add tests
  • Keep backward compatibility (see https://symfony.com/bc).
  • Bug fixes must be submitted against the lowest maintained branch where they apply (see https://symfony.com/releases)
  • Features and deprecations must be submitted against the 6.3 branch.

Review the GitHub status checks of your pull request and try to solve the reported issues. If some tests are failing, try to see if they are failing because of this change.

When two Symfony core team members approve this change, it will be merged and you will become an official Symfony contributor!
If this PR is merged in a lower version branch, it will be merged up to all maintained branches within a few days.

I am going to sit back now and wait for the reviews.

Cheers!

Carsonbot

@carsonbot carsonbot changed the title Fix memory limit problems in BinaryFileResponse [HttpFoundation] Fix memory limit problems in BinaryFileResponse Jan 14, 2023
@nicolas-grekas
Copy link
Member

Thanks for the PR. Can you please add a test case?

@glady
Copy link
Contributor Author

glady commented Jan 18, 2023

I tried to have a look on a reproduction and a test case. I can add a test, that chunks are used with my changes, but I failed in reproducing hitting memory_limit.

I analyzed, why it happened in our application and found that there were two levels of output buffering. Closing both/all (instead of just one) did work well for us.

So my findings are:

  • without output_buffering active (ob_get_level() === 0), the download works with -1 (as it is implemented)
  • with one level of output_buffering active (ob_get_level() === 1), the download fails with -1 and works when I do stream_copy_to_stream in chunks (this PR), but I not fully understand why.
  • with two levels of output_buffering, my patch does not work too.

So my PR is not valid for all cases, as developer I would have expected either closing all output buffers internally via

            while (ob_get_level()) {
                ob_end_clean();
            }

or just throw an exception, if there is any output buffer active. For me it would help to see this dependency somehow, when using BinaryFileResponse. Its just not visible, that the file fully is loaded to memory instead of sent until you hit a file that does not fit in your memory limit.

@glady
Copy link
Contributor Author

glady commented Jan 24, 2023

I thought about this some days now. Closing all output-buffers (via while) is possible in our specific case, so we need not to wait for this PR, but indeed especially for end-to-end functional tests it would be nice to be able to keep at least one buffer open in order to be able to check the content of the binary file response. In the test case, it will not exceed the memory limit. To change the code from -1 to $filesize would support this. In this case we would be able to close our one "known" buffer that catches all for some other reasons, but still would allow to put any code around it with separate handling like a test that checks a buffer afterwards.

I will check, if I can add testcases in order to demonstrate this.

@nicolas-grekas
Copy link
Member

Thank you @glady.

@nicolas-grekas nicolas-grekas merged commit bee9131 into symfony:5.4 Apr 17, 2023
6 of 9 checks passed
This was referenced Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants