[HttpKernel] Fix that the Store would not save responses with the X-Content-Digest header present
#36833
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mpdude
force-pushed
the
cache-upstream-content-digest-34
branch
2 times, most recently
from
aa18ef9
to
6726bac
Compare
fabpot
approved these changes
May 19, 2020
…-Content-Digest header present
nicolas-grekas
force-pushed
the
cache-upstream-content-digest-34
branch
from
60512dc
to
d8964fb
Compare
|
Thank you @mpdude. |
This was referenced May 26, 2020
Merged
Merged
Merged
Merged
mpdude
added a commit
to mpdude/symfony
that referenced
this pull request
Jun 10, 2020
fabpot
added a commit
that referenced
this pull request
Jun 11, 2020
…sponse body correctly (mpdude) This PR was squashed before being merged into the 3.4 branch. Discussion ---------- [HttpKernel] Fix regression where Store does not return response body correctly | Q | A | ------------- | --- | Branch? | 3.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Fix #37174 | License | MIT | Doc PR | Since #36833, the `Store` no longer uses or trusts the `X-Content-Digest` header present on a response, since that may come (in the case of using `CachingHttpClient`) from upstream HTTP sources. Instead, the `X-Content-Digest` is re-computed every time a response is written by the `Store`. Additionally, the `Store` is implemented in a way that when restoring responses, it does _not_ actually load the response body, but just keeps the file path to the content on disk in another internal header called `X-Body-File`. It is up to others (`HttpCache`, for example) to actually load the content from there. For reasons I could not determine, the file path is also set as the response body. When the `HttpCache` performs revalidations, it may happen that it wants the `Store` to persist a previously restored response. In that case, the `Store` fails to honor its own `X-Body-File` header. Instead, it would compute (since #36833) the `X-Content-Digest`, which now is a hash of the cache file path. So, we end up with a response that still carries `X-Body-File` for the original, correct response. Since the `HttpCache` honors this value, we don't immediately notice that. But inside the `Store`, the request is now associated with the _new_ (bogus) content entry. It takes another round of looking up the content in the `Store` to now get a response where the `X-Body-File` _also_ points to the wrong content entry. Although I feel a bit uncomfortable with trusting headers that seemingly need to be evaluated in different classes and may come from elsewhere, my suggestion is to skip the write inside `Store` if `X-Body-File` and `X-Content-Digest` are both present and consistent with each other. Additionally, a `file_exists` check could be added to provide additional assertions, at the cost of accessing the filesystem. Commits ------- 176e769 [HttpKernel] Fix regression where Store does not return response body correctly
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Responses fetched from upstream sources might have a
X-Content-Digestheader, for example if the Symfony Cache is used upstream. This currently prevents theStorefrom saving such responses. In general, the value of this header should not be trusted.As I consider this header an implementation detail of the
Store, the fix tries to be local to that class; we should not rely on theHttpCacheor other classes to remove untrustworthy headers for us.This fixes the issue that when using the
HttpCachein combination with the Symfony HttpClient, responses that have also been cached upstream in an instance ofHttpCacheare not cached locally. It adds the overhead of re-computing the content digest every time theHttpCachesuccessfully re-validated a response.