experimental: Allow query plan & parsed/validated document cache config #3755
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nt cache.
This introduces two experimental configuration options for existing internal
cache stores:
- `experimental_approximateDocumentStoreSizeMiB` is added to the `ApolloServer`
constructor options and controls the _approximate_[1] size, in MiB, of the
request pipeline document store, which is used to avoid repeated parsing and
validation passes on queries which equate to the same `queryHash`[2].
- `experimental_approximateQueryPlanStoreMiB` is added to the `ApolloGateway`
constructor options and controls the _approximate_[1] size of the query plan
cache, which is used to avoid recalculation on repeated queries which equate
to the same `queryHash`[2].
These are currently experimental because a more complete solution might
more dynamically account for memory of various internal caches and vary
their sizes based based on available memory and the overall server
performance, rather than needing hard-coded values which are less than
precise.
[1]: "Approximate", because it's based on the `JSON.stringify`'d byte-length
of the value being cached.
[2]: The `queryHash` is a SHA-256 value of the incoming operation.
abernix
added a commit
to apollographql/federation
that referenced
this pull request
Sep 4, 2020
…llographql/apollo-server#3755) * experimental: Allow adjusting of query plan & parsed/validated document cache. This introduces two experimental configuration options for existing internal cache stores: - `experimental_approximateDocumentStoreSizeMiB` is added to the `ApolloServer` constructor options and controls the _approximate_[1] size, in MiB, of the request pipeline document store, which is used to avoid repeated parsing and validation passes on queries which equate to the same `queryHash`[2]. - `experimental_approximateQueryPlanStoreMiB` is added to the `ApolloGateway` constructor options and controls the _approximate_[1] size of the query plan cache, which is used to avoid recalculation on repeated queries which equate to the same `queryHash`[2]. These are currently experimental because a more complete solution might more dynamically account for memory of various internal caches and vary their sizes based based on available memory and the overall server performance, rather than needing hard-coded values which are less than precise. [1]: "Approximate", because it's based on the `JSON.stringify`'d byte-length of the value being cached. [2]: The `queryHash` is a SHA-256 value of the incoming operation. * Add CHANGELOG.md for apollographql/apollo-server#3755. * docs: Add documentation for apollographql/apollo-server#3755. Apollo-Orig-Commit-AS: apollographql/apollo-server@21651bd
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This introduces two experimental configuration options for existing internal cache stores:
experimental_approximateDocumentStoreSizeMiBis added to theApolloServerconstructor options and controls the approximate[1] size, in MiB, of the request pipeline document store, which is used to avoid repeated parsing and validation passes on queries which equate to the samequeryHash[2].experimental_approximateQueryPlanStoreMiBis added to theApolloGatewayconstructor options and controls the approximate[1] size of the query plan cache, which is used to avoid recalculation on repeated queries which equate to the samequeryHash[2].These are currently experimental because a more flexible, zero-config approach could more effortlessly account for memory of various internal caches (e.g. these two caches together) and vary their sizes based based on available memory and the overall server performance, rather than needing hard-coded values which are less than precise and require experimentation and observability to set properly.
We have a number of internal cache stores right now that could benefit from such behavior:
I'm hoping some existing native object cache implementations may already exist as external libraries which we could rely on. I haven't been able to explore options yet, but such a library would be immensely useful to Apollo Server! For example, if a server where Apollo Server is running has 2GiB of free memory, it's a shame for us to not try to use it while it's available. Conversely, we have to be careful (as we are right now, out of respect for FaaS providers) about running with unbounded cache configurations on platforms where memory is constrainted.
Of course, the right solution will likely still allow manual configuration and limits, but a better zero-config option seems obtainable. If anyone is willing to explore and put together a proposal, please feel free to leave a comment or open an issue!
[1]: "Approximate", because it's based on the
JSON.stringify'd byte-length of the value being cached.[2]: The
queryHashis a SHA-256 value of the incoming operation.