experimental: Allow query plan & parsed/validated document cach… (#3755)

* experimental: Allow adjusting of query plan & parsed/validated document cache.

This introduces two experimental configuration options for existing internal
cache stores:

- `experimental_approximateDocumentStoreSizeMiB` is added to the `ApolloServer`
  constructor options and controls the _approximate_[1] size, in MiB, of the
  request pipeline document store, which is used to avoid repeated parsing and
  validation passes on queries which equate to the same `queryHash`[2].
- `experimental_approximateQueryPlanStoreMiB` is added to the `ApolloGateway`
  constructor options and controls the _approximate_[1] size of the query plan
  cache, which is used to avoid recalculation on repeated queries which equate
  to the same `queryHash`[2].

These are currently experimental because a more complete solution might
more dynamically account for memory of various internal caches and vary
their sizes based based on available memory and the overall server
performance, rather than needing hard-coded values which are less than
precise.

[1]: "Approximate", because it's based on the `JSON.stringify`'d byte-length
     of the value being cached.
[2]: The `queryHash` is a SHA-256 value of the incoming operation.

* Add CHANGELOG.md for #3755.

* docs: Add documentation for #3755.

CHANGELOG.md

@@ -9,6 +9,7 @@ The version headers in this history reflect the versions of Apollo Server itself
 - Move TContext generic from requestDidStart method to ApolloServerPlugin Interface [#3525](https://github.com/apollographql/apollo-server/pull/3525)
 - `apollo-server-express`: Support `CorsOptionsDelegate` type on `cors` parameter to `applyMiddleware`, to align with the supported type of the underlying [`cors`](https://npm.im/cors) middleware [itself](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/31483b781ac30f98bdf4d40a517e921f2fc2ce37/types/cors/index.d.ts#L32). [PR #3613](https://github.com/apollographql/apollo-server/pull/3613)
 - `apollo-server-core`: Allow asynchronous initialization of datasources: the `initialize` method on datasources may now return a Promise, which will be settled before any resolvers are called. [#3639](https://github.com/apollographql/apollo-server/pull/3639)
+- `apollo-server-core`: experimental: Allow configuration of the parsed/validated document store by introducing an `experimental_approximateDocumentStoreMiB` property to the `ApolloServer` constructor options which overrides the default cache size of 30MiB. [#3755](https://github.com/apollographql/apollo-server/pull/3755)
 
 ### v2.9.16
 

docs/source/api/apollo-gateway.mdx

@@ -91,6 +91,18 @@ example of using `ApolloGateway`, see [Implementing a federated graph](/federati
     If `true`, the gateway logs startup messages, along with the query plan for
     each incoming request. The default value is `false`.
 
+  * `experimental_approximateQueryPlanStoreMiB`: `number`
+
+    > **This property is experimental.**  It may be removed or change at any time, even within a patch release.
+
+    When set, this sets the approximate size of the query plan store (in MiB).
+    This cache is used to save query plans for re-use on subsequent queries
+    which resolve to the same `queryHash` (a SHA-256 of incoming operation).
+
+    When this property is omitted, the cache is still enabled with a default
+    size of 30MiB, which is generally sufficient unless the server is
+    processing a high number of unique operations.
+
 #### Returns
 
 An `ApolloGateway` instance, which you then pass as the `gateway` configuration option to the `ApolloServer` constructor, like so:

docs/source/api/apollo-server.md

@@ -144,6 +144,19 @@ new ApolloServer({
 
   Pass the integration-specific CORS options. `false` removes the CORS middleware and `true` uses the defaults. This option is only available to `apollo-server`. For other server integrations, place `cors` inside of `applyMiddleware`.
 
+* `experimental_approximateDocumentStoreSizeMiB`: `number`
+
+  > **This property is experimental.**  It may be removed or change at any time, even within a patch release.
+
+  When set, this sets the approximate size of the parsed/validated document
+  store (in MiB).  This cache is used to save the already parsed and validated
+  `DocumentNode`s for re-use on subsequent queries which resolve to the same
+  `queryHash` (a SHA-256 of incoming operation).
+
+  When this property is omitted, the cache is still enabled with a default
+  size of 30MiB, which is generally sufficient unless the server is processing
+  a high number of unique operations.
+
 #### Returns
 
 `ApolloServer`

packages/apollo-gateway/CHANGELOG.md

@@ -7,6 +7,7 @@
 * Reduce interface expansion for types contained to a single service [#3582](https://github.com/apollographql/apollo-server/pull/3582)
 * Instantiate one `CachedFetcher` per gateway instance.  This resolves a condition where multiple federated gateways would utilize the same cache store could result in an `Expected undefined to be a GraphQLSchema` error. [#3704](https://github.com/apollographql/apollo-server/pull/3704)
 * Gateway: minimize downstream request size [#3737](https://github.com/apollographql/apollo-server/pull/3737)
+* experimental: Allow configuration of the query plan store by introducing an `experimental_approximateQueryPlanStoreMiB` property to the `ApolloGateway` constructor options which overrides the default cache size of 30MiB. [#3755](https://github.com/apollographql/apollo-server/pull/3755)
 
 # v0.11.6
 

packages/apollo-gateway/src/index.ts

@@ -58,6 +58,7 @@ interface GatewayConfigBase {
   experimental_updateServiceDefinitions?: Experimental_UpdateServiceDefinitions;
   experimental_didUpdateComposition?: Experimental_DidUpdateCompositionCallback;
   experimental_pollInterval?: number;
+  experimental_approximateQueryPlanStoreMiB?: number;
 }
 
 interface RemoteGatewayConfig extends GatewayConfigBase {
@@ -178,6 +179,8 @@ export class ApolloGateway implements GraphQLService {
   // how often service defs should be loaded/updated (in ms)
   protected experimental_pollInterval?: number;
 
+  private experimental_approximateQueryPlanStoreMiB?: number;
+
   constructor(config?: GatewayConfig) {
     this.config = {
       // TODO: expose the query plan in a more flexible JSON format in the future
@@ -219,6 +222,9 @@ export class ApolloGateway implements GraphQLService {
       this.experimental_didUpdateComposition =
         config.experimental_didUpdateComposition;
 
+      this.experimental_approximateQueryPlanStoreMiB =
+        config.experimental_approximateQueryPlanStoreMiB;
+
       if (
         isManagedConfig(config) &&
         config.experimental_pollInterval &&
@@ -609,7 +615,9 @@ export class ApolloGateway implements GraphQLService {
       // only using JSON.stringify on the DocumentNode (and thus doesn't account
       // for unicode characters, etc.), but it should do a reasonable job at
       // providing a caching document store for most operations.
-      maxSize: Math.pow(2, 20) * 30,
+      maxSize:
+        Math.pow(2, 20) *
+        (this.experimental_approximateQueryPlanStoreMiB || 30),
       sizeCalculator: approximateObjectSize,
     });
   }

packages/apollo-server-core/src/ApolloServer.ts

@@ -153,6 +153,8 @@ export class ApolloServerBase {
   /** @deprecated: This is undefined for servers operating as gateways, and will be removed in a future release **/
   protected schema?: GraphQLSchema;
   private toDispose = new Set<() => void>();
+  private experimental_approximateDocumentStoreMiB:
+    Config['experimental_approximateDocumentStoreMiB'];
 
   // The constructor should be universal across all environments. All environment specific behavior should be set by adding or overriding methods
   constructor(config: Config) {
@@ -176,6 +178,7 @@ export class ApolloServerBase {
       playground,
       plugins,
       gateway,
+      experimental_approximateDocumentStoreMiB,
       ...requestOptions
     } = config;
 
@@ -716,7 +719,9 @@ export class ApolloServerBase {
       // only using JSON.stringify on the DocumentNode (and thus doesn't account
       // for unicode characters, etc.), but it should do a reasonable job at
       // providing a caching document store for most operations.
-      maxSize: Math.pow(2, 20) * 30,
+      maxSize:
+        Math.pow(2, 20) *
+        (this.experimental_approximateDocumentStoreMiB || 30),
       sizeCalculator: approximateObjectSize,
     });
   }

packages/apollo-server-core/src/types.ts

@@ -112,6 +112,7 @@ export interface Config extends BaseConfig {
   uploads?: boolean | FileUploadOptions;
   playground?: PlaygroundConfig;
   gateway?: GraphQLService;
+  experimental_approximateDocumentStoreMiB?: number;
 }
 
 export interface FileUploadOptions {