Schema-first GraphQL mocking

[kettanaito]

Although you can use the existing graphql.operation() handler + the graphql package to resolve any query against a mock schema, this approach has a number issues:

1. It's not accessible. A lot of developers don't even know you can do that;
2. It's not flexible enough to cover real-world test cases. Developers need type/resolver-based mock data, fallback data for various scalars. etc. Basically, a composable schema-driven mock system.

In this regard, I propose to kick off this discussion to improve the schema-firsts mocking in GraphQL.

The goal

To make GraphQL testing easier in existing apps. Mocking operation-first is beneficial when prototyping as you don't have to invest in schemas/resolvers/etc. but when testing an existing apps, the operation-based approach doesn't scale that well. This is where a schema-based approach is preferable.

Existing apps would also likely have the schema defined in some shape or form, so reusing that schema could make the mocking faster and error-proof.

Initial proposal

I think it'd be good to add a new graphql.schema request handler.

import { graphql } from 'msw'

const api = graphql.schema(
  `
  type User {
    id: ID!
    name: String!
  }

  type Query {
    user(id: ID!): User
  }
  `,
  {
    // Mock data for the "User" type.
    // Gets returned anywhere you query this type.
    User(root, args, ctx) {
      return { id: crypto.uuid(), name: faker.name() }
    },
    // Mock data for the "user" query.
    user(root, args, ctx) {
      return {} // return a User object.
    },
  }
)

This API may act as a fallback definition for your schema (mocked or not) with the default mock data described in the second argument (essentially, rootValue). This API may also return a subset of the graphql API in MSW to allow the developer to override particular types/scalars/resolvers/operations on a per-test basis.

For example, if I want for User.name to return a different value in a particular test, I can do this:

const api = graphql.schema(schema, rootValue)

it('my test', () => {
  // The "name" field on the "User" type will always
  // return the string literal "John" within this test.
  api.field(['User', 'name'], () => 'John')
})

We can have things like .field(), .query() and .mutation() (if I want to override the entire operation but still use the fallback graphql.schema to resolve the fields I haven't provided in the override.

Core principles

Like the rest of MSW, I want this API to resemble the way you'd actually implement GraphQL. This means nailing this tricky balance between the accessibility of the mocks and the relevance of the code you produce. The more we rely/replicate the principles present in GraphQL development (maybe even some of the APIs used), the more beneficial this API becomes to the developer in terms of their effort investment.

[kettanaito]

Some of the current questions I have about this proposal:

1. How does this new API sits relatively to the rest of the MSW's logic? When/how would it return a request handler? Would you still want to be able to access the actual GraphQL request or do you opt-in in the schema-driven world and treat requests as implementation detail?
2. Making it all type-safe. This shouldn't be a problem if using types generated from the schema (with tools like GraphQL Code Generator). This largely comes down to the format of the schema, which I recommend to be either a template string written in GraphQL SDL or a buildSchema() function call over that template string from the graphql package itself.
3. How to combine this cascading data setup with the actual handlers? Should it be baked in into a handler or just an external function that does all this composition of the default/fallback values + override values and then returns a resolver of sorts that MSW just calls on the request.
4. How do you mock field-level errors with this?

[kettanaito]

How do you mock field-level errors with this?

I think returning an error may just be the most straightforward API:

// Returns an error for the entire "User" type.
api.field(['User'], () => new Error('Failed to fetch a user'))

// Returns an error for a particular field in the "User" type.
// Other User fields won't be affected. Handy for partial successful responses. 
api.field(['User', 'messages', () => new Error('Network error'))

[kettanaito]

@phryneas, does this look like similar to the direction you initially proposed?

[phryneas]

How does this new API sits relatively to the rest of the MSW's logic? When/how would it return a request handler? Would you still want to be able to access the actual GraphQL request or do you opt-in in the schema-driven world and treat requests as implementation detail?

This is something that we didn't really touch on in our call, but that is actually super important:
We somehow need a split between a "real graphql schema" and "msw handlers" here. While msw is great, there are situations where it cannot be used or is just the wrong tool for the job - but such a mocking schema helper is still very useful, e.g. when someone uses ApolloClient, but cannot use msw they could still use a realgraphql schema with the ApolloClient mockLink.

So this should probably be split up in

• msw.graphql.schema(url?, schema) - returns a handler
• createMockSchema(SDLstring) - creates a basic mock schema that can then be enhanced on a Scalar, Type and field level.

At that point, also the offer from our side: We would be very happy to flesh out & maintain the createMockSchema part of this - that would make it possible for msw to concentrate on the network layer, while we have a very direct line to a lot of users for user feedback on the graphql side.

Making it all type-safe.

Yeah, graphql-codegen can generate all the types we need here.

How to combine this cascading data setup with the actual handlers? Should it be baked in into a handler or just an external function that does all this composition of the default/fallback values + override values and then returns a resolver of sorts that MSW just calls on the request.

I would keep this as a "lowest level" handler, and people could still use the existing graphql.operation handlers on top (once) in their tests if it makes sense in specific tests

How do you mock field-level errors with this?

There isn't really a 100% standardized form for those in the first place, so we could provide a ctx.addError callback in every resolver and the resolver itself could return null. On top of that we could allow for throwing errors from resolvers, but errors of that kind might not 100% resemble errors coming from the real backend, so it might be of questionable value.

Like the rest of MSW, I want this API to resemble the way you'd actually implement GraphQL. This means nailing this tricky balance between the accessibility of the mocks and the relevance of the code you produce. The more we rely/replicate the principles present in GraphQL development (maybe even some of the APIs used), the more beneficial this API becomes to the developer in terms of their effort investment.

💯! That's why I want this to resemble "real resolvers" as closely as possible - just without the need to create every last resolver, but to automock the bulk of them.

[phryneas]

PS: we could also make this "low-effort" standalone:

graphql.useSchema(url?: string, executeConfig: {
  schema: GraphQLSchema | string,
  contextValue?: object | (req: Request) => object // this deviates a bit from `execute`, but the callback here probably makes sense,
  rootValue?: Record<string, resolver>
})

=> in case schema is a string, it would be run through buildSchema internally to convert it to a basic GraphQLSchema that would then fall back to rootValue. (See this example from the graphql docs that is so far at the start of the tutorials series that I forgot this even was a thing ^^)

PS to the PS: I will freely admit that I haven't written a schema from ground up in a while, so in the whole discussion here I might be missing the point on some things (I think GraphQLSchema probably has error handling included, which is different from the answer I gave above). I'll have to put some time aside and dig into this, to get myself up to speed again. Too much client developement, too little server dev ;)

[kettanaito]

Perhaps one thing we can agree from the very beginning: endpoint URL won't be a part of the new graphql.schema API in MSW. Instead, you can scope the schema execution to a particular endpoint using graphql.link, which builds on top of the already familiar experience:

// Resolves ANY operations against this schema.
graphql.schema(...args)

const github = graphql.link('https://api.github.com/graphql')
// Resolves only those operations whose endpoint URL
// matches the one provided in the .link call.
github.schema(...args)

My reasoning here is the same as the reasoning behind not making GraphQL mocking endpoint-first in MSW: clients rarely communicate with more than one GraphQL server. But in case they do, we have graphql.link as an explicit choice to use.

There is a "standard in the ecosystem", and that's essentially graphql's GraphQLSchema type, as returned from new GraphQLSchema({ query, mutation}) or makeExecutableSchema.

That's precisely what I meant. We should likely build on top of the GraphQLSchema, which you can achieve by buildSchema/makeExecutableSchema. The input nature can also vary and I'm okay with this, just like we currently support DocumentNode and TypedDocumentNode as inputs to graphql.query/graphql.mutation handlers for convenience.

Regarding the API shape, I think we can satisfy both standalone behavior and MSW + Apollo if we make the second argument to graphql.schema a function of intercepted request.

// If the second argument is a plain object,
// it acts as a literal object of root values.
// This way you're getting schema-first mocking
// without the need to invest into the layered
// mock setup we're brainstorming right now. 
graphql.schema(schema: string, rootValue: object)

// If the second argument is a function, MSW
// exposes all the GraphQL request information to it,
// similar as it does to other GraphQL request handlers,
// and expects that function to return the `rootValue` object
// for this particular operation.
graphql.schema(
  schema: string,
  resolver: (info: {
    request: Request,
    query: string,
    operationName: string,
    varaibles: object
  }) => object
)

This way you have an accessible point of entry with a plain rootValue object, and also a more flexible, powerful API with the resolver function. It's into that resolver function that our layered mock API can fit nicely: it will accept all the info about the request and the underlying package would be responsible for resolving temporary field overrides and etc before yielding the root value to resolve the incoming query against.

[alessbell]

Jumping into this great discussion :)

This way you have an accessible point of entry with a plain rootValue object, and also a more flexible, powerful API with the resolver function.

The ability to define resolvers is one of the reasons I'm exploring what is currently possible with makeExecutableSchema with its resolvers API. graphql-tools is a widely used graphql mocking library that provides overlapping functionality with some of the ideas we're discussing, so I'm eager to find out what its limitations are when combined with MSW, or possibly consider an API that can be built on top of it for more flexible mocking/unmocking in individual tests, and ultimately what the gaps where MSW can play a role.

When pulling in @graphql-tools/schema I ran into the error Ensure that there is only one instance of "graphql" in the node_modules directory. graphql/graphql-js#2801. My graphql package looks to be deduped (confirmed via npm ls graphql), and I'm forcing it anyway via vite's dedupe config, so there must be something I'm missing. Currently digging into this a bit more to make sure I don't have anything misconfigured, but that linked issue also notes the way graphql does its instanceof check requires a shared instance of the constructor. I'm using setupServer, so if anything jumps out at you @kettanaito let me know :)

[kettanaito]

Appreciate your input on this!

I'm eager to find out what its limitations are when combined with MSW

This is a great exploration effort, let me know if you find any limitations. As far as I know, there's nothing MSW would limit the developer in, request-wise. We expose all the information we can get (query, operation name, variables).

There may be setup obstacles, such as the duplicated graphql package you mentioned. This should be solved on the developer's level, although I agree this is an annoyance. MSW treats graphql as a dependency, so the consumer may have another version installed. I wonder if minor/patch version mismatches produce that duplicate graphql package error too? That'd be unfortunate.

I wish JavaScript ecosystem had a proper support for optional peer dependencies. There's some package.json magic but none of that is reliable and is up to the package manager to interpret.

[alessbell]

In my case it wound up being related to the "dual package hazard" of how graphql ships both CJS/ESM in v16: Vite understands pkg.module, but Node doesn't and loads pkg.main even in ESM code, so some .js files were being imported by graphql-tools libraries instead of the equivalent .mjs files. For posterity: I used patch-package to force graphql-tools to import from graphql/index.mjs, but apparently this also works.

MSW treats graphql as a dependency, so the consumer may have another version installed. I wonder if minor/patch version mismatches produce that duplicate graphql package error too?

This came up, too: graphql's v17 alphas only ship ESM, so when I went to try that version I ran into one of the other duplicated package cases where MSW installed its own copy. I think most other graphql-dependent packages have it either as a peerDependency or optional peerDependency for this reason, so that would be a helpful adjustment if possible.

This is a great exploration effort, let me know if you find any limitations.

This works really nicely! I have a src/mocks/handlers.ts that looks like this:

import { graphql } from 'msw';
import { graphql as graphqlRequest, buildSchema } from 'graphql';
import { addMocksToSchema } from '@graphql-tools/mock';
import { makeExecutableSchema } from '@graphql-tools/schema';
import SpotifySchema from '../../schema.graphql';

const schema = buildSchema(SpotifySchema?.loc?.source.body || '');

const mocks = {
  Int: () => 6,
  Float: () => 22.1,
  String: () => 'string',
};

const resolvers = {
  Query: {
    // resolved: () => 'Resolved',
  },
};

const executableSchema = addMocksToSchema({
  schema: makeExecutableSchema({
    typeDefs: schema,
    resolvers,
  }),
  mocks,
});

export const handlers = [
  graphql.operation(async (req, res, ctx) => {
    const { query } = await req.json();
    const payload = await graphqlRequest({
      schema: executableSchema,
      source: query,
      variableValues: req.variables,
    });

    return res(ctx.data(payload.data || {}));
  }),
];

Which allows me to render a full Apollo Client app without mocking individual operations 🎉 The next thing I'm exploring is how we can adjust the resolvers/mocks in individual tests, I'll share my findings next week :)