[GraphQL] The consumer contract test fails always when the query has an input on the third-level

[OlgaLa]

Software versions

Please provide at least OS and version of pact-js

• Consumer Pact library: @pact-foundation/pact: 11.0.2
• Node Version: v18.17.0

Issue Checklist

Please confirm the following:

• [+] I have upgraded to the latest
• [+] I have the read the FAQs in the Readme
• [+] I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
• [+] I have set my log level to debug and attached a log file showing the complete request/response cycle
• For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Expected behaviour

I have a consumer contract test for a query with specific input. Please see the query example in the Steps. And the test fails always with 500 error code:

    GraphQL Error (Code: 500): {"response":{"error":"Request-Mismatch : HttpRequest { method: \"POST\", path: \"/graphql\", query: None, headers: Some({\"content-length\": [\"559\"], \"accept\": [\"*/*\"], \"user-agent\": [\"node-fetch/1.0 (+https://github.com/bitinn/node-fetch)\"], \"accept-encoding\": [\"gzip\", \"deflate\"], \"connection\": [\"close\"], \"host\": [\"127.0.0.1:3200\"], \"content-type\": [\"application/json\"]}), body: Present(b\"{\\\"query\\\":\\\"query getItems($id: ID!, $startTime: Time!, $endTime: Time!, $status: Status) {\\\\n  element(id: $id) {\\\\n    config {\\\\n      items(\\\\n        filter: {timeRange: {start: $startTime, end: $endTime}, status: $status}\\\\n      ) {\\\\n        edges {\\\\n          node {\\\\n            id\\\\n          }\\\\n        }\\\\n      }\\\\n    }\\\\n  }\\\\n}\\\",\\\"variables\\\":{\\\"id\\\":\\\"123\\\",\\\"startTime\\\":\\\"2024-01-01T00:00:00Z\\\",\\\"endTime\\\":\\\"2024-01-01T07:09:04Z\\\",\\\"status\\\":\\\"PAYED\\\"},\\\"operationName\\\":\\\"getItems\\\"}\", Some(ContentType { main_type: \"application\", sub_type: \"json\", attributes: {}, suffix: None }), None), matching_rules: MatchingRules { rules: {} }, generators: Generators { categories: {} } }","status":500,"headers":{}},"request":{"query":"query getItems($id: ID!, $startTime: Time!, $endTime: Time!, $status: Status) {\n  element(id: $id) {\n    config {\n      items(\n        filter: {timeRange: {start: $startTime, end: $endTime}, status: $status}\n      ) {\n        edges {\n          node {\n            id\n          }\n        }\n      }\n    }\n  }\n}","variables":{"id":"123","startTime":"2024-01-01T00:00:00Z","endTime":"2024-01-01T07:09:04Z","status":"PAYED"}}}

Actual behaviour

The test should pass. I added tests for another queries but they do not have the same input.

Steps to reproduce

I want to add a consumer contract test for graphql query.
Here is the example of the query:

query getItems($id: ID!, $startTime: Time!, $endTime: Time!, $status: Status) {
  element(id: $id) {
    config {
      items(filter: { timeRange: { start: $startTime, end: $endTime }, status: $status }) {
        edges {
          node {
            id
          }
        }
      }
    }
  }
}

I added the following tests:

describe("description", () => {
        beforeEach(() => {
          const graphqlQuery = new GraphQLInteraction()
            .given("given")
            .uponReceiving("items")
            .withOperation("getItems")
            .withQuery(
                `query getItems($id: ID!, $startTime: Time!, $endTime: Time!, $status: Status) {
                      element(id: $id) {
                        config {
                          items(filter: { timeRange: { start: $startTime, end: $endTime }, status: $status }) {
                            edges {
                              node {
                                id
                              }
                            }
                          }
                         }
                        }
                      }`
            )
            .withRequest({
              path: "/graphql",
              method: "POST"
            })
            .withVariables({
              id: "123",
              startTime: "2024-01-01T00:00:00Z",
              endTime: "2024-01-01T07:09:04Z",
              status: Status.PAYED, 
            })
            .willRespondWith({
              status: 200,
              headers: {
                "Content-Type": "application/json; charset=utf-8"
              },
              body: {
                data: {
                    element: {
                        config: [
                            {
                              items: {
                                edges: [
                                  {
                                    node: {
                                      id: Matchers.somethingLike("456987")
                                    }
                                  }
                                ]
                              }
                            }
                          ]
                    }
                }
              }
            });
          return provider.addInteraction(graphqlQuery);
        });
    
        it("returns the correct response", async () => {
          const pactUrl = provider.mockService.baseUrl;
          const sdk = getSdk(new GraphQLClient(`${pactUrl}/graphql`));
          const resp = JSON.stringify(
            await sdk.getItems({
                id:"123",
                startTime: "2024-01-01T00:00:00Z",
                endTime: "2024-01-01T07:09:04Z",
                status: Status.PAYED
            })
          );
          expect(resp).toContain("456987");
        });
    });

The test fails always with 500 error.

[mefellows]

Thanks for this. Could you please see if you can create a minimal version of this that suffers the same fate?

Can you explain a bit further what you mean by "third level"?

[OlgaLa]

Regarding "third level" I mean items level:

...
        items(filter: { timeRange: { start: $startTime, end: $endTime }, status: $status }) 
...

Items have a filter by timeRange and status and I think it is the place which causes the 500 error.

I have also tests for the queries like this:

query getIssues($id: ID!, $startTime: Time!, $endTime: Time!) {
  element(id: $id) {
    issues(where: { timeRange: { start: $startTime, end: $endTime } }) {
      edges {
        node {
          id
        }
      }
    }
  }
}

and I do not have any issues with tests.

But once again, if you compare these two queries there is only one difference - the level where the filter is used.

• in the problem query:

query getItems($id: ID!, $startTime: Time!, $endTime: Time!, $status: Status) {
                      element(id: $id) {    // >>>>>> 1 level
                        config {                  // >>>>>> 2 level
                          items(filter: { timeRange: { start: $startTime, end: $endTime }, status: $status }) { // >>>>>> 3 level

• another query:

query getIssues($id: ID!, $startTime: Time!, $endTime: Time!) {
  element(id: $id) {    // >>>>>> 1 level
    issues(where: { timeRange: { start: $startTime, end: $endTime } }) {                  // >>>>>> 2 level

I hope it is more clear now.

"Could you please see if you can create a minimal version of this that suffers the same fate?"
Sorry, I did not get it. What do you mean?

[mefellows]

"Could you please see if you can create a minimal version of this that suffers the same fate?"
Sorry, I did not get it. What do you mean?

From the new issue template:

Think of it this way - as a maintainer, we have heaps on our plate. If you can create a github repo that we can clone that reproduces your issue, now all we need to do is focus on fixing the problem (or identifying the cause of the issue).

---

Steps to reproduce

How can someone else reproduce this bug?

Provide a Minimal, Reproducible Example. You can create your own repository, or use this template that's already hooked up to CI and everything.

[OlgaLa]

Ok, I will try to create a repo which you can use to reproduce the issue. But how do you test your releases? Looks like you do not check this case.

[YOU54F]

you've probably answered your own question ;)

there is a graphql example project, if your case isn't covered, you can add it in so that it

1. reproduces the issue
2. allows someone to resolve the issue
3. tests the issue in future for regressions

releases are tested in ci, you can see previous action runs

[mefellows]

But how do you test your releases? Looks like you do not check this case.

We don't test every possible GraphQL query. GraphQL is just an abstraction over HTTP. The query property structure itself is not something Pact understands, but expects the request you sent to match what you configured in the test. Pact JS makes checking this a bit less brittle, by allowing additional whitespace. This is achieved through a regex.

So it's possible the regex is not properly working across levels, although I suspect that's just coincidence and the issue is unrelated to levels. The repro will help us figure this out, and if tests need to be adjusted to prevent future regressions.

[mefellows]

Closing due to inactivity.