globby returns inconsistent order of found files

[yoavain-sundaysky]

Describe the bug
Each time I'm running the graphql-codegen, I get a different result.
All results are valid, but order of types changes.

This creates 2 problems for us.

1. We think we need to commit generated files each time we run the tool.
2. I wrote a test that makes sure all developers are running the tool and commit the new generated files, and the test is broken now..

I went over your changes from 1.4 to 1.5 and my guess is this:
You replaced glob with globby, and from a quick search I found this bug that suggests that globby results are non-deterministic:
sindresorhus/globby#131

To Reproduce

1. Run graphql-codegen, commit result.
2. Run graphql-codegen again, check for changes.

Our schema is too complex to share, but if you can't reproduce it, I'll try to create a live sandbox.

My codegen.yml config file:

overwrite: true
schema:
  - ./server/graphql/**/*.graphql
documents:
  - ./client/**/*.graphql
generates:
  ./common/graphqlGeneratedTypes/graphqlClient.tsx:
    config:
      withComponent: false
      typesPrefix: "GqlClient"
      nonOptionalTypename: true
      namingConvention:
        typeNames: change-case#pascalCase
        enumValues: change-case#upperCase
    plugins:
      - typescript
      - typescript-operations
      - typescript-react-apollo

Expected behavior
consistent generated files.

Environment:

• OS: Windows
• @graphql-codegen/1.5.0:
• NodeJS: 10.x

[ardatan]

What is the difference between outputs? There are some changes in 1.5, so that might cause a diff. Is there any specific issue like TS error or something?

[yoavain-sundaysky]

@ardatan the issue is not between 1.4 and 1.5.
I'm okay with the additions of the 1.5 version.
Problem is the non-deterministic result.
Types are ordered differently every time running the codegen of version 1.5.

[dotansimha]

Hi @yoavain-sundaysky !

Why should the order matter? the generated file shouldn't go to source-control, so you shouldn't get effected by the order of the generated output.

We are using globby now because it provides a better solution for specifying multiple glob patterns, and it's easier to exclude files with it.

I'm not sure if we can introduce a solution for that at the moment (maybe just sorting it will work?) - but if your generated files will be excluded from source-control - you shouldn't get effected by that.

[dotansimha]

@yoavain-sundaysky I added documents sort by file path, so it might solve this issue.
Can you please try 1.5.1-alpha-a6dbc965.21?

[jkillian]

Just FWIW, we also check in our generated types! And then we have CI generate them as well, and assert that the generated types are equal to the checked-in ones. This is a nice sanity check for us to make sure that everyone is operating with a correct version of types and has the correct local environment.

In other words, it's necessary for us to have a stable file generation process also. So thanks for the fix!

[yoavain-sundaysky]

Thanks @dotansimha,
1.5.1-alpha-a6dbc965.21 generated files seems to be consistent again.

To answer your previous questions, we currently have these generated files committed (maybe it's bad practice. I'm not sure). Because of that, we have a test that makes sure that if someone changes schema or document files, he can't forget to commit these files...

Thanks for the fix

[dotansimha]

Fixed in 1.6.0

[yoavain-sundaysky]

I'm still having issues with consistency. This time it's in the fields order. Not sure if it's the same issue.

Example:
_?: Maybe<Scalars["Boolean"]>; (under GqlQuery)
appears sometimes at the beginning and sometimes at the end.

Example 2:
Order of ResolverTypeWrapper under GqlResolversTypes changes between executions.

Example 3:
Order of fields of GqlResolversParentTypes changes between executions.

Tested on 1.7.0

@dotansimha?

[dotansimha]

@yoavain-sundaysky let's track this here: #2402

[dotansimha]

@yoavain-sundaysky note that we fixed that, you can try the latest alpha.
We'll release 1.8.0 soon :)