Experimental: use Normalized fields instead of lazy collect field

[andimarek]

This change leverages the normalized query structure to collect the merged subfields instead of calling collectFields on demand.

This aims to improve the performance for especially large and deep queries with a lot of overlapping fields: the theory is that a one time build Normalized query is faster than the lazy/on demand collectFields code.

If this really improves the performance needs to be verified.

This potentially also enables caching of a normalized query which means the merging of overlapping fields could be cached.
One aspect to consider is that a NQ is currently not independent of the variables: it needs all variables present to be build.

[andimarek]

@dfa1 hey Davide .... It would be really great if you could try out this experimental version and let us know if it improved your performance. Thanks a lot

[dfa1]

@andimarek of course! It looks very interesting... :-)

Would be possible to publish this branch in maven central?

[andimarek]

@dfa1 It is published as 230521-nf-execution

[dfa1]

@andimarek there is a breaking change in a @PublicApi class:

  symbol:   method getObjectType()
  location: variable field of type graphql.schema.SelectedField

Not sure if this replacement is good enough for production usage:

field.getObjectTypes().get(0)

[dfa1]

another breaking change is:

symbol:   method getFieldDefinition()
location: variable field of type graphql.schema.SelectedField

[andimarek]

@dfa1 yes, this PR relies on two other PRs: #2338 and #2325. The latter contains some breaking changes.

[dfa1]

@andimarek using yourkit with cpu tracing profilter. I cannot see anymore the "collectField" hotspot. This is really nice result! 🥇

However, I can't push this version up to the integration env since I'm not sure about my fixes for the breaking changes in SelectedField. The PR you mentioned doesn't change SelectedField at all... please advise :)

[andimarek]

hi @dfa1 .... I am sorry, the PR with the breaking changes is this: #2345

SelectedField now can reference multiple object types and not just one: depending on your schema and use case you might be ok to just use the first one, but it really depends.

[bbakerman]

This feels like a bit of code that could be in NormalizedQueryTree

MergedSelectionSet subFields = normalizedQueryTree.getSelectionSet(normalisedField, resolvedObjectType);

[dfa1]

@andimarek what is the result of this experiment?

[timward60]

This potentially also enables caching of a normalized query which means the merging of overlapping fields could be cached.
One aspect to consider is that a NQ is currently not independent of the variables: it needs all variables present to be build.

Super interested to understand the feasibility and any thoughts on how it could be achieved (especially what would need to be break out the query runtime variables).

Our project relies on DataFetchingSelectionSets with some fairly large/deep queries. We are seeing a large portion of time spent calculating this, so if we can cache this it would be a large win.

[github-actions]

Hello, this pull request has been inactive for 60 days, so we're marking it as stale. If you would like to continue working on this pull request, please make an update within the next 30 days, or we'll close the pull request.