Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple gateways in the same process #261

Closed
wants to merge 1 commit into from

Conversation

Enrico2
Copy link
Contributor

@Enrico2 Enrico2 commented Oct 28, 2020

This PR makes it so that getting a query planner creates a new planner every time and never overrides the global instance.
However, because ApolloGateway can roll over its schema on the fly, we've added a new function named updatePlannerSchema that overrides a specific planner.

I opted for the simplest solution that solves the problem described in #210 , without having to manage memory from JS and implementing complex freeing/reference counting.

This hopefully fixes #210
This implementation is suggested instead of #226 to keep things simple.

This PR also moves away from using raw pointers, to using an index to an array. That was a good suggestion in #226 as well.

@jaredly Can you verify this works for you? As far as I could tell, this solves the problem, and I've used the test you wrote in #226 (thanks for that!). I'd like to get confirmation from you before I merge. Thanks!

This PR makes it so that getting a query planner creates a new planner every time and never overrides it.
We've added a new function named `updatePlannerSchema` that overrides a specific planner.

This hopefully fixes #210
This implementation is suggested instead of #226 to keep things simple.
@Enrico2 Enrico2 marked this pull request as ready for review October 28, 2020 23:22
#[wasm_bindgen_test]
fn multiple_query_planners() {
let schema_multiple_keys = include_str!(
"../../stargate/crates/query-planner/tests/features/multiple-keys/csdl.graphql"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason we're stringly typing paths

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this seems to pass on windows but i am surprised)

let planner_basic = get_query_planner(JsString::from(schema_basic));
let query_basic = "query { me { name } }";

let options = QueryPlanningOptionsBuilder::default().build().unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should at least make these unwraps expects.. if these were to fail it would be very unpleasant to debug

@abernix
Copy link
Member

abernix commented Nov 2, 2020

@jaredly You should be able to test this using the npm tarball artifacts that were generated in the CI build process. Specifically, you probably need to npm install (or yarn add):

  • @apollo/query-planner-wasm (which @apollo/gateway depends on):

    npm install https://990-252760990-gh.circle-artifacts.com/0/packed/%40apollo/query-planner-wasm/query-planner-wasm-0.0.6.tgz
    
  • @apollo/gateway:

    npm install https://990-252760990-gh.circle-artifacts.com/0/packed/%40apollo/gateway/gateway-0.21.0.tgz
    
  • and maybe @apollo/federation:

    npm install https://990-252760990-gh.circle-artifacts.com/0/packed/%40apollo/federation/federation-0.20.4.tgz
    

Note, these will expire 30 days after they were generated, so don't use these beyond just a quick verification that they solve the problem this PR aims to fix.

@dkapadia
Copy link

dkapadia commented Nov 2, 2020

@abernix thank you for the clear steps on how to test this out locally! FYI for anyone else testing, I also found I needed to add the following to my package.json to have the correct library picked up by the @apollo/gateway library, otherwise it seemed to use an older version that was somehow still included in my local node_modules.

    "resolutions": {
      "@apollo/gateway/@apollo/query-planner-wasm": "https://990-252760990-gh.circle-artifacts.com/0/packed/%40apollo/query-planner-wasm/query-planner-wasm-0.0.6.tgz"
    }

Copy link
Contributor

@ashleygwilliams ashleygwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so i generally think that this static mut architecture is a liability but for the purposes of changing this so that it works for the moment, this seems Fine.

@abernix abernix assigned abernix and trevor-scheer and unassigned abernix Feb 8, 2021
@abernix abernix requested review from abernix and removed request for queerviolet February 11, 2021 15:37
Copy link
Contributor

@martijnwalraven martijnwalraven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know much about best practices for interactions between JavaScript and WebAssembly, but I'm wondering if there's a way to avoid manually managing pointers on the Rust side.

Looking at the was-bindgen documentation, it seems annotating functions should create JavaScript bridge classes for structs. That class also has a free() method that can be used to deallocate memory on the Rust side.

Relying on the wasm-bindgen generated code may also make it easier to switch to weak references or the new externref type if/when that makes sense.

The difficulty here is that query-planner-wasm has the wasm-bindgen dependency, not the query-planner crate it depends on. And we may not want to have query-planner depend on wasm-bindgen. Not sure if there's a way to still have wasm-bindgen generate code for a function defined in another crate.

@abernix
Copy link
Member

abernix commented Apr 16, 2021

This should no longer be necessary with the landing of #622 (specifically, the changing of the query planner back to TypeScript). Put another way, the code this changes is no longer present and the query planners are now scoped within the Gateways that generate and execute them.

@abernix abernix closed this Apr 16, 2021
@abernix abernix deleted the ran/support_multiple_gateways branch April 16, 2021 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for multiple wasm query planner objects?
6 participants