Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: avoid early release of PostgresQueryRunner (#7109) #7185

Merged
merged 4 commits into from
Dec 29, 2020
Merged

fix: avoid early release of PostgresQueryRunner (#7109) #7185

merged 4 commits into from
Dec 29, 2020

Conversation

alvisetrevisan
Copy link
Contributor

@alvisetrevisan alvisetrevisan commented Dec 13, 2020

Don't release the queryRunner too early from SelectQueryBuilder when streaming queries.

Description of change

Fixes #7109

In release 0.2.26, streaming queries for postgres were broken. The problem is that SelectQueryBuilder releases the queryRunner too early (when it might not be connected yet), and therefore prevents release at the appropriate moment (using the release callback) to happen (since it will think that it's already released, and quit).

This PR fixes this by delegating the release of the query runner to the stream callback functions, ensuring that it's called when a connection is certainly established.

To test that streaming queries now work again, I had to add pg-query-stream to the dev dependencies.

The unit test I added fails as expected without this PR's changes, but passes with the changes.

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run lint passes with this change
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change
  • Documentation has been updated to reflect this change (N/A)
  • The new commits follow conventions explained in CONTRIBUTING.md

If the QueryRunner is not connected yet, release is a no-op
@alvisetrevisan
Copy link
Contributor Author

Tests are failing on CI, but I don't have access to the results. How can I see what goes wrong?

@nebkat
Copy link
Contributor

nebkat commented Dec 13, 2020

You should have access to the CI if you log in with your GitHub account, but otherwise you can run the tests locally with the help of Docker.

Here is the error anyway:

query builder > count
    ✓ Count query should of empty table should be 0
    ✓ Count query should count database values (79ms)
    ✓ Count query should handle ambiguous values (154ms)
    ✓ counting joined query should count database values (59ms)
    ✓ counting joined queries should handle ambiguous values (171ms)
    1) "after all" hook for "counting joined queries should handle ambiguous values"


  368 passing (3m)
  8 pending
  1 failing

  1) query builder > count
       "after all" hook for "counting joined queries should handle ambiguous values":
     Error: Timeout of 60000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/typeorm/build/compiled/test/functional/query-builder/count/query-builder-count.js)
      at listOnTimeout (internal/timers.js:549:17)
      at processTimers (internal/timers.js:492:7)

Revert no-op driver-wide early release, fix only for stream case.
Restrict stream unit test to drivers who implement the stream interface
Restrict unit test to drivers whose stream implementation works in master
@alvisetrevisan
Copy link
Contributor Author

Hi @nebkat I resolved the problems with the unit tests. This PR should be ready for review. I could not find in the CONTRIBUTING.md doc any more pointers on who I should add as reviewer, or whether anything else is needed from my side.

@starzou
Copy link

starzou commented Dec 23, 2020

Hi @alvisetrevisan So when do we call queryRunner.release(); ?

@alvisetrevisan
Copy link
Contributor Author

Hi @alvisetrevisan So when do we call queryRunner.release(); ?

This is done using the onEnd and onError callback (which ensures that the runner is released only after it's connectd) and is implemented already by the drivers supporting streaming queries.

@pleerock
Copy link
Member

Looks correct. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stream() bug from 0.2.25 to 0.2.26 with postgresql
4 participants