Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relation id and afterAll hook performance fixes #8169

Merged
merged 5 commits into from Feb 16, 2022

Conversation

Lennard-Dietz
Copy link
Contributor

This PR aims to increase the performance of some very specific typeorm parts.
This includes the entity transformation after a query, when using the @RelationId decorator as well as broadcasting the afterAll event to all subscribers.
All it does is refactor some internal class logic for performance optimization. Therefore no new tests are needed, as the results of all the processes are still the same.

Motivation
When using typeorm, we saw that running queries which return > 500 entities are having a very bad performance and in our case, as we are using the @RelationId decorator a lot, we even saw a quadratic increase in runtime of the queries when increasing the number of entities.
Therefore we did a deep dive and found several performance optimizations. By far the biggest one is in the RawSqlResultsToEntityTransformer, as there was a quadratic calculation when using @RelationId. We were able to get this down to a linear runtime, which helped increase the performance of our queries a lot, around 3 times faster when returning 1000 entities.

It is related to the following issues, depending on the used entities, but does not fix them completely:
#2381
#3857

tl;dr

  • Replace quadratic runtime in RawSqlResultsToEntityTransformer with linear runtime
  • Use object instead of array for faster duplicate checks in RelationIdLoader
  • Calculate the fitting subscribers for the given entity only once and not for every result in Broadcaster

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run lint passes with this change
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change "N/A"
  • Documentation has been updated to reflect this change "N/A"
  • The new commits follow conventions explained in CONTRIBUTING.md

Replace a quadratic runtime with a linear in results transformer
Remove duplicate calculations of fitting subscribers for entity
Use object instead of array for faster access
@antoniusostermann
Copy link

This would solve a lot of our problems! Any plans on when to merge this?

@xdrago1
Copy link

xdrago1 commented Oct 12, 2021

Can someone give an update on this? would be great if it gets merged soon

Copy link
Contributor

@rluvaton rluvaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a maintainer, just ideas for improvement

src/subscriber/Broadcaster.ts Outdated Show resolved Hide resolved
src/subscriber/Broadcaster.ts Outdated Show resolved Hide resolved
@Lennard-Dietz
Copy link
Contributor Author

Thank you @rluvaton for your little review 👍

@antoniusostermann
Copy link

@pleerock - Since it's already reviewed and mergeable with master, any plans on when to merge this?

@pleerock pleerock merged commit 31f0b55 into typeorm:master Feb 16, 2022
@pleerock
Copy link
Member

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants