Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce higher pipeline parallelism when the number of reading streams is low #63673

Open
nickitat opened this issue May 12, 2024 · 0 comments
Open
Assignees

Comments

@nickitat
Copy link
Member

Noticeable when there are not so many rows with heavy aggregate function states. Because we create as many AggregatingTransforms as there are input reading streams we deteriorate performance (more granular parallelisation would be in fact useful since processing of each row is computationally nontrivial).

@nickitat nickitat self-assigned this May 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant