Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concurrent versions of forEach/some/every/find? #6

Open
bakkot opened this issue Feb 8, 2023 · 3 comments
Open

concurrent versions of forEach/some/every/find? #6

bakkot opened this issue Feb 8, 2023 · 3 comments

Comments

@bakkot
Copy link
Collaborator

bakkot commented Feb 8, 2023

These aren't strictly necessary given a bufferAhead helper, because you can always replace .some(pred) with .map(pred).bufferAhead(3).some(x => x) (or similar). But that's kind of gross.

I definitely don't want to add a "concurrency" parameter to the iterator-producing helpers, since it's much easier to use when concurrency of an iterator chain is determined by how many times the consumer pulls from it. (Some more discussion of that in the second half of this comment.)

But these methods are the consumer, and might themselves be async and able to run their functions concurrently. Should we have concurrent versions of the consumers?

@benlesh
Copy link

benlesh commented Jun 8, 2023

RxJS (and other observable implementations) implement forEach to take a callback and return a Promise<void> when it's done.

I definitely don't want to add a "concurrency" parameter

I would avoid concurrency parameters in EVERYTHING if possible. RxJS exposes a "concurrency" parameter in our mergeMap operator and it's very rarely used or useful.

@laverdet
Copy link

Could bufferAhead pass along information in the derived iterable which is used to signal ahead that the iterable is concurrent? Something like { [Symbol.asyncConcurrency]: 3, next() { ... } }. Folding operations could use that signal to evaluate concurrently.

It definitely gives me the icks but given the strong hesitation against adding concurrency parameters it seems like one of few choices.

@bakkot
Copy link
Collaborator Author

bakkot commented Apr 11, 2024

bufferAhead already passes concurrency information to the iterator it's consuming simply by calling next multiple times concurrently. And passing the information to the iterator that's consuming bufferAhead isn't so useful: you wouldn't want iter.bufferAhead(N).forEach(foo) to call foo concurrently; that's something that the person calling forEach would need to opt in to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants