Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a WithKeyedResult option to the result pool #125

Open
zhongdai opened this issue Dec 8, 2023 · 1 comment
Open

Add a WithKeyedResult option to the result pool #125

zhongdai opened this issue Dec 8, 2023 · 1 comment

Comments

@zhongdai
Copy link
Contributor

zhongdai commented Dec 8, 2023

I have been using the result pool to fan-out some I/O operations, eg. fetch data from database. The code is something like,

p := pool.NewWithResults[[]record]().WithErrors().WithContext(ctx)

// queries is []string type
for _, query := range queries {
  p.Go(
    func(context.Context) ([]record, error) {
      return databaseCall(ctx, query)
    })
}

result, err := p.Wait()

In this example, the record is a struct to map to a database record, and each databaseCall() would return N records as a []record type.

If any of the query failed to run (eg. timeout), the result (typed [][]record) contains only the succeed returned []record. So the len(queries) != len(result).

In the case, I need to re-try the failed queries, I don't know which ones are failed. So if we can have a option to return a map[string][]record, we can check if which batch(es) are failing.

@camdencheek
Copy link
Member

Hi @zhongdai! Thanks for the request. I have a draft PR open to make the set of results from a ResultPool match the order of the tasks were submitted in. In combination with p.CollectErrored(), that would mean that result[i] always corresponds to queries[i]. Does that solve your use case?

Side note: now that result order is deterministic, I'm considering making CollectErrored() default for the 1.0 release. However, that's a breaking change, so I didn't implement that in the original PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants