Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a new ThreadPoolWorker #1804

Open
ccrvlh opened this issue Feb 10, 2023 · 0 comments · May be fixed by #1803
Open

Create a new ThreadPoolWorker #1804

ccrvlh opened this issue Feb 10, 2023 · 0 comments · May be fixed by #1803
Labels

Comments

@ccrvlh
Copy link
Collaborator

ccrvlh commented Feb 10, 2023

This adds to the list of concurrency models we may want to implement (along side a WorkerPool, GeventWorker, AsyncWorker and MultiprocessWorker), also related to #45. Some work being done on #1803.

We replicated all tests that Worker went through using the ThreadpoolWorker, that means the basic Worker test suite, dependencies tests, callbacks tests and registry tests. Most of the tests pass, an overview on the issues so far (I haven't spend time looking into them):

Failed tests:

  • Timeout & Failed registry: a job that times out doesn't seem to be sent to the FailedJobRegistry
  • Dependency's Race Condition: we don't seem to be able to get the job.result when job's being executed
  • Main dependency test fails: jobs seems to be stuck at queued while we should be finished
  • Dependencies are met at execution time: we can't seem to get the job result

Other todo's:

  • Test the worker is not eager: doesn't take more jobs than the available idle threads (this should be working fine, but should be better tested)
  • Save the worker class and the pool size on the worker registry
  • Add the scheduler (not implemented)
  • Any other thread-safety issues we want to test?

Any help on this is very much appreciated!

@ccrvlh ccrvlh added the feature label Feb 10, 2023
@ccrvlh ccrvlh linked a pull request Feb 10, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant