Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Workers aware of schedule concurrency limits #13208

Open
abrookins opened this issue May 2, 2024 · 0 comments
Open

Make Workers aware of schedule concurrency limits #13208

abrookins opened this issue May 2, 2024 · 0 comments

Comments

@abrookins
Copy link
Collaborator

abrookins commented May 2, 2024

  • Workers are not currently aware of which schedule generated a flow run. After we start tracking this with the CreatedBy column, the worker can find the schedule using that field.
  • When a deployment schedule has max_active_runs, the worker should use a global concurrency limit whose slots are equal to max_active_runs
    • This will require the client-side interface to global concurrency limits to support a timeout option
  • The worker could create this limit but will need to fail gracefully if another worker tries to create the same limit at the same time (i.e. there should be a unique constraint)
  • We could also create the limit when editing or creating the deployment schedule, in which case the worker should fail gracefully if no limit exists (it shouldn't run the run and should probably leave it scheduled)
  • If the worker can't acquire a slot within a timeout period, it will mark the run as Cancelled with the name Skipped
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant