Dynamic semaphores implementation discussion #9382
robertkotcher
started this conversation in
Ideas
Replies: 2 comments
-
Hope you don't mind me cc'ing you @sarabala1979 since you had proposed looking into enhancing the semaphore feature. |
Beta Was this translation helpful? Give feedback.
0 replies
-
did u solve @robertkotcher ? i am interested in counting towards 2 different semaphores at once:
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
This discussion is a continuation of a few high-level conversations we've had in maintainers meetings, a smaller ad-hoc maintainer meeting, and the following issue: 8982.
Proposed feature and example
In these prior conversations, the requested feature has been referred to as "resource sharing". The tldr; is that there are situations in which Workflows are created on behalf of some set of end users, and one or more Templates in this Workflow use enumerable resources that must be shared equally amongst those end users. Examples (and my initial POC implementation) can be found in these slides. We want the controller to adjust how many of a template can run at any given time for each individual end user. Hopefully the following example makes this a bit more clear:
example
For a more concrete example, suppose some template T can have at most 10 instances running, and I have 2 workflows (let's call this particular workflow W) from user A and 1 workflow W from user B running. Each instance of W is currently trying to run 10 instances of T at the same time, so in total I have 30 instances of T trying to run at the same time. But I can only run 10 at any given instant. So at this point in time, we want to run 5 Ts for user A, and 5 Ts for user B.
Proposed changes to my approach: enhance semaphore feature
In the most recent maintainer meeting in which this was brought up this one we established that it'd be better to try and enhance the semaphore feature instead of introducing an entirely new set of concepts.
Right now I can use semaphores to configure "running at most N instances of template T at any given time", but since there is no concept of "submitted-on-behalf-of-user", there no way to adjust which instances of T can run now (and which must wait). So I can constrain the running instances of T but I can't control which of them are running.
Question
For those more familiar with the codebase than me, where would be a good place to start? I have implemented by design proposal (see slides above), but am not seeing immediately how this would naturally extend from the existing semaphore concept. Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions