Replies: 5 comments
-
So I had another idea and that was to do some logicn in my job. The idea was this
Turns out that its not possible to reschedule the current execution and so below you see it commented out. The best I can do is just return the logic early and avoid doing any logic. This is not great either because a) the job has no chance of being picked up any sooner than its next scheduled run and b) the same thing could happen next time. I think the only idea I have left is to use a CustomTask, where I skip the main logic and then in the
then another node should get to pick it up shortly
|
Beta Was this translation helpful? Give feedback.
-
which part of db-scheduler could I (easily) extend to customise that would allow me to pre-empt the update of picked_by? Maybe there is some part of the lifecycle logic that I could query the database and depending what I find , just return/bail out early thus preventing the picked_by getting set |
Beta Was this translation helpful? Give feedback.
-
seems to me the best bet would be to fork the code and add the logic to FetchCandidates.PickDue.call() |
Beta Was this translation helpful? Give feedback.
-
Even distribution is, as you have discovered, not a problem db-scheduler aim to solve. And tbh I am not sure how I would solve it. |
Beta Was this translation helpful? Give feedback.
-
Hi all,
As a test - I have added 4 different recurring tasks to a single scheduler and told them all to run at the same time periodically.
I have also given the scheduler 4 threads (.threads(4)).
Now if I run this code on 4 separate nodes - backed by the same database of course, then I have observed that sometimes the 4 different jobs can be running at the same time on the same node.. while all the other nodes are idle.
How can I get db-scheduler to try to distribute the jobs across the cluster better?
The way I see it, I should aim for fewer schedulers, i.e have one scheduler instance manage many jobs (which I am doing in my test) while at the same time reducing the thread pool of that scheduler- that way it can only do a max number of jobs at once giving other nodes a chance to pick a task
or has anyone any better/simpler ideas ?
thank you
Beta Was this translation helpful? Give feedback.
All reactions