Is it possible to use multiple threadpools to run the different types of scheduled tasks?

[softKarma]

Is it possible to have separate thread pools for separate type of tasks. For example I have a looot of one time tasks scheduled with a short validity period and recurring tasks that may be used to schedule long running tasks or trigger some job etc. I think having separate thread pools would make sure each have a certain number of dedicated threads. Does this make sense :-) ?

[kagkarlsson]

Yeah, that is a valid use-case and could be risky using the same table. There is a feature-request for task-priority that likely would solve this, but not sure when that will land. For now I have recommended using two separate tables.

[kagkarlsson]

Yeah, sort of. Risky in the sense that some executions might be delayed due to "congestion". It might then cause a recurring task to miss some of its execution-times in the schedule (say if it is delayed 10 mins and supposed to run every minute)

[dfjmax]

@kagkarlsson is there any example that shows how to have multiple tables/schedulers in the repo using the spring boot starter?

From what I've seen in the source code, a new Scheduler is created by the starter using an injected data source/list of "knownTasks".

If my thoughts are correct, for this to work, I should not use the spring boot starter and manually create 2 schedulers, that use (or not) the same dataSource, different tables and have different lists of "knownTasks".

This way I would be able to have individual schedulers using different thread pools.

[meggarr]

Happened to see this! It's a good idea to have two scheduler/tables in one application - because I see there are cases the scheduling is skewed when instances of one type of task are too many, while another important task has only one instance. In this case the one-instance task is scheduled slowly not even after 100sec of the target schedule time.

One straight question is can I set the reschedule time to current minus 2hours to get the priority ?

[kagkarlsson]

One straight question is can I set the reschedule time to current minus 2hours to get the priority ?

Currently that would work yeah, polling results are ordered by ascending execution-time. (so only works for things scheduled to run "now")

[kagkarlsson]

If my thoughts are correct, for this to work, I should not use the spring boot starter and manually create 2 schedulers, that use (or not) the same dataSource, different tables and have different lists of "knownTasks".

Sounds about right! (boot-starter does not work with multiple schedulers as observed)