[BUG] Recurring job not invoked and dashboard not responding

[igr]

JobRunr Version

6.1.4

JDK Version

17

Your SQL / NoSQL database

Postgres 15

What happened?

Recuring job has not started and is not listed.

Using SpringBoot 3.0.6 (latest), Kotlin 1.8.21 (latest).

I have one recurring job (that used to work). There is a record in the table jobrunr_recurring_job:

This is the json:

{
  "version": 0,
  "jobSignature": "xxxx.service.dispatcher.TimeDispatcherJob.run(xxxx.service.dispatcher.TimeDispatcherJobRequest)",
  "jobName": "TimeDispatcherJob",
  "amountOfRetries": 2,
  "labels": [
    "cron"
  ],
  "jobDetails": {
    "className": "xxxx.service.dispatcher.TimeDispatcherJob",
    "staticFieldName": null,
    "methodName": "run",
    "jobParameters": [
      {
        "className": "xxxx.service.dispatcher.TimeDispatcherJobRequest",
        "actualClassName": "xxxx.service.dispatcher.TimeDispatcherJobRequest",
        "object": {}
      }
    ],
    "cacheable": true
  },
  "id": "xxxx.service.dispatcher.TimeDispatcherJob.run(xxxx.service.dispatcher.TimeDispatcherJobRequest)",
  "scheduleExpression": "*/15 * * * * *",
  "zoneId": "Europe/Belgrade",
  "createdAt": "2023-05-11T01:34:38.567623Z"
}

application.yaml:

org:
  jobrunr:
    background-job-server:
      enabled: true
      worker-count: 1
    job-scheduler:
      enabled: true
    dashboard:
      enabled: true

However, the job is not picking up - the JobRunr does not execute it.

When I go to the JobRunnr dashboard, I can see there is 1 recurring job, but when I click on that page, the JobRunr API never returns

How to reproduce?

jobRequestScheduler.createRecurrently(
            aRecurringJob()
            .withJobRequest(TimeDispatcherJobRequest())
            .withName(name)
            .withLabels("cron")
            .withCron(Cron.every15seconds())
        )

class TimeDispatcherJobRequest : JobRequest {
    override fun getJobRequestHandler() = TimeDispatcherJob::class.java
}

@Component
class TimeDispatcherJob(
    val jobRequestScheduler: JobRequestScheduler,
) : JobRequestHandler<TimeDispatcherJobRequest> {
    override fun run(jobRequest: TimeDispatcherJobRequest) {
    }
}

Relevant log output

2023-05-11T03:45:51.564+02:00  INFO 83421 --- [  restartedMain] j.s.t.ScheduledThreadPoolJobRunrExecutor : ThreadManager of type 'ScheduledThreadPool' started
2023-05-11T03:45:51.578+02:00  INFO 83421 --- [  restartedMain] o.j.dashboard.JobRunrDashboardWebServer  : JobRunr Dashboard using PostgresStorageProvider started at http://localhost:8000/dashboard
2023-05-11T03:45:51.578+02:00  INFO 83421 --- [pool-3-thread-1] org.jobrunr.server.BackgroundJobServer   : JobRunr BackgroundJobServer (aae00f74-d404-4fe1-a8f5-5bc3ee81bc8c) using PostgresStorageProvider and 1 BackgroundJobPerformers started successfully
2023-05-11T03:45:51.579+02:00  INFO 83421 --- [pool-3-thread-1] org.jobrunr.server.ServerZooKeeper       : Server aae00f74-d404-4fe1-a8f5-5bc3ee81bc8c is master (this BackgroundJobServer)
2023-05-11T03:45:51.613+02:00  INFO 83421 --- [pool-4-thread-1] o.j.server.tasks.MigrateFromV5toV6Task   : Start migration of scheduled jobs from v5 to v6
2023-05-11T03:45:51.616+02:00  INFO 83421 --- [pool-4-thread-1] o.j.server.tasks.MigrateFromV5toV6Task   : Found 0 scheduled jobs to migrate.
2023-05-11T03:45:51.616+02:00  INFO 83421 --- [pool-4-thread-1] o.j.server.tasks.MigrateFromV5toV6Task   : Finished migration of scheduled jobs from v5 to v6

More info

The same happens to any job I add to JobRunr:

        jobScheduler.create(
            aJob().withDetails { println("!!!!") }
        )

Now the "Jobs" page of the dashboard can not be retrieved. There is a relevant record in the table.

It looks like JobRunr didn't start somehow?

[rdehuyss]

Can you create a repo that reproduces this?

[rdehuyss]

Is this on windows?

[igr]

No, this happens on MacOS (latest) and whatever unix is on behind heroku (aws)

[rdehuyss]

Can you create a simple repo that reproduces this?

[igr]

I will try. First, I figured that things were working a few commits ago. Trying to debug the issue.

[igr]

@rdehuyss I have isolated the issue.

It is with the Jackson. When I update to version 2.15.0 (the latest), it is not working (as explained above). Jackson version 2.14.3 (latest 2.14.x) works.

I also use jackson-module-kogera with version 2.14.2-alpha8 which is compatible with 2.15.0, and the rest of my code is working, so it does not seem to be relevant.

My object mapper configuration:

 builder
                .serializationInclusion(JsonInclude.Include.NON_NULL)
                .featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
                .modules(JavaTimeModule(), kotlinModule())

[rdehuyss]

Hmm, all my test with Jackson 2.15.0 pass (part of v6.2.0 branch).

[rdehuyss]

I just ran 100.000 jobs in less than a minute with Jackson 2.15.

[igr]

Ok, let me see what is wrong here on my side; it must be some specific combination/configuration I am using.

Need to work now, but I will let you know asap when I find something (or if I create minimal reproducible project). My gestimate is that something is failing when json is deserialized from database, but, again, this is just a wild guess.

[igr]

Here: https://github.com/jobrunr/jobrunr/blob/master/core/src/main/java/org/jobrunr/jobs/mappers/JobMapper.java#L28

The following line:

jsonMapper.deserialize(serializedJobAsString, RecurringJob.class);

throws java.lang.ClassNotFoundException: com.fasterxml.jackson.core.exc.StreamConstraintsException.

Still looking to resolve this; but could the Dashboard API return 500 for the exception?

[igr]

@rdehuyss I fixed the dashboard by providing the explicit dependency:

		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
		</dependency>

(that fixed the exception mentioned in the previous post). However, the job is still not picked up for the execution.

[rdehuyss]

It looks to me that you have Jackson multiple times on the class path. See FasterXML/jackson-core#948 where the exact same issue is discussed.

As I can confirm that JobRunr works fine with latest Jackson version, I'm quite sure this is in your end and I don't want to put more of my limited free time in this

I'll close this issue for now. Feel free to reopen if you can setup a reproducible repo.

[rdehuyss]

P.s.: I hope you understand but I spend too much time on the OSS version. Again, if you can provide me with a reproducible scenario, I don't mind to have a look.

[igr]

Sure, I do understand, no worries.

I will try to debug it on my own; at least I am not blocked, and I have an idea where the issue might be. Thanx!

[rdehuyss]

Were you able to solve it?

[igr]

I just tried today :)

There is a new version of SpingBoot, 3.1, so I have updated all dependencies, including the Jackson to 2.15.2 and I don't see any issues so far. I will examine it a bit more tomorrow, but so far, it looks ok