Certain combinations of check periods and check intervals prevent service status updates.

[yoshi314]

Describe the bug

To Reproduce

I had a service that would check ssl cert validity. Mistakenly i configured it to have

check_interval = 24h
check_period = "workhours"

which meant it would execute 7-21 on business days.

This check was set up to go critical if certificate was 7 days away from expiration.

For a while this worked fine. but i've noticed that my check was stuck in warning state, saying i have 23 days left until Feb 2 - which (as of yesterday) was obviously incorrect. And the certificate was expiring in 2 days. So i had a stale service, except it seemed that it was being checked on schedule every 24 hours.

Upon inspection the check scheduled itself to run at ~4am, and apparently stopped updating service state for a few weeks since it was outside of its timeperiod. Manual reschedule of the check fixed the issue temporarily.

It appeared as if the check was running on schedule, just not updating service state.

I had to readjust the check to have notification timeperiod assigned instead.

Expected behavior

I assumed that icinga2 would try to schedule the check within the timeperiod window - especially if it was running late. Or would update the service state on the beginning of check_timeperiod.

I've seen this on 2.14.1 and 2.14.2 on Debian 10 (packages from icinga repository).

Maybe some kind of warning at config reload would suffice if check_interval may cause the service to "fall out" of its check_period window.

[xeiss]

I also have found this behavior in my environment (2.14.2 on Debian 12). With my config history I found that this problem exists for me since year 2019. As a workaround I don’t use check_interval = 24h + check_period = "workhours" since then.
The checks will with time, when it is planed outside check_period timeperiod and then will never go back to work until manual reschedule.

I also have a productive example: HTTPS Cert Check on a printer which goes in deep sleep in the night and check will fail, I have to manual reschedule this to "daytimes" and it will work again for months until the checktime will slide back to a nighttime, but with a working check_period = "workhours" I could mitigate this problem completely.

Possible Solution: The scheduler should have the check_period in mind and should plan only checks when they are inside the next timeperiod, also when the 24h check will be planned for 28h later. Or 70h later if it goes over weekend.

[Al2Klimov]

The scheduler should have the check_period in mind and should plan only checks when they are inside the next timeperiod

@julianbrost This would also solve our little overdue problem.

[Al2Klimov]

Manual reschedule of the check fixed the issue temporarily.

This is indeed a legit workaround. You could also make the check interval just 1h lower than your check window, 13h I guess.