zjobs: high cpu usage

[Jack-Ji]

I'm trying to parallel frontend of my software rasterizer using zjobs. The JobQueue with default configuration occupied at least 80% cpu, even when I'm not scheduling any jobs.

OS: windows 11 x64
zig: latest master

[michal-z]

@garettbass is this known issue?

[hazeycode]

zjobs has threads busy wait using sleep. You can configure the sleep time for your JobQueue by changing idle_sleep_ns form the default of 10 ns. Note this is tiny! And is probably below the accuracy error of the OS's scheduler. But by making it bigger you should see less CPU usage. By doing this you are trading "wake up" performance i.e. how long it can take a job to get picked up by a thread. You basically get to pick between wasting energy or increasing latency of jobs starting.

You can get more efficiency and lower latency by not using sleep and instead suspending and signalling threads but it's kinda complicated to do cross-platform.
Zig's WIP (discontinued?) standard event loop source is a good read: https://github.com/ziglang/zig/blob/master/lib/std/event/loop.zig
cyptocode wrote a nice blog post, that show a high level usage (with async/await but the guts are the same): https://dev.to/stein/async-cpu-bound-workers-in-zig-24b0

[Jack-Ji]

I've thought of adjusting sleeping interval actually. However, I also think the default configuration should works without obviously problem across platforms.

[hazeycode]

I've thought of adjusting sleeping interval actually. However, I also think the default configuration should works without obviously problem across platforms.

10 ns is really tiny! I'd guess that means basically that the thread is suspended and immediately woken by the OS scheduler.

[Jack-Ji]

I've thought of adjusting sleeping interval actually. However, I also think the default configuration should works without obviously problem across platforms.

10 ns is really tiny! I'd guess that means basically that the thread is suspended and immediately woken by the OS scheduler.

Yeah, I agree with you. May be we can change it to some more reasonable value? Anyway, I'll try it later to see what happens.

[hazeycode]

The problem is it's difficult to pick a number that is going to work well everywhere. You can provide a different value per target OS but you will also probably want to tune it per use case in your program.

If efficiency is a concern of the default behavior then 10 ns probably isn't a reasonable value for any scheduler (iirc). And it could be argued that there's little value in having a default sleep time at all i.e. the user always makes the decision with regard to the their use case (importance? cpu bound? granularity...)

[garettbass]

This is a known issue. I have not yet tried to address it, but I have some ideas how I might.

Please consider that I have written the simplest possible implementation I could implement reliably. Pull requests from devs more experienced with this problem space are quite welcome.

[garettbass]

Sorry, I failed to address the original request about idle CPU usage. I do have some changes in progress to limit idle CPU usage using a condition variable when a thread attempts to dequeue a job when none are available. The thread will be suspended by the condition variable until a job is placed in the queue, then the condition variable will be signaled to wake the first waiting thread.

I wrote this code about a month ago, and some test cases for it, but I have yet to integrate these changes into the JobQueue. I hope to get some free time to work on it again soon.

[michal-z]

@garettbass you already did great job with job system :) All the improvements are of course awesome and very welcome but don't worry if you don't have time for them. We will do it.

I will be working on a software fractal renderer as a showcase for zjobs and zmath - this will give me an opportunity to delve into zjobs sources.

[Avokadoen]

This issue relates to #130