High CPU Utilization

[jgentner2]

Thank you for providing such an awesome logging library! I'm currently facing an issue where my backend worker thread appears to be consuming an entire CPU. In my attempt to understand BackendWorker.h, I'm not able to identify any blocking code in the _main_loop() function when there are no events to process. It does some minor processing/bookkeeping and calls yield() before iterating on the loop again. I noticed a comment in the code mentioning sleeping, but it doesn't seem to translate into actual sleep time. As a hack to further understand this issue, I added a forced sleep of 25ms, which resulted in the processor utilization dropping to nearly zero by my application. Can you help me understand what I might be doing wrong or not understanding? Thank you!

[odygrd]

hey and thanks for checking out the library. Since the library is designed mainly to offer low latency the logging thread is constantly spinning even when there are no events to log.

The thread will spin and check if there are any messages to log if not it will do some housekeeping work.
You can configure it to sleep if you want but I won't recommend you to set a sleep higher than 1 us. The reason for that is that it needs to wake up to pick up the log messages, because if it doesn't and the log messages are too many they can cause a queue allocation that will impact the hot threads.
Also you can pin the logging thread to a non critical core https://github.com/odygrd/quill/blob/master/quill/include/quill/Config.h#LL138C12-L138C40

In v2.7.0 sleep was replaced by yield but sleep is now back in master.

So you can use something like

  quill::Config cfg;
  cfg.backend_thread_sleep_duration = std::chrono::nanoseconds{700};
  quill::configure(cfg);
  quill::start();

[niclar]

"In v2.7.0 sleep was replaced by yield but sleep is now back in master" -that "yield" change should have warranted a clearer release note/warning imho... ("Replace backend_thread_sleep_duration with backend_thread_yield in Config.h")

[odygrd]

It is also now possible to use a high sleep duration for the backend logging thread and then wake it up on demand to process the logs from any thread.

#280

See example
https://github.com/odygrd/quill/blob/master/examples/example_wake_up_logging_thread.cpp

The backend logging thread still spins / busy waits and when it goes into sleep it can be waken up by any thread if you call quill::wake_up_logging_thread(). This will have some latency cost on the thread that it is calling but it comes as an alternative when you want to avoid backend logging thread consuming some cpu cycles spinning