chore(log): Restructure and cleanup logging code

[srebhan]

Summary

This PR moves all logging code into the logger package and simplifies the code. It also exposes all required functions in the telegraf.Logger interface to provide an abstraction for changing the underlying logging implementation. This is in preparation to provide more fine-grained logging as requested in #6584 or adding structured logging as requested in #8815.

Checklist

• No AI generated code was used in this PR

Related issues

resolves #10212

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[powersj]

Took an initial pass at this today:

• Verified the --debug, --quiet, --once options print the same text as today
• Verified startup negative cases, e.g. config file does not exist
• inputs.kafka logging enabled/disabled
• outputs.kafka logging enabled/disabled
• Tested deprecation notice using inputs.net and ignore_protocol_stats config option
• inputs.exec with debug enabled and disabled
• Introduce a panic call in inputs.exec ensured the report the panic message still printed

TODO

• goshim test
• windows service

[powersj]

FYI I accidently hit submit on the previous comment too soon, but I have updated it.

For the goshim test, I grabed https://github.com/ssoroka/rand and added some logging:

diff --git plugins/inputs/rand/rand.go plugins/inputs/rand/rand.go
index b5f6847..b3cf571 100644
--- plugins/inputs/rand/rand.go
+++ plugins/inputs/rand/rand.go
@@ -54,6 +54,10 @@ func (r *RandomNumberGenerator) Description() string {
 }
 
 func (r *RandomNumberGenerator) Gather(a telegraf.Accumulator) error {
+       r.Log.Error("error message")
+       r.Log.Warn("warn message")
+       r.Log.Info("info message")
+       r.Log.Debug("debug message")
        r.sendMetric(a)
        return nil

Build the plugin and used this config:

[agent]
  debug = true
  omit_hostname = true

[[inputs.execd]]
  command = ["/home/powersj/rand/rand", "--config", "/home/powersj/rand/plugin.conf"]
  signal = "none"

[[outputs.file]]

Which produced:

2024-04-26T16:30:17Z I! [inputs.execd] Starting process: /home/powersj/rand/rand [--config /home/powersj/rand/plugin.conf]
2024-04-26T16:30:18Z E! [inputs.execd] stderr: "2024/04/26 10:30:18 E! error message"
2024-04-26T16:30:18Z E! [inputs.execd] stderr: "2024/04/26 10:30:18 W! warn message"
2024-04-26T16:30:18Z E! [inputs.execd] stderr: "2024/04/26 10:30:18 I! info message"
2024-04-26T16:30:18Z E! [inputs.execd] stderr: "2024/04/26 10:30:18 D! debug message"
...
random value=7i 1714149026350162901
2024-04-26T16:30:27Z D! [outputs.file]  Wrote batch of 9 metrics in 64.99µs
2024-04-26T16:30:27Z D! [outputs.file]  Buffer fullness: 0 / 10000 metrics

This is the same as what I see from master currently.

[powersj]

Built and installed it as a service and still looks good.

Thanks!

[srebhan]

@powersj can you confirm that the last log line is now present in the eventlog?

[powersj]

@powersj can you confirm that the last log line is now present in the eventlog?

I only saw the first message about loading the config file.

[srebhan]

@DStrand1 I remove the RFFR label for now until @powersj had a chance to test the "last line is in eventlog" thing... Will let you know once this is good to finally go.

[powersj]

Here is my updated config:

[agent]
    debug = true
    logtarget = "eventlog"
    
[[inputs.cpu]]
[[outputs.file]]

Then in an admin powershell:

> .\telegraf.exe --service install --config .\config.toml
> net stat telegraf
The Telegraf Data Collector Service service is starting.
The Telegraf Data Collector Service service was started successfully.

This is what I see in the Event Viewer:

I get the same with your branch or master.

Executable By Hand

If I run the executable by hand with that same config, now all the messages go to the event log:

And this is the final message:

Invalid Config

If I do the same with an invalid config, nothing new shows up in the event viewer and I get this message on the CLI:

PS C:\Users\Josh Powers\telegraf-srebhan> .\telegraf.exe --config .\config.toml
2024-05-08T20:47:04Z I! Loading config: .\config.toml
2024-05-08T20:47:04Z E! error loading config file .\config.toml: error parsing outpts, undefined but requested input: outpts

We never even get to the event viewer.

Error in output

If I update the config like that in #14144 with:

[agent]
    debug = true
    logtarget = "eventlog"
    
[[inputs.cpu]]
[[outputs.file]]
[[inputs.nvidia_smi]]
  bin_path = "C:\\Windows\\System32\\nvidia-smi.exe"

The last message is:

As opposed to the expected:

2024-05-08T20:50:08Z E! [telegraf] Error running agent: could not initialize input inputs.nvidia_smi: nvidia-smi not found in "C:\\Windows\\System32\\nvidia-smi.exe" and not in PATH; please make sure nvidia-smi is installed and/or is in PATH

fwiw I don't see the following debug message in the event viewer either:

2024-05-08T20:50:08Z D! [agent] Initializing plugins

Thoughts?

[srebhan]

@powersj I will remove the reference to #14144 then and need to investigate the issue further. Are you still good with the PR otherwise?

[powersj]

Yes!

[DStrand1]

Looks great! Excited to see how this turns out