Default log file for nrunner based tests

[richtja]

In the #4804 issue @clebergnu asks for a default log file for nrunner based tests which will have a predefined path, so users will be able to find it more easily and will be able to work with it in their own scripts. In the legacy runner we have this ability because there is just one candidate for default log file the ~/avocado/job-results/latest/test-results/*/debug.log. In the nrunner the situation is more complicated, because different runners use different files for logs. For example, avocado-instrumented tests use debug.log, the exec-test use stdout and stderr and vt-runner uses all of them. Now the question is how the user find out which file is the default one.

Form discussion with @clebergnu, there are two possible solutions:

1. Every runner will decide which logs should be saved inside the default log file, and the other logs will be saved same as now. So the user will have the most important information inside the default log file and if he will need more logs there still be debug.log, stdout and stderr.
2. There will exist just one log file as default and all the logs will be saved there with a mark of their type at the beginning of each line. So user will be able to print stdout with something like ~/avocado/job-results/latest/test-results/*/log | grep stdout

With @clebergnu we incline to the second option, but I created this discussion to know your opinion about this topic, and I will be grateful for your comments, thanks.

[clebergnu]

@ana @beraldoleal @willianrampazzo any thoughts on this?

[beraldoleal]

In the #4804 issue @clebergnu asks for a default log file for nrunner based tests which will have a predefined path, so users will be able to find it more easily and will be able to work with it in their own scripts. In the legacy runner we have this ability because there is just one candidate for default log file the ~/avocado/job-results/latest/test-results/*/debug.log. In the nrunner the situation is more complicated, because different runners use different files for logs. For example, avocado-instrumented tests use debug.log, the exec-test use stdout and stderr and vt-runner uses all of them. Now the question is how the user find out which file is the default one.

Form discussion with @clebergnu, there are two possible solutions:

1. Every runner will decide which logs should be saved inside the default log file, and the other logs will be saved same as now. So the user will have the most important information inside the default log file and if he will need more logs there still be `debug.log`, `stdout` and `stderr`.

2. There will exist just one log file as default and all the logs will be saved there with a mark of their type at the beginning of each line. So user will be able to print stdout with something like `~/avocado/job-results/latest/test-results/*/log | grep stdout`

I'm ok with this second option, however IIUC, spiting the stdout/stderr but preserving the order will be tricky (but feasible). Probably we will have to use a timestamp with the pipes.

My two cents: rename log to debug.log or something more meaningful.

[willianrampazzo]

Having one log file by test, like option 2, seems reasonable for me. I don't think all the information from stdout and stderr should be inside it, but that is an implementation detail.

[pevogam]

2\. There will exist just one log file as default and all the logs will be saved there with a mark of their type at the beginning of each line. So user will be able to print stdout with something like `~/avocado/job-results/latest/test-results/*/log | grep stdout`

This sounds good, I just assume you mean ~/avocado/job-results/latest/test-results/*.log | grep stdout potentially with the debug.log there and no separate folder or anything of this sort. In short: preserving a fairly standard Avocado behavior when it comes to debugging tests.

[ana]

2. There will exist just one log file as default and all the logs will be saved there with a mark of their type at the beginning of each line. So user will be able to print stdout with something like ~/avocado/job-results/latest/test-results/*/log | grep stdout

I took at time for looking the behavior before replying, the second option seems better for me. With extra care so the log is easily greap-able.