Add support for logging lines without target name prefix

[fheinecke]

What existing functionality needs improvement?

Currently when executing a target in an Earthfile, every line written to stdout by a RUN directive is prefixed by the target name. For example, running this Earthfile:

do-some-stuff:
    LOCALLY
    RUN echo "line 1"
    RUN echo "line 2"

Would produce:

$ earthly +do-some-stuff
+do-some-stuff | line 1
+do-some-stuff | line 2

Certain CI systems like Github actions support a form of "rich output" where lines starting with specific prefixes change how they are displayed in the web interface. For example, a line such as ::error Some error message would output error: Some error message in red, and log the message to the workflow run summary page. However, this functionality does not work when Earthly prefixes each line with the + <target name> | .

Expected Behavior

There should be an option to omit the "target prefix" portion of output lines. Ideally this would be configurable on a per-target basis, so that other targets would remain prefixed with the target name.

There are certain "output modifiers" that would still not work properly with this in all cases due to parallelism/output stream interleaving. For example, this Earthfile

build-something:
    ARG TARGETARCH
    LOCALLY
    RUN echo "::group:: build something group for $TARGETARCH"
    RUN echo "The target arch is $TARGETARCH"
    RUN echo "::endgroup::"

build-all:
    BUILD --platform=os/arch1 --platform=os/arch2 +build-something

would run both build-something in parallel (once for each arch in the build-all target). The output could look something like this:

::group:: build something group for arch1
The target arch is arch1
::group:: build something group for arch2
The target arch is arch1
::endgroup::
::endgroup::

Which would not be grouped properly. However, this would be fine for any case where multiple targets are not ran in parallel. This issue could potentially be solved by having an option to group output lines, at the cost of not having run log lines until the target completes. Personally, I see value in having this feature regardless of whether or not the parallel/log grouping issue is solved.

[vladaionescu]

Note that Earthly uses stderr for all the build output. It might be useful to have the ability to tell Earthly to use stdout for some things. This would also allow you to pipe the stdout to another tool if needed. Plus, I believe that GitHub Actions only interprets these workflow commands if they're on stdout.

Possible syntax idea:

    RUN --raw-stdout echo "::group:: build something group for $TARGETARCH"
    RUN --raw-stdout echo "The target arch is $TARGETARCH"
    RUN --raw-stdout echo "::endgroup::"

And it would be up to the user to manage parallelism via WAIT, if they have multiple things going on in parallel.

[fheinecke]

That'd be fantastic.

I thought that GHA supported parsing stderr as well, but I may be mistaken. I checked the docs and nothing explicitly states what output streams are actually supported. If it is helpful then I can test this in one of our repos.

[idelvall]

Supporting both cases makes sense to me.
But, I think the need for keeping the logs is contextual rather than intrinsic, so setting this at earthly command makes more sense to me.

[alexcb]

relates to #2268 which requires outputting logs like:

::error file=app.js,line=1::Missing semicolon

without any prefix, so github can pickup annotations

[adamgordonbell]

No target ( --raw-output ) and sending output to stdout feel like two separate features that are both needed for this specific GitHubActions use case.

#4014 could solve the stdout portion of this and this ticket can solve the need for RUN specific --raw-output commands.

[adamgordonbell]

Complete in #4023