Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON stream progress writer #11478

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

felixfontein
Copy link
Contributor

What I did
I added a JSON stream progress writer. This allows to write machine readable output to stderr, which makes it a lot easier to use Docker Compose from other tools, like Ansible.

Related issue
Fixes #10872.

(not mandatory) A picture of a cute animal, if possible in relation to what you did
A cute cat with rather sharp claws, watch out when petting her

@felixfontein
Copy link
Contributor Author

I still need to run make docs and commit the changes. Unfortunately it never comes remotely near to completion with the not so great internet connection I have on this train, so that has to wait until later...

@felixfontein
Copy link
Contributor Author

Docs are updated, CI passes (checked in felixfontein#1), but while testing this (now that I was actually able to build this) I noticed that Docker Compose apparently expects Docker Buildx and Docker CLI itself to also have a JSON progress writer. I'm getting invalid progress mode json errors when it tries to pull images, and building also doesn't seem to work.

@felixfontein felixfontein marked this pull request as draft February 8, 2024 13:35
@felixfontein
Copy link
Contributor Author

I eventually noticed that in case of errors, the error is always printed regularly and not via the progress API. I added code in d3b85f9 that in case --progress json it also formats the error as JSON.

@felixfontein felixfontein marked this pull request as ready for review February 8, 2024 20:42
@felixfontein
Copy link
Contributor Author

This is now ready for review.

@felixfontein
Copy link
Contributor Author

It probably makes sense to set logging to JSON (logrus.SetFormatter(&logrus.JSONFormatter{}), see https://github.com/sirupsen/logrus/blob/master/README.md) to avoid random non-JSON logging messages to show up on stderr.

@@ -67,10 +67,15 @@ func (opts buildOptions) toAPIBuildOptions(services []string) (api.BuildOptions,
builderName = os.Getenv("BUILDX_BUILDER")
}

// The build backends do not have a "json" progress; use "plain" instead
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use rawjson so buildkit will render raw json solver events and you get a clean json stream for all events

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created moby/buildkit#4769 since I think it would be better if the rawjson output would write one JSON object per line, instead of using multiple lines to show a more human-readable JSON record (that's not really that human readable anyway).

Signed-off-by: Felix Fontein <felix@fontein.de>
Signed-off-by: Felix Fontein <felix@fontein.de>
Signed-off-by: Felix Fontein <felix@fontein.de>
Signed-off-by: Felix Fontein <felix@fontein.de>
Signed-off-by: Felix Fontein <felix@fontein.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

docker-compose - dry-run - json output
2 participants