dockerfile/1.4.1

Usage

# syntax=docker.io/docker/dockerfile:1.4.1

Notable changes

• Fix named context resolution for cross-compilation cases from input when input is built for a different platform #2742

v0.10.0

Welcome to the v0.10.0 release of buildkit!

Please try out the release binaries and report any issues at
https://github.com/moby/buildkit/issues.

Notable changes:

• Builtin Dockerfile frontend has been updated to v1.4.0 https://github.com/moby/buildkit/releases/tag/dockerfile%2F1.4.0

• New LLB operation MergeOp allows efficiently chaining groups of layers together without the need to access the individual files. This allows to build layers separately and merge them together later without making them depend on each other. MergeOp can work with remote references, for example, it can be used to rebase an image on top of a newer base image without ever pulling or pushing any layers. #2335 #2614

• New LLB operation DiffOp allows computing a difference between two points in LLB graph containing the files that have been added and whiteouts for files that have been removed. When DiffOp inputs are based on the same layer chain DiffOp can work directly with remote layer references and doesn't need to pull layers to access individual files. Files of the DiffOp result can be accessed directly or used as input to a MergeOp. #2517 #2434 #2563

• New build information structures are generated with build metadata that allows you to see all the sources (images, git repositories) that were used by the build with their exact versions and also the configuration that was passed to the build. This information can also be embedded into the image configuration if one is generated. Build sources are added to the image config by default. The build configuration is not currently embedded by default to avoid credential leaks in poorly written Dockerfiles but the intention is to enable it in the future. #2311 #2476 #2654 #2657 #2645 #2684

• Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache import #2651

• When possible, blobs are now created with native OverlayFS differ with increased performance. This Differ can directly use files in OverlayFS upper directory instead of scanning for differences between snapshots. #2181 #2491 #2480 #2388 #2390 #2662

• Frontends can now send warning messages that are shown on the progressbar. Warnings can be associated with specific LLB vertex and contain additional information like URL to documentation or location in original source code. #2482 #2498

• Layer blobs can now be exported in Zstd compression format with -o compression=zstd. Zstd provides smaller files and faster decompression than gzip. #2344

• Layers can now be exported with eStargz compression type -o compression=estargz #2246 #2603 #2352 #2674

• A compression level can be set with -o compression-level=N to configure the compressor when new layer blobs are created. This can be used to create smaller blobs by spending more time on the compression step. #2591

• Remote cache inlined in image configuration now supports arbitrary configuration of image layers as cache sources #2501

• Enable eStargz-based lazy pulling on registry cache importer #2648

• Support exporting non-distributable blob descriptors. -o prefer-nondist-layers=true exports layer with a non-distributable mediatype and external URL if such URL was provided when blob was pulled from the remote registry. Descriptors with non-distributable mediatypes are skipped on push. #2561

• Build metadata now provides access to OCI descriptor of the result if one was generated. Previously only digest of the descriptor could be accessed. #2610

• Builder now understands AMD64 Microarchitecture levels, e.g. linux/amd64/v2, linux/amd64/v4. The default variant remains v1 and is normalized to linux/amd64. These variants allow creating more optimized versions of your images that run when the container is running on a more modern CPU. #2588

• LLB now supports progress groups for grouping multiple steps together so they appear as the same row in build progress #2513

• LLB ExecOp now supports build secrets that are exposed as environment variables #2579

• Interactive container API now supports sending signals to processes from the client #2590

• Logs now use a rolling buffer to show the last logs for a process that ended with an error even if regular logs have been clipped because they have reached the max-logs limit. #2577

• Remote cache exporting now support setting the same compression options available to image outputs #2685

• Frontends can now access their own definition and call additional processes on their own image #2443

• Tmpfs mounts can now set a size limit #2411 #2422

• Custom Cgroup parent can now be set the LLB ExecOp #2430

• Add support for Ulimit resource limits in LLB ExecOp #2398

• Add extra hosts supports to Gateway Exec API #2294

• Fix security mode config being dropped on Gateway Exec start #2290

• Git source can now correctly clone annotated tag references #2570

• OpenTelemetry tracing propagation environment variables have been updated to TRACEPARENT and TRACESTATE to match the changes in upstream. Old variables are deprecated and will be removed in the next release. #2572

• Make sure supplementary groups are loaded for the default user configuration #2428

• Allow exporting inline cache when blobs exist in multiple compressions #2405

• Github cache backend retry logic on hitting rate limits has been improved #2506

• Color schema on TTY progressbar has been enhanced on Windows for better readability #2368

• Build status stream now supports ProgressGroup object to group multiple LLB steps into a single progress item #2668

• Fixes to progress stream handling of multiple "complete" events during cache import #2675 #2641

• Fix possible out of order indexes in plain progress mode #2688

• Extra progress step has been added to the step where blobs are prepared for inline cache #2658

• Allow insecure security mode to work on environments where all capabilities are not available #2394

• Use standard user umask for Git processes #2356

• Fix tracing indicators showing up in logs even when tracing is not enabled #2351

• Handling of doublestar (**) pattern has been improved on transporting local sources. #2319 #2486

• QEMU embedded emulators have been updated to v6.2.0 #2634

• Alpine has been updated to 3.15 on release image #2582

• External registry requests now show BuildKit major version in User-Agent header (previously Containerd) #2593

• Fix caching of weak ETag references when pulling HTTP URLs #2629

• Avoid gRPC size limits when transferring lots of logs #2456

• Correct FileOp.Rm to not follow symlinks to the target #2474

• Validate manifest blobs mediatypes against their content #2469

• Make Git checkouts more deterministic for better cache reuse #2397

• Containerd worker now supports rootless mode #2660

• Fix handling tokens with multiple scopes #2431

• Fix possible leaking processes when using external decompressor #2620

• Fix possible issues when converting blobs to another compression #2600

• Fix symlink handling on doing copy with includePath filters #2318

• Performance of creating layer blobs has been improved in some cases #2601

• Request token first with a POST request and fall back to GET request if needed #2553

• Gracefully handle the case where a crash happens after snapshot commit #2564

• Improve Authority pseudo-header handling with new gRPC #2518

• Clean up temporary mounts on a restart that might have leaked after crash #2652 #2670

• Fix possible panic on deduplicating mounts #2519

• Fix shared cache mounts resulting in overlay corruption #2637

• Fix remote cache imports when invoking builds through gateway API #2659

• Fix possible panic on loading invalid config #2650

• Fixes for session handling for parallel builds from local sources #2585

• Fixes for scheduler inconsistency detector #2556

• Allow listening buildkitd on multiple sockets with --addr #2649

• Systemd definitions have been updated with Rootless and notify support #2473

Contributors

• Tõnis Tiigi
• CrazyMax
• Erik Sipsma
• Akihiro Suda
• Kohei Tokunaga
• Sebastiaan van Stijn
• Aaron Lehmann
• Koichi Shiraishi
• Alex Couture-Beil
• Brian Goff
• Justin Chadwell
• Cory Bennett
• Anders F Björklund
• Davis Schirmer
• Edgar Lee
• Jacob MacElroy
• Maxime Lagresle
• Andrey Smirnov
• Christian Weichel
• Csaba Apagyi
• Edgar Lee
• Hans van den Bogert
• Jonathan Giannuzzi
• Morlay
• Shingo Omura
• Shiwei Zhang
• Solomon Hykes
• Vlad A. Ionescu
• lugeng
• sunchunming

Dependency Changes

• github.com/Azure/go-ansiterm d6e3b3328b78 -> d185dfc1b5a1
• github.com/Microsoft/go-winio v0.4.17 -> v0.5.1
• github.com/Microsoft/hcsshim v0.8.18 -> v0.9.2
• github.com/armon/circbuf 5111143e8da2 new
• github.com/containerd/cgroups v1.0.1 -> v1.0.3
• github.com/containerd/console v1.0.2 -> v1.0.3
• github.com/containerd/containerd v1.5.3 -> v1.6.1
• github.com/containerd/continuity v0.1.0 -> v0.2.2
• github.com/containerd/go-cni v1.0.2 -> v1.1.3
• github.com/containerd/stargz-snapshotter v0.6.4 -> v0.11.2
• github.com/containerd/ttrpc v1.0.2 -> v1.1.0
• github.com/containernetworking/cni v0.8.1 -> v...

dockerfile/1.4.0-labs

Usage

# syntax=docker.io/docker/dockerfile:1.4.0-labs

dockerfile/1.4.0

Usage

# syntax=docker.io/docker/dockerfile:1.4.0

Notable changes

• COPY --link and ADD --link allow copying files with increased cache efficiency and rebase images without requiring them to be rebuilt. --link copies files to a separate layer and then uses new LLB MergeOp implementation to chain independent layers together. #2596 #2672 Documentation

• Heredocs support have been promoted from labs channel to stable. This feature allows writing multiline inline scripts and files #2589 Documentation

• Additional named build contexts can be passed to build to add or overwrite a stage or an image inside the build. A source for the context can be a local source, image, Git, or HTTP URL. #2521 #2550 #2549 #2693 Buildx Documentation

• When using a cross-compilation stage, the target platform for a step is now seen on progress output #2576

• BUILDKIT_SANDBOX_HOSTNAME build-arg can be used to set the default hostname for the RUN steps. #2373

• Fixes for some cases where Heredocs incorrectly removed quotes from content #2442

v0.10.0-rc2

Welcome to the 0.10.0-rc2 release of buildkit!
This is a pre-release of buildkit

Please try out the release binaries and report any issues at
https://github.com/moby/buildkit/issues.

Notable changes

• Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache import #2651
• Build status stream now supports ProgressGroup object to group multiple LLB steps into a single progress item #2668
• Fixes to tracking references to blobs with multiple compressions #2674
• Fixes to progress stream handling of multiple "complete" events #2675 #2641
• Extra progress step has been added to the step where blobs are prepared for inline cache #2658
• Buildinfo attributes now contain information about the additional named contexts #2654
• Fix possible panic with eStargz on restart #2670
• Clean up temporary mounts on a restart that might have leaked after crash #2652
• Fix remote cache imports when invoking builds through gateway API #2659
• Containerd worker now supports rootless mode #2660
• Enable eStargz-based lazy pulling on registry cache importer #2648
• Fix possible panic on buildinfo generation #2657
• Buildinfo can now be exported independently from exporting an image #2645
• Fix possible panic on loading invalid config #2650
• Allow listening buildkitd on multiple sockets with --addr #2649
• Fix shared cache mounts resulting in overlay corruption #2637
• Update CNI to avoid possible panic in previous release #2640

Dependency Changes

• github.com/containerd/containerd v1.6.0-rc.2 -> v1.6.0
• github.com/containerd/go-cni v1.1.2 -> v1.1.3
• github.com/containerd/stargz-snapshotter v0.11.0 -> b1ce4c8d8294
• github.com/containerd/stargz-snapshotter/estargz v0.11.0 -> b1ce4c8d8294
• github.com/docker/docker 40bb9831756f -> c78f6963a1c0
• github.com/google/go-cmp v0.5.6 -> v0.5.7
• github.com/klauspost/compress v1.14.2 -> v1.14.3
• github.com/prometheus/client_golang v1.12.0 -> v1.12.1
• go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.28.0 -> v0.29.0
• go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace 2bb24f926b80 -> v0.29.0
• go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp 7e31ebe04030 -> v0.29.0
• go.opentelemetry.io/otel v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/exporters/jaeger v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/internal/metric v0.26.0 -> v0.27.0
• go.opentelemetry.io/otel/metric v0.26.0 -> v0.27.0
• go.opentelemetry.io/otel/sdk v1.3.0 -> v1.4.1
• go.opentelemetry.io/otel/trace v1.3.0 -> v1.4.1
• go.opentelemetry.io/proto/otlp v0.11.0 -> v0.12.0

Previous release can be found at v0.10.0-rc1

dockerfile/1.4.0-rc2

Usage

# syntax=docker.io/docker/dockerfile-upstream:1.4.0-rc2

Notable changes

• Fix COPY --link resetting previous platform/environment information in some cases #2672

v0.10.0-rc1

Welcome to the 0.10.0-rc1 release of buildkit!
This is a pre-release of buildkit

Please try out the release binaries and report any issues at
https://github.com/moby/buildkit/issues.

Notable changes:

• New LLB operation MergeOp allows efficiently chaining groups of layers together without the need to access the individual files. This allows to build layers separately and merge them together later without making them depend on each other. MergeOp can work with remote references, for example, it can be used to rebase an image on top of a newer base image without ever pulling or pushing any layers. #2335 #2614

• New LLB operation DiffOp allows computing a difference between two points in LLB graph containing the files that have been added and whiteouts for files that have been removed. When DiffOp inputs are based on the same layer chain DiffOp can work directly with remote layer references and doesn't need to pull layers to access individual files. Files of the DiffOp result can be accessed directly or used as input to a MergeOp. #2517 #2434 #2563

• New build information structures are generated with build metadata that allows you to see all the sources (images, git repositories) that were used by the build with their exact versions and also the configuration that was passed to the build. This information can also be embedded into the image configuration if one is generated. Build sources are added to the image config by default. The build configuration is not currently embedded by default to avoid credential leaks in poorly written Dockerfiles but the intention is to enable it in the future. #2311 #2476

• When possible, blobs are now created with native OverlayFS differ with increased performance. This Differ can directly use files in OverlayFS upper directory instead of scanning for differences between snapshots. #2181 #2491 #2480 #2388 #2390

• Frontends can now send warning messages that are shown on the progressbar. Warnings can be associated with specific LLB vertex and contain additional information like URL to documentation or location in original source code. #2482 #2498

• Layer blobs can now be exported in Zstd compression format with -o compression=zstd. Zstd provides smaller files and faster decompression than gzip. #2344

• Layers can now be exported with eStargz compression type -o compression=estargz #2246 #2603 #2352

• A compression level can be set with -o compression-level=N to configure the compressor when new layer blobs are created. This can be used to create smaller blobs by spending more time on the compression step. #2591

• Remote cache inlined in image configuration now supports arbitrary configuration of image layers as cache sources #2501

• Support exporting non-distributable blob descriptors. -o prefer-nondist-layers=true exports layer with a non-distributable mediatype and external URL if such URL was provided when blob was pulled from the remote registry. Descriptors with non-distributable mediatypes are skipped on push. #2561

• Build metadata now provides access to OCI descriptor of the result if one was generated. Previously only digest of the descriptor could be accessed. #2610

• Builder now understands AMD64 Microarchitecture levels, e.g. linux/amd64/v2, linux/amd64/v4. The default variant remains v1 and is normalized to linux/amd64. These variants allow creating more optimized versions of your images that run when the container is running on a more modern CPU. #2588

• LLB now supports progress groups for grouping multiple steps together so they appear as the same row in build progress #2513

• LLB ExecOp now supports build secrets that are exposed as environment variables #2579

• Interactive container API now supports sending signals to processes from the client #2590

• Logs now use a rolling buffer to show the last logs for a process that ended with an error even if regular logs have been clipped because they have reached the max-logs limit. #2577

• Frontends can now access their own definition and call additional processes on their own image #2443

• Tmpfs mounts can now set a size limit #2411 #2422

• Custom Cgroup parent can now be set the LLB ExecOp #2430

• Add support for Ulimit resource limits in LLB ExecOp #2398

• Add extra hosts supports to Gateway Exec API #2294

• Fix security mode config being dropped on Gateway Exec start #2290

• Git source can now correctly clone annotated tag references #2570

• OpenTelemetry tracing propagation environment variables have been updated to TRACEPARENT and TRACESTATE to match the changes in upstream. Old variables are deprecated and will be removed in the next release. #2572

• Make sure supplementary groups are loaded for the default user configuration #2428

• Allow exporting inline cache when blobs exist in multiple compressions #2405

• Github cache backend retry logic on hitting rate limits has been improved #2506

• Color schema on TTY progressbar has been enhanced on Windows for better readability #2368

• Allow insecure security mode to work on environments where all capabilities are not available #2394

• Use standard user umask for Git processes #2356

• Fix tracing indicators showing up in logs even when tracing is not enabled #2351

• Handling of doublestar (**) pattern has been improved on transporting local sources. #2319 #2486

• QEMU embedded emulators have been updated to v6.2.0 #2634

• Alpine has been updated to 3.15 on release image #2582

• External registry requests now show BuildKit major version in User-Agent header (previously Containerd) #2593

• Fix caching of weak ETag references when pulling HTTP URLs #2629

• Avoid gRPC size limits when transferring lots of logs #2456

• Correct FileOp.Rm to not follow symlinks to the target #2474

• Validate manifest blobs mediatypes against their content #2469

• Make Git checkouts more deterministic for better cache reuse #2397

• Fix cross-repo push fallback when credentials for source repository become invalid #2630

• Fix handling tokens with multiple scopes #2431

• Fix possible leaking processes when using external decompressor #2620

• Fix possible issues when converting blobs to another compression #2600

• Fix symlink handling on doing copy with includePath filters #2318

• Performance of creating layer blobs has been improved in some cases #2601

• Request token first with a POST request and fall back to GET request if needed #2553

• Gracefully handle the case where a crash happens after snapshot commit #2564

• Improve Authority pseudo-header handling with new gRPC #2518

• Fix possible panic on deduplicating mounts #2519

• Fixes for session handling for parallel builds from local sources #2585

• Fixes for scheduler inconsistency detector #2556

• Systemd definitions have been updated with Rootless and notify support #2473

Contributors

• Tõnis Tiigi
• CrazyMax
• Erik Sipsma
• Akihiro Suda
• Kohei Tokunaga
• Sebastiaan van Stijn
• Aaron Lehmann
• Koichi Shiraishi
• Alex Couture-Beil
• Brian Goff
• Justin Chadwell
• Cory Bennett
• Anders F Björklund
• Davis Schirmer
• Jacob MacElroy
• Maxime Lagresle
• Andrey Smirnov
• Christian Weichel
• Csaba Apagyi
• Edgar Lee
• Hans van den Bogert
• Jonathan Giannuzzi
• Morlay
• Shingo Omura
• Shiwei Zhang
• Solomon Hykes
• lugeng
• sunchunming

Dependency Changes

• github.com/Microsoft/go-winio v0.4.17 -> v0.5.1
• github.com/Microsoft/hcsshim v0.8.18 -> v0.9.2
• github.com/containerd/cgroups v1.0.1 -> v1.0.3
• github.com/containerd/console v1.0.2 -> v1.0.3
• github.com/containerd/containerd v1.5.3 -> v1.6.0-rc.2
• github.com/containerd/continuity v0.1.0 -> v0.2.2
• github.com/containerd/go-cni v1.0.2 -> v1.1.2
• github.com/containerd/stargz-snapshotter v0.6.4 -> v0.11.0
• github.com/containerd/ttrpc v1.0.2 -> v1.1.0
• github.com/containernetworking/cni v0.8.1 -> v1.0.1
• github.com/dimchansky/utfbom v1.1.1 new
• github.com/docker/cli v20.10.7 -> v20.10.12
• github.com/docker/distribution v2.7.1 -> v2.8.0
• github.com/docker/docker ef4d47340142 -> 40bb9831756f
• github.com/docker/docker-credential-helpers v0.6.3 -> v0.6.4
• github.com/gogo/googleapis v1.4.0 -> v1.4.1
• github.com/google/uuid v1.2.0 -> v1.3.0
• github.com/grpc-ecosystem/go-grpc-middleware v1.2.0 -> v1.3.0
• github.com/hanwen/go-fuse/v2 v2.1.0 -> f57e95bda82d
• github.com/klauspost/compress v1.12.3 -> v1.14.2
• github.com/moby/sys/signal v0.6.0 new
• github.com/moby/term bea5bbe245bf -> 3f7ff695adc6
• github.com/opencontainers/image-spec v1.0.1 -> 693428a734f5
• **github.com/opencontainer...

dockerfile/1.4.0-rc1

Usage

# syntax=docker.io/docker/dockerfile-upstream:1.4.0-rc1

Notable changes

• COPY --link and ADD --link allow copying files with increased cache efficiency and rebase images without requiring them to be rebuilt. --link copies files to a separate layer and then uses new LLB MergeOp implementation to chain independent layers together. #2596 Documentation

• Heredocs support have been promoted from labs channel to stable. This feature allows writing multiline inline scripts and files #2589 Documentation

• Additional named build contexts can be passed to build to add or overwrite a stage or an image inside the build. A source for the context can be a local source, image, Git, or HTTP URL. #2521 #2550 #2549 Buildx Documentation

• When using a cross-compilation stage, the target platform for a step is now seen on progress output #2576

• BUILDKIT_SANDBOX_HOSTNAME build-arg can be used to set the default hostname for the RUN steps. #2373

• Fixes for some cases where Heredocs incorrectly removed quotes from content #2442

v0.9.3

Notable changes

• Workaround for deadlock in containerd pusher #2461
• Update containerd to 1.5.8 . Contains patch for CVE-2021-41190 #2470 #2462

v0.9.2

https://hub.docker.com/r/moby/buildkit

Notable changes

• Fix handling authentication requests with multiple scopes #2431
• Fix a possible deadlock when pushing items from one registry to another #2403
• Fix issues with concurrent cache export requests #2410
• Fix handling external blob deletions when reexporting cache with Github Actions backend #2433