Partial fix for Buildkit performance issues (#3143)

This tests, and implements earthly/buildkit#20
as a partial performance resolution to #3085. Some of the higher
resource utilization at the end exists (per user reports testing this
fix), but the pauses appear to be gone.

CHANGELOG.md

@@ -9,6 +9,7 @@ All notable changes to [Earthly](https://github.com/earthly/earthly) will be doc
 
 ### Fixed
 - Fixed a bug, where the command to create tokens with a set expiration failed.
+- Long pauses at the end of builds, which were characterized by apparent freezes or delays with the message `Waiting on Buildkit...`. 
 
 ### Changed
 - Setting env vars like  `FORCE_COLOR`, or `EARTHLY_FULL_TARGET` to `0`, `false`, `FALSE`, or `` (an empty-string) will no longer force the color, use any other value like `1`, `true`, or `yesplease`.

buildkitd/Earthfile

@@ -12,7 +12,7 @@ buildkitd:
             ARG BUILDKIT_BASE_IMAGE=$BUILDKIT_PROJECT+build
         END
     ELSE
-        ARG BUILDKIT_BASE_IMAGE=github.com/earthly/buildkit:cf348cf54d34478bb7c0ac62b0036802a6d59dd8+build
+        ARG BUILDKIT_BASE_IMAGE=github.com/earthly/buildkit:2b7fc7d64928ffef4cdbb29cf541a149ca1183c0+build
     END
     ARG EARTHLY_TARGET_TAG_DOCKER
     ARG TAG="dev-$EARTHLY_TARGET_TAG_DOCKER"

go.mod

@@ -139,6 +139,6 @@ replace (
 	github.com/jdxcode/netrc => github.com/mikejholly/netrc v0.0.0-20221121193719-a154cb29ec2a
 	github.com/jessevdk/go-flags => github.com/alexcb/go-flags v0.0.0-20210722203016-f11d7ecb5ee5
 
-	github.com/moby/buildkit => github.com/earthly/buildkit v0.0.0-20230714193834-cf348cf54d34
+	github.com/moby/buildkit => github.com/earthly/buildkit v0.0.0-20230725173349-2b7fc7d64928
 	github.com/tonistiigi/fsutil => github.com/earthly/fsutil v0.0.0-20230530182746-82567e6e2a92
 )

go.sum

@@ -169,8 +169,8 @@ github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4
 github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
-github.com/earthly/buildkit v0.0.0-20230714193834-cf348cf54d34 h1:CczFcEbYRQzV/3wQz2I9YIGNBrFNVFeTeIiDd9tocwg=
-github.com/earthly/buildkit v0.0.0-20230714193834-cf348cf54d34/go.mod h1:RV8GAHclVvcjdPGdNAL4DnV/Mca1Lv5JnOwWuvw/IPo=
+github.com/earthly/buildkit v0.0.0-20230725173349-2b7fc7d64928 h1:wvVw10dGOap5vcmWv7yWeAPXrK2WbnJRQtobOzXGy50=
+github.com/earthly/buildkit v0.0.0-20230725173349-2b7fc7d64928/go.mod h1:RV8GAHclVvcjdPGdNAL4DnV/Mca1Lv5JnOwWuvw/IPo=
 github.com/earthly/cloud-api v1.0.1-0.20230718201745-618092fd204c h1:hSfK+slM1jjAPLQJ12c6vsVoNPOvTgzHSW3NHMouu4E=
 github.com/earthly/cloud-api v1.0.1-0.20230718201745-618092fd204c/go.mod h1:rU/tYJ7GFBjdKAITV2heDbez++glpGSbtJaZcp73rNI=
 github.com/earthly/fsutil v0.0.0-20230530182746-82567e6e2a92 h1:dy3w3B4OiDyQS+iHi/beUgMDJ1+fAAgbtr7OZ0d9ks0=