Fix handling of platform flag

[giggsoff]

In case of 'FROM --platform' defined I can see 'ERROR: no match for platform in manifest: not found'. Let's handle this case using arch-specific images with suffixes.

Signed-off-by: Petr Fedchenkov giggsoff@gmail.com

[deitch]

@giggsoff first, thanks not only for the PR, but especially for the tests.

Can you describe the scenario a bit better? Is the FROM --platform= in the lkt cache? Or is it failing to get from the registry (which should work, but I haven't tried in some time)?

If the first, why would it fail? Should it not just find the multi-arch index for it?

[giggsoff]

The problem come when trying to use image from lkt cache.

If the first, why would it fail? Should it not just find the multi-arch index for it?

I agree with you, it should not fail. Moreover, lkt returns digest of the index. But seems problem comes from some kind of filtering (by platform) on buildkit side.

[deitch]

That really is strange. I would like us to see if we can figure out what is going on. Can we create a really simple example to recreate it? Rather than work around the issue, I would like to fix it. If it is a bug in buildkit, I can raise it upstream; if it is an issue in how we are passing parameters, let's fix that.

[giggsoff]

Can we create a really simple example to recreate it?

It is inside of test in this PR. Just one image with defined --platform chained from another in lkt cache (not pushed). Please try the test if you will have a time. I will try to locate the place in buildkit where we hit the problem (the error come from dependencies, I will try to find where they are called).

[deitch]

It is inside of test in this PR

Oh yeah, good point. I will try to work with it. Let's see what we get.

[deitch]

@giggsoff I managed to set up a test... and it works and does not. Strange.

I am running buildkitd locally as root via

# buildkitd --allow-insecure-entitlement network.host --addr unix:///run/buildkit/buildkitd.sock --debug

Then I ran your first build, saved it, and then modified the second to have a simple name:

FROM --platform=linux/amd64 test222  as src
FROM --platform=linux/amd64 alpine:3.15
COPY --from=src /foo /foo
RUN cat /foo
# copy output of uname commands from first build and compare with current
COPY --from=src /uname* /
RUN uname -m >/uname_m_current && uname -s >/uname_s_current && cat /uname_m_current && cat /uname_s_current
RUN diff /uname_m /uname_m_current && diff /uname_s /uname_s_current

And finally:

$ buildctl build --frontend dockerfile.v0 --output type=image,name=docker.io/foo/bar --local context=/tmp/build/b2 --local dockerfile=/tmp/build/b2 --opt context:test222=oci-layout://dump@sha256:bd04a5b26dec16579cd1d7322e949c5905c4742269663fcbc84dcb2e9f4592fb --oci-layout dump=/tmp/lkt[+] Building 0.2s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                       0.0s
 => => transferring dockerfile: 441B                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/alpine:3.15                                                                                             0.1s
 => [context test222] load metadata for dump@sha256:bd04a5b26dec16579cd1d7322e949c5905c4742269663fcbc84dcb2e9f4592fb                                       0.0s
 => [stage-1 1/6] FROM docker.io/library/alpine:3.15@sha256:cf34c62ee8eb3fe8aa24c1fab45d7e9d12768d945c3f5a6fd6a63d901e898479                               0.0s
 => => resolve docker.io/library/alpine:3.15@sha256:cf34c62ee8eb3fe8aa24c1fab45d7e9d12768d945c3f5a6fd6a63d901e898479                                       0.0s
 => [context test222] OCI load from client                                                                                                                 0.0s
 => => resolve oci-layout/dummy@sha256:bd04a5b26dec16579cd1d7322e949c5905c4742269663fcbc84dcb2e9f4592fb                                                    0.0s
 => CACHED [stage-1 2/6] COPY --from=src /foo /foo                                                                                                         0.0s
 => CACHED [stage-1 3/6] RUN cat /foo                                                                                                                      0.0s
 => CACHED [stage-1 4/6] COPY --from=src /uname* /                                                                                                         0.0s
 => CACHED [stage-1 5/6] RUN uname -m >/uname_m_current && uname -s >/uname_s_current && cat /uname_m_current && cat /uname_s_current                      0.0s
 => CACHED [stage-1 6/6] RUN diff /uname_m /uname_m_current && diff /uname_s /uname_s_current                                                              0.0s
 => exporting to image                                                                                                                                     0.0s
 => => exporting layers                                                                                                                                    0.0s
 => => exporting manifest sha256:22b2d7701933aceec3a4793c3723bf949d900cf36c2422cd2ff65365f8d06300                                                          0.0s
 => => exporting config sha256:91616bcf7ab152779dc3f21e7a93f4c11ed4062d4abe8a69dc03b33790582e23

Well, that worked. It is finding the right image and right platform. So now I change it just to change the name:

FROM --platform=linux/amd64 docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6  as src
# rest is the same

And run the modified command:

$ buildctl build --frontend dockerfile.v0 --output type=image,name=docker.io/foo/bar --local context=/tmp/build/b2 --local dockerfile=/tmp/build/b2 --opt context:docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6=oci-layout://dump@sha256:bd04a5b26dec16579cd1d7322e949c5905c4742269663fcbc84dcb2e9f4592fb --oci-layout dump=/tmp/lkt
[+] Building 0.1s (4/4) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                       0.0s
 => => transferring dockerfile: 567B                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/alpine:3.15                                                                                             0.1s
 => ERROR [internal] load metadata for docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a6  0.1s
------
 > [internal] load metadata for docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6:
------
Dockerfile:1
--------------------
   1 | >>> FROM --platform=linux/amd64 docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6  as src
   2 |     FROM --platform=linux/amd64 alpine:3.15
   3 |     COPY --from=src /foo /foo
--------------------
error: failed to solve: docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6: docker.io/linuxkit/chained-build-platform-image1-test-does-not-exist-anywhere-else-123456789:0bffed5b7a288b70e50a638fb5afb0e5d5aa03a6: not found

Not sure what is happening here.

[giggsoff]

I put # syntax=docker/dockerfile-upstream:master, as the problem seems to be fixed by moby/buildkit#3397. Let's wait for the stable tag.

[deitch]

I like having the test in here, so thank you @giggsoff .

I am going to hold on merging this until you remove it as draft, but also until we see when it ends up in a regular moby/buildkit docker image. When it does, we should be able to remove the # syntax line.

[deitch]

@giggsoff can you remove the # syntax line and update this line to reference the hash daadc9f6dc5829c9dfa68ef9f4292c2d5b2fddf848046e02e43db53037373dda? That should include it.

[giggsoff]

Sorry for the large diff. I decided to update imported buildkit version as well.