Support multi-platform builds

[LuaxY]

Is it possible to support multi-platform (e.g. amd64, arm64, arm v7) image build with docker buildx feature?
With docker command, by using compatible builder, we can now provide --platform flag to specify target platform.

docker [buildx] build --platform linux/amd64,linux/arm64,linux/arm/v7 .

https://docs.docker.com/buildx/working-with-buildx/

[adamgordonbell]

Hi @LuaxY, I believe this should be possible and should be a feature we add. Thank you for submitting it.

[vladaionescu]

Experimental support for multi-platform builds is available in v0.4.3. Here is a simple example: https://github.com/earthly/earthly/blob/main/examples/multiplatform/Earthfile .

If you try this feature out, I'd be very curious about your experience with it - either here or on our Slack.

[LuaxY]

Tested on my project, work perfectly, big thanks!!

Question about local artifacts, example:

main.go

package main

import "fmt"

func main() {
	fmt.Println("test")
}

Earthfile

FROM golang:1.15-alpine
RUN apk --update --no-cache add git
WORKDIR /mailtrap

all:
  BUILD \
    --platform=linux/amd64 \
    --platform=linux/arm64 \
    --platform=linux/arm/v7 \
    --platform=linux/arm/v6 \
    +build

build:
  COPY main.go main.go
  RUN go build -o build/test main.go
  SAVE ARTIFACT build/test AS LOCAL build/test

This will build binary for each platform but also override the local artifcat

+build | Artifact test:master+build/test as local build/test
+build | Artifact test:master+build/test as local build/test
+build | Artifact test:master+build/test as local build/test
+build | Artifact test:master+build/test as local build/test

The final binary is the last platform stage to finish the SAVE ARTIFACT

$ file build/test 
build/test: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, [...]

This is the intended behavior? or there is a way to control the artifact name depending the platform?

[vladaionescu]

Great!

There are four new built-in args available, which you can use for this:

• TARGETPLATFORM
• TARGETOS
• TARGETARCH
• TARGETVARIANT

You need to predeclare them before using them:

ARG TARGETPLATFORM
RUN echo "The platform is $TARGETPLATFORM"

So in your example, you could have

build:
  COPY main.go main.go
  RUN go build -o build/test main.go
  ARG TARGETPLATFORM
  SAVE ARTIFACT build/test AS LOCAL build/$TARGETPLATFORM/test

[LuaxY]

Perfect thank you!

[bhansconnect]

Is there a way to remerge a build split by platform instead of propagating it through all dependant builds?
Ex:

get-dep-linux-arm64:
    # Download/setup arm64 version of dep

get-dep-linux-amd64:
    # Download/setup amd64 version of dep

build-app:
    # ideally I would want this to pick the right version of get dep. And just say something like:
    FROM +get-dep

Instead, i believe with the current setup, i would need to make build-app-linux-arm64 and build-app-linux-amd64

[vladaionescu]

You should have get-dep defined in a single target. You can use builtin args like TARGETPLATFORM if you need to do a bash conditional. In the future we plan on having a built-in IF command.

After that, you can reference FROM +get-dep and it'll use the same platform build-app is being built as.

[bhansconnect]

Yeah, I saw the if proposal, it would definitely solve my issue.

Frankly all I need to do to solve my specific issue is to map targetplatform.
Essentially:
amd64 -> x86_64
arm64 -> aarch64
etc
And then I would be able to just directly download the file and extract the file.
Probably will just make a small bash script with that if to deal with the conversion.

[n-rodriguez]

Hi there! Do you plans for the --platform CLI option?

For now it says :

earthly --platform=linux/amd64 --platform=linux/arm64 --build-arg DISTRO_NAME=bullseye +build 
           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
Error: multi-platform builds are not yet supported on the command line. You may, however, create a target with the instruction BUILD --plaform ... --platform ... +build

An Earthfile with BUILD is not really an option since it must be called dynamically.

Thank you!

[vladaionescu]

No plans yet.. wondering what you mean by "dynamically". I believe the Earthfile should have enough flexibility in terms of parameters / IF,ELSE etc. Wondering if you have a use-case where the Earthfile language isn't enough.

[n-rodriguez]

wondering what you mean by "dynamically"

From a ruby script :earthly --platform ... +build

The arch list is stored in a YAML file read by Ruby which calls earthly.

Putting the arch list in Earthfile would defeat the purpose of the whole thing.

[n-rodriguez]

Wondering if you have a use-case where the Earthfile language isn't enough.

The issue is not the Earthfile language but the CLI.

[n-rodriguez]

wondering what you mean by "dynamically"

read it as generic

[vladaionescu]

Got it - yeah makes sense. We should support that. Mind opening a new issue about it?