Support to build images for multiple platforms

[aitorhh]

Hi, I was wondering if there are any plans to build images for multiple platforms. At least to provide images for the platform capstan is running.

So far, I detect some changes needed:

1. add cross compilation at make-dist
   env GOOS=linux GOARCH=arm64 go build -a -ldflags "-X main.VERSION$link_operator'$version' -w -s" -tags netgo -v -o dist/linux_arm64/capstan $package env GOOS=linux GOARCH=arm64 go build -a -ldflags "-X main.VERSION$link_operator'$version' -w -s" -tags netgo -v -o dist/linux_arm64/capstan $package
2. The architecture was hardcoded when looking for the qemu binaries to execute.

I see some more changes needed regarding the options for the qemu-system-aarch64 like -machine.

But I was wondering more about the package that are already used to build the new package (osv-loader, osv.bootloader,....) I guess they are provided for x86_64.

[miha-plesko]

@aitorhh congratulations, you've just realized one of the most painful drawbacks of the Capstan packaging approach 🎆 🍾 😄 Problem that you're describing is that packages compiled on one platform are not always compatible with those compiled on other platform. In other words, if osv.bootstrap package was compiled on Ubuntu14.04 and openjdk8-zulu-compact1 on Ubuntu 16.04, they probalby won't play well together because of symbols issue.

All this has nothing to do with Capstan itself, because Capstan builds no packages - it only copies files into the unikernel. There is even a blog post on this topic, please see the "Incompatibility issues" chapter if you're interested.

Is there a workaround? Yes - compile all packages on the same platform. To reach this, we use Docker container (see this repository) that currently bases on Ubuntu 14.04. We compile both the base unikernel and all the packages there, making them compatible one with each other. If you run following command, for example:

$ capstan package search openjdk
Name                                               Description                                        Version         Created              Platform
openjdk7                                           OpenJDK 1.7.0                                      0.2             2018-01-14 12:29     Ubuntu-14.04
openjdk8-zulu-compact1                             OpenJDK 1.8.0_112                                  0.2             2018-01-14 09:24     Ubuntu-14.04
openjdk8-zulu-compact3-with-java-beans             OpenJDK 1.8.0_112 zulu-compact3-with-java-beans    0.2             2018-01-14 10:50     Ubuntu-14.04
openjdk8-zulu-full                                 OpenJDK 1.8.0_112 zulu-full                        0.1             2018-01-28 13:40     Ubuntu-14.04

then in the very last column you can see that all of them were built on Ubuntu-14.04 platform.

Bottomline, to answer your original question: no, it is not possible, at the moment, to make it possible to mix packages compiled on different platforms, because we don't know how to resolve the symbols issue. Therefore we have workaround for it.

[aitorhh]

Great introduction! :) Thanks for providing all the references, that's more than I was expecting. This is actually unsolvable problem as you mentioned.

In docker, they used Image Manifest V 2, Schema 2. I've been using that for a while and I feel it's a good solution. Basically you push a different image for each platform (os, architecture) and then push a manifest with the multi-platform image.

Maybe the same principle could be applicable here :)

[miha-plesko]

Actually, this is partially done for Capstan repository already, because each base unikernel and each package has its own metafile (in yaml format) where we store the platform where it was built on (that's literally what Capstan printed in right-most column in previous comment). BUT we never had time to integrate those limitations into Capstan, other than printing them... Thanks for pointing me to the Docker's solution!