Publish and test arm docker images for every PR

[friedrich-at-adobe]

Is your feature request related to a problem? Please describe.
Many users are already trying to use cortex with ARM.

Describe the solution you'd like
For every PR we should run tests in ARM and Intel.

Describe alternatives you've considered
I don't think there is an alternative, we have to support natively what users are doing.

There is already some support in:

cortex/Makefile

Lines 52 to 59 in cd78607

	push-multiarch-build-image:
	@echo
	# Build image for each platform separately... it tends to generate fewer errors.
	$(SUDO) docker buildx build --platform linux/amd64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) build-image/
	$(SUDO) docker buildx build --platform linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) build-image/
	# This command will run the same build as above, but it will reuse existing platform-specific images,
	# put them together and push to registry.
	$(SUDO) docker buildx build -o type=registry --platform linux/amd64,linux/arm64 --build-arg=revision=$(GIT_REVISION) --build-arg=goproxyValue=$(GOPROXY_VALUE) -t $(IMAGE_PREFIX)build-image:$(IMAGE_TAG) build-image/

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

[otterley]

Several of my customers at AWS have asked about this. Is there anything I can do to help move this along? I can provide arm64 build resources if needed.

[alvinlin123]

Currently we only publish ARM for build images. We still need to build the actual Cortex image for ARM.

[otterley]

@alvinlin123 I'd like to help produce the multi-arch image and publish it. I can provide ARM build resources for you as needed.

[alvinlin123]

@otterley do you mean you have someone who can work on updating the build workflow?

I have not dive deep into what needs to be done, but at high level I think it involves making some modifications to the GitHub Action workflow. If you work on modifying and testing the workflow, I can provide some guidance on what I know :)

[otterley]

@alvinlin123 Let's take this offline - can you email me at fiscmi at amazon ?

[SuperMatt]

Hi, is there any movement on this? My company is looking to move some infra to AWS Graviton instances, and would love a native arm64 image for this. With 1.14 coming out soon, it would be a good time to start.

[alanprot]

Hi @SuperMatt

I just created an multi arch Image on dockerhub based on master branch and i could run both on graviton and on amd64, you wanna give it a try?

PR in progress: #5041

amd64 ourput:

docker run cortexproject/cortex:HEAD-939e890

level=info ts=2022-12-14T07:44:54.0103856Z caller=main.go:194 msg="Starting Cortex" version="(version=1.14.0, branch=HEAD, revision=939e890)"
level=info ts=2022-12-14T07:44:54.0118135Z caller=server.go:306 http=[::]:80 grpc=[::]:9095 msg="server listening on addresses"
level=error ts=2022-12-14T07:44:54.0156121Z caller=log.go:117 msg="error running cortex" err="failed to initialize querier: failed to create bucket client: no s3 endpoint in config file\nerror initialising module: store-queryable\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).initModule\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:108\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).InitModuleServices\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:78\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/__w/cortex/cortex/pkg/cortex/cortex.go:398\nmain.main\n\t/__w/cortex/cortex/cmd/cortex/main.go:196\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"

amr64 output:

[root@ip-172-31-79-0 ec2-user]# docker run cortexproject/cortex:HEAD-939e890

level=info ts=2022-12-14T07:44:50.226950297Z caller=main.go:194 msg="Starting Cortex" version="(version=1.14.0, branch=HEAD, revision=939e890)"
level=info ts=2022-12-14T07:44:50.227581233Z caller=server.go:306 http=[::]:80 grpc=[::]:9095 msg="server listening on addresses"
level=error ts=2022-12-14T07:44:50.229156834Z caller=log.go:117 msg="error running cortex" err="no s3 endpoint in config file\ngithub.com/thanos-io/objstore/providers/s3.validate\n\t/__w/cortex/cortex/vendor/github.com/thanos-io/objstore/providers/s3/s3.go:348\ngithub.com/thanos-io/objstore/providers/s3.NewBucketWithConfig\n\t/__w/cortex/cortex/vendor/github.com/thanos-io/objstore/providers/s3/s3.go:219\ngithub.com/cortexproject/cortex/pkg/storage/bucket/s3.NewBucketClient\n\t/__w/cortex/cortex/pkg/storage/bucket/s3/bucket_client.go:17\ngithub.com/cortexproject/cortex/pkg/storage/bucket.NewClient\n\t/__w/cortex/cortex/pkg/storage/bucket/client.go:102\ngithub.com/cortexproject/cortex/pkg/purger.createBucketClient\n\t/__w/cortex/cortex/pkg/purger/tenant_deletion_api.go:122\ngithub.com/cortexproject/cortex/pkg/purger.NewTenantDeletionAPI\n\t/__w/cortex/cortex/pkg/purger/tenant_deletion_api.go:29\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).initTenantDeletionAPI\n\t/__w/cortex/cortex/pkg/cortex/modules.go:676\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).initModule\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:106\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).InitModuleServices\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:78\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/__w/cortex/cortex/pkg/cortex/cortex.go:398\nmain.main\n\t/__w/cortex/cortex/cmd/cortex/main.go:196\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_arm64.s:1172\ncreate bucket client\ngithub.com/cortexproject/cortex/pkg/purger.createBucketClient\n\t/__w/cortex/cortex/pkg/purger/tenant_deletion_api.go:124\ngithub.com/cortexproject/cortex/pkg/purger.NewTenantDeletionAPI\n\t/__w/cortex/cortex/pkg/purger/tenant_deletion_api.go:29\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).initTenantDeletionAPI\n\t/__w/cortex/cortex/pkg/cortex/modules.go:676\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).initModule\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:106\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).InitModuleServices\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:78\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/__w/cortex/cortex/pkg/cortex/cortex.go:398\nmain.main\n\t/__w/cortex/cortex/cmd/cortex/main.go:196\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_arm64.s:1172\nerror initialising module: tenant-deletion\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).initModule\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:108\ngithub.com/cortexproject/cortex/pkg/util/modules.(*Manager).InitModuleServices\n\t/__w/cortex/cortex/pkg/util/modules/modules.go:78\ngithub.com/cortexproject/cortex/pkg/cortex.(*Cortex).Run\n\t/__w/cortex/cortex/pkg/cortex/cortex.go:398\nmain.main\n\t/__w/cortex/cortex/cmd/cortex/main.go:196\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_arm64.s:1172

[SuperMatt]

@alanprot thanks for the update. I'm away for work now until the new year, so I won't be able to give this a try, but thank you so much for doing this. I look forward to testing it out as soon as I am able.