Fix missing ghcr.io/epinio/epinio-unpacker:v1.10.0-rc1

[mmartin24]

Issue:

Many UI tests started to fail with this error during deployment (as described in epinio/epinio-end-to-end-tests#381 (comment)):

AssertionError: Timed out retrying after 240000ms: Expected to find content: '===> EXPORTING' but never did.

The error may be related to missing manifest:

➜  ~ docker pull ghcr.io/epinio/epinio-unpacker:v1.10.0-rc1
Error response from daemon: manifest unknown

We should find the way to fix it.

---

Adding some notes from @thehejik on this issue in case it wants to be considered:

• Suggestion 1: Image was deleted by: https://github.com/epinio/epinio/actions/runs/6218917882/job/16876092871#step:2:163 , so it may be wise to stop it
• Suggestion 2: restore deleted package by: https://docs.github.com/en/rest/packages/packages?apiVersion=2022-11-28#restore-package-version-for-an-organization

[thehejik]

This is not relate only to epinio-unpacker but also to epinio-server images - this is probably why our multiarch/cosigned images are not available after a while.

[thehejik]

From chat:

The unpacker:v1.10.0-rc1 image has been deleted via this job https://github.com/epinio/epinio/actions/runs/6218917882/job/16876092871#step:2:163
Deleted old image: epinio-unpacker:125581570

This is epinio-unpacker pkg gh link pointing to ghcr.io/epinio/epinio-unpacker:v1.10.0-rc1 where the sha256 checksum is visible https://github.com/epinio/epinio/pkgs/container/epinio-unpacker/125581570

Note: The sha256 of the original image can be obtained if the original image is still pulled locally somewhere by

$ crictl inspecti ghcr.io/epinio/epinio-unpacker:v1.10.0-rc1
...
    "repoDigests": [
      "ghcr.io/epinio/epinio-> unpacker@sha256:771d72248ddd893399ebc96075cf6f002d27533cd629b1ea359492d264e9c53b"
...

Also the deleted package can be restored https://docs.github.com/en/rest/packages/packages?apiVersion=2022-11-28#restore-package-version-for-an-organization

The Delete image cleanup job has been disabled just before releasing v1.10.0-rc2.

[thehejik]

The manifests are still disappearing even after disabling the cleanup job.

This time it was ghcr.io/epinio/epinio-server:v1.10.0-rc2 released a day ago. Another manifest with the same name was created 9 hours ago for some unknown reason https://github.com/epinio/epinio/pkgs/container/epinio-server/129950793?tag=v1.10.0-rc2

But this image is not cosigned/multiarch anymore.

[thehejik]

@enrichman found the culprit:
I found the issue: https://github.com/epinio/epinio/actions/runs/6256478299/job/16987443577#step:13:277
It's in the installation tests, no idea why we are building and pushing the image

epinio/acceptance/helpers/epinio/epinio.go

Lines 64 to 73 in 102befe

	By("Pushing server image to GHCR ...")
	// PUBLIC_CLOUD is present
	// Pushing new image into ghcr for the public cluster to pull from
	out, err = proc.RunW("docker", "tag", local+":latest", remote)
	Expect(err).NotTo(HaveOccurred(), out)
	By(out)
	out, err = proc.RunW("docker", "push", remote)
	Expect(err).NotTo(HaveOccurred(), out)
	By(out)

introduced here: #1764

[enrichman]

We found that some tests in the https://github.com/epinio/epinio-end-to-end-tests were breaking.
This because after the v1.10.0-rc2 release the tests were still trying to fetch the old epinio-unpacker:v1.10.0-rc1, that "disapperead" and was deleted by other tests.

We should fix this, and have the tests running always against the latest versions. We are patching the epinio-server but not the unpacker.