Upgrading to 2.20.2 seems to lead to "The database schema version does not align with the server version" Error

[captainkev76]

Before you start please confirm the following.

• Yes, I've searched similar issues on GitHub.
• Yes, I've checked whether this issue is covered in the Portainer documentation or knowledge base.

Problem Description

Portainer was working perfectly. I saw the note on the Dashboard saying that portainer 2.20.2 was available, so thought I'd go ahead and upgrade. I stopped the portainer container, removed the container, removed the image and then pulled the latest "latest" image, and re-launched the container (all following the instructions at https://docs.portainer.io/start/upgrade). When I done so, I saw portainer container restart every few seconds, and the logs shows these steps recurring:

2024/05/13 02:11PM INF github.com/portainer/portainer-ee/api/cmd/portainer/main.go:448 > encryption key file not present | filename=portainer
2024/05/13 02:11PM INF github.com/portainer/portainer-ee/api/cmd/portainer/main.go:482 > proceeding without encryption key |
2024/05/13 02:11PM INF github.com/portainer/portainer-ee/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db
2024/05/13 02:11PM FTL github.com/portainer/portainer-ee/api/cmd/portainer/main.go:489 > The database schema version does not align with the server version. Please consider reverting to the previous server version or addressing the database migration issue. |

I repeated the above process with version 2.20.1 instead of latest, and it runs fine. To check it wasn't a weirdo thing, I repeated the upgrade attempt and sure enough the problem returned.

Expected Behavior

Upgrade would go ahead and migrate database to new version or format if required.

Actual Behavior

Database does not seem to get upgraded or reformatted

Steps to Reproduce

docker container stop portainer
docker container rm portainer
docker image rm [image name]
docker pull portainer/portainer-ee:latest
docker run -d -p 8000:8000 -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ee:latest

Portainer logs or screenshots

No response

Portainer version

2.20.2

Portainer Edition

Business Edition (BE/EE) with 5NF / 3NF license

Platform and Version

Docker 26.1.2

OS and Architecture

Ubuntu 24.04 LTS, Intel 64-bit i7

Browser

Firefox latest

What command did you use to deploy Portainer?

docker run -d -p 8000:8000 -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ee:latest

Additional Information

No response

[captainkev76]

Potentially this was a weirdo image-related thing after all. I updated explicitly to 2.20.2 and it worked ok, so I then deleted all images and went back to :latest and it's working again.

Feel free to close this!

[jamescarppe]

The 2.20 branch is not on the :latest tag as it is our STS (Short-Term Support) release. To use the STS releases you'll either want to use the explicit version tags (like :2.20.2) as you have above, or the :sts meta tag. The new features and fixes in the 2.20 branch will eventually make it to the main (LTS) branch in 2.21, which will also be tagged as the :latest tag.

At present, :latest is still pointed to the 2.19 branch, and our "major" versions can include database schema changes so we bump the database version as part of them. Essentially what you've done is gone from 2.20 and it's database schema version to 2.19 (via :latest) with an older schema version, which isn't something we support (and hence the error message).