2024.4.2 update broke my docker-compose.yml

[SynVisions]

Steps To Reproduce

edit: I believe I have the root cause (and thus steps to reproduce) here: #4078 (comment) -- it seems that docker-compose 1.24.1 does not work with bitwarden-server 2024.4.2 despite current documentation stating that it should work.

I'm not sure if this is easily reproducible as I've been running the same self-hosted Bitwarden instance for years now, but this is what occurred in my environment. After updating from self-hosted bitwarden 2024.4.1 to 2024.4.2, my docker-compose.yml is now invalid, and bitwarden will no longer start.

The update was applied in this fashion:

1. ./bitwarden.sh updateself
2. ./bitwarden.sh update

Expected Result

When running ./bitwarden.sh start after updating I would expect to see something like this (except replace 2024.4.1 with 2024.4.2):

Apr 21 02:01:12 bitwarden bitwarden.sh[1746]:  _     _ _                         _
Apr 21 02:01:12 bitwarden bitwarden.sh[1746]: | |__ (_) |___      ____ _ _ __ __| | ___ _ __
Apr 21 02:01:12 bitwarden bitwarden.sh[1746]: | '_ \| | __\ \ /\ / / _` | '__/ _` |/ _ \ '_ \
Apr 21 02:01:12 bitwarden bitwarden.sh[1746]: | |_) | | |_ \ V  V / (_| | | | (_| |  __/ | | |
Apr 21 02:01:12 bitwarden bitwarden.sh[1746]: |_.__/|_|\__| \_/\_/ \__,_|_|  \__,_|\___|_| |_|
Apr 21 02:01:12 bitwarden bitwarden.sh[1747]: Open source password management solutions
Apr 21 02:01:12 bitwarden bitwarden.sh[1747]: Copyright 2015-2024, 8bit Solutions LLC
Apr 21 02:01:12 bitwarden bitwarden.sh[1747]: https://bitwarden.com, https://github.com/bitwarden
Apr 21 02:01:12 bitwarden bitwarden.sh[1747]: ===================================================
Apr 21 02:01:12 bitwarden bitwarden.sh[1745]: bitwarden.sh version 2024.4.1
Apr 21 02:01:12 bitwarden bitwarden.sh[1752]: Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1
Apr 21 02:01:13 bitwarden bitwarden.sh[1759]: docker-compose version 1.24.1, build 4667896b
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling mssql         ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling web           ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling attachments   ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling api           ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling identity      ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling sso           ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling admin         ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling icons         ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling notifications ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling events        ...
Apr 21 02:01:15 bitwarden bitwarden.sh[1777]: Pulling nginx         ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1777]: [2.8K blob data]
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating network "docker_default" with the default driver
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating network "docker_public" with the default driver
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-identity ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-attachments ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-web         ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-icons       ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-events      ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-mssql       ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-notifications ...
Apr 21 02:01:19 bitwarden bitwarden.sh[1793]: Creating bitwarden-api           ...
Apr 21 02:01:18 bitwarden bitwarden.sh[1793]: Creating bitwarden-sso           ...
Apr 21 02:01:20 bitwarden bitwarden.sh[1793]: [91B blob data]
Apr 21 02:01:21 bitwarden bitwarden.sh[1793]: [533B blob data]
Apr 21 02:01:25 bitwarden bitwarden.sh[1793]: [55B blob data]
Apr 21 02:01:25 bitwarden bitwarden.sh[3050]: 2024.4.1: Pulling from bitwarden/setup
Apr 21 02:01:25 bitwarden bitwarden.sh[3050]: Digest: sha256:b69e7a14ccbc1675128eba8bf5f0db519b83e26913efa830c20d8102306342d1
Apr 21 02:01:25 bitwarden bitwarden.sh[3050]: Status: Image is up to date for bitwarden/setup:2024.4.1
Apr 21 02:01:25 bitwarden bitwarden.sh[3050]: docker.io/bitwarden/setup:2024.4.1
Apr 21 02:01:54 bitwarden bitwarden.sh[3068]: Bitwarden is up and running!
Apr 21 02:01:54 bitwarden bitwarden.sh[3068]: ===================================================
Apr 21 02:01:54 bitwarden bitwarden.sh[3068]: visit https://bitwarden.dhcp
Apr 21 02:01:54 bitwarden bitwarden.sh[3068]: to update, run `./bitwarden.sh updateself` and then `./bitwarden.sh update`

Actual Result

What I see now:

sean@bitwarden:~$ ./bitwarden.sh start
 _     _ _                         _
| |__ (_) |___      ____ _ _ __ __| | ___ _ __
| '_ \| | __\ \ /\ / / _` | '__/ _` |/ _ \ '_ \
| |_) | | |_ \ V  V / (_| | | | (_| |  __/ | | |
|_.__/|_|\__| \_/\_/ \__,_|_|  \__,_|\___|_| |_|

Open source password management solutions
Copyright 2015-2024, 8bit Solutions LLC
https://bitwarden.com, https://github.com/bitwarden

===================================================

bitwarden.sh version 2024.4.2
Docker version 24.0.5, build 24.0.5-0ubuntu1~22.04.1
docker-compose version 1.24.1, build 4667896b

ERROR: The Compose file '/home/sean/bwdata/docker/docker-compose.yml' is invalid because:
Unsupported config option for services: 'notifications'
Unsupported config option for networks: 'public'
ERROR: The Compose file '/home/sean/bwdata/docker/docker-compose.yml' is invalid because:
Unsupported config option for services: 'identity'
Unsupported config option for networks: 'public'

I get different sets of errors each time I run it, for example the next time I got these:

ERROR: The Compose file '/home/sean/bwdata/docker/docker-compose.yml' is invalid because:
Unsupported config option for services: 'api'
Unsupported config option for networks: 'default'
ERROR: The Compose file '/home/sean/bwdata/docker/docker-compose.yml' is invalid because:
Unsupported config option for networks: 'default'
Unsupported config option for services: 'web'

Screenshots or Videos

No response

Additional Context

This is my docker-compose.yml:

#
# Useful references:
# https://docs.docker.com/compose/compose-file/
# https://docs.docker.com/compose/reference/overview/#use--f-to-specify-name-and-path-of-one-or-more-compose-files
# https://docs.docker.com/compose/reference/envvars/
#
#########################################################################
# WARNING: This file is generated. Do not make changes to this file.    #
# They will be overwritten on update. If you want to make additions to  #
# this file, you can create a `docker-compose.override.yml` file in the #
# same directory and it will be merged into this file at runtime. You   #
# can also manage various settings used in this file from the           #
# ./bwdata/config.yml file for your installation.                       #
#########################################################################

services:
  mssql:
    image: bitwarden/mssql:2024.4.2
    container_name: bitwarden-mssql
    restart: always
    stop_grace_period: 60s
    volumes:
      - ../mssql/data:/var/opt/mssql/data
      - ../logs/mssql:/var/opt/mssql/log
      - ../mssql/backups:/etc/bitwarden/mssql/backups
    env_file:
      - mssql.env
      - ../env/uid.env
      - ../env/mssql.override.env

  web:
    image: bitwarden/web:2024.4.2
    container_name: bitwarden-web
    restart: always
    volumes:
      - ../web:/etc/bitwarden/web
    env_file:
      - global.env
      - ../env/uid.env

  attachments:
    image: bitwarden/attachments:2024.4.2
    container_name: bitwarden-attachments
    restart: always
    volumes:
      - ../core/attachments:/etc/bitwarden/core/attachments
    env_file:
      - global.env
      - ../env/uid.env

  api:
    image: bitwarden/api:2024.4.2
    container_name: bitwarden-api
    restart: always
    volumes:
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/api:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  identity:
    image: bitwarden/identity:2024.4.2
    container_name: bitwarden-identity
    restart: always
    volumes:
      - ../identity:/etc/bitwarden/identity
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/identity:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  sso:
    image: bitwarden/sso:2024.4.2
    container_name: bitwarden-sso
    restart: always
    volumes:
      - ../identity:/etc/bitwarden/identity
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/sso:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  admin:
    image: bitwarden/admin:2024.4.2
    container_name: bitwarden-admin
    restart: always
    depends_on:
      - mssql
    volumes:
      - ../core:/etc/bitwarden/core
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/admin:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  icons:
    image: bitwarden/icons:2024.4.2
    container_name: bitwarden-icons
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/icons:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
    networks:
      - default
      - public

  notifications:
    image: bitwarden/notifications:2024.4.2
    container_name: bitwarden-notifications
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/notifications:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  events:
    image: bitwarden/events:2024.4.2
    container_name: bitwarden-events
    restart: always
    volumes:
      - ../ca-certificates:/etc/bitwarden/ca-certificates
      - ../logs/events:/etc/bitwarden/logs
    env_file:
      - global.env
      - ../env/uid.env
      - ../env/global.override.env
    networks:
      - default
      - public

  nginx:
    image: bitwarden/nginx:2024.4.2
    container_name: bitwarden-nginx
    restart: always
    depends_on:
      - web
      - admin
      - api
      - identity
    ports:
      - '80:8080'
      - '443:8443'
    volumes:
      - ../nginx:/etc/bitwarden/nginx
      - ../letsencrypt:/etc/letsencrypt
      - ../ssl:/etc/ssl
      - ../logs/nginx:/var/log/nginx
    env_file:
      - ../env/uid.env
    networks:
      - default
      - public


networks:
  default:
    internal: true
  public:
    internal: false

Build Version

2024.4.2

Environment

Self-Hosted

Environment Details

• OS: Ubuntu 22.04.4 LTS
• Docker
• Hardware: VM with 4 CPU cores and 8GiB of memory

Issue Tracking Info

• I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.

[SynVisions]

I was able to fix the issue by installing the latest docker-compose and deleting the old one (which was not installed via apt). So one bug seems to be that the documentation here says that these are the requirements:

Minimum	Recommended
Engine 19+ and Compose 1.24+	Engine 19+ and Compose 1.24+

As the output in the original issue report shows, I was previously running Docker Engine 24.0.5 and Compose 1.24.1. I think updating the documentation is prudent, but it would also be nice I think to fail the upgrade with a precondition check on the docker-compose version, as this upgrade broke my bitwarden installation.

[Ayitaka]

@djsmith85 please note this is NOT a duplicate of #4059 and should not be closed as such the way #4067 and #4068 were incorrectly closed.

This error is due to many systems not updating to the latest composer version via normal update channels (apt) and so they still default to version: "1", which obviously lacks some of the config options used in later versions (e.g. 3).

[Ayitaka]

I was able to fix the issue by installing the latest docker-compose and deleting the old one (which was not installed via apt).

@SynVisions Would you mind sharing the steps you took to updating your docker-compose, please?

[tobiasbaehr]

Had the same problem. Just delete old docker-compose binary.

The bitwarden.sh script use the old docker-compose by default.

if command -v docker-compose &> /dev/null
then
    dccmd='docker-compose'
else
    dccmd='docker compose'
fi

[SynVisions]

I was able to fix the issue by installing the latest docker-compose and deleting the old one (which was not installed via apt).

@SynVisions Would you mind sharing the steps you took to updating your docker-compose, please?

First I followed the steps here (you may need to use different steps depending upon your OS and distro): https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository

This did not fully solve the problem because as @tobiasbaehr calls out, the shell script looks for the existence of docker-compose (which is the old version) first. I don't remember how that was installed originally, but I simply did a sudo rm /usr/local/bin/docker-compose (path where it was in my case) and re-ran ./bitwarden.sh start and it then started using the new version of docker compose.

[LufoX11]

Ubuntu 18.04 here. Solution for me was a mix of everything.

1. First I upgraded docker via PPA:

~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
~# add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
~# apt update
~# apt-cache policy docker-ce
~# apt install docker-ce -y

Check docker is properly running with systemctl status docker.

2. Install docker-compose-plugin and remove docker-compose. docker-compose-plugin is the replacement for docker-compose binary so, from now on, ./docker-compose will be invoked as docker compose:

~# apt remove docker-compose
~# apt install docker-compose-plugin

3. (you may not need this). After running sudo -u bitwarden /opt/bitwarden/bwdata/scripts/updatebw.sh, it failed with

error getting credentials - err: exit status 1, out: `Cannot autolaunch D-Bus without X11 $DISPLAY`

So I had to delete golang-docker-credential-helpers:

~# apt remove golang-docker-credential-helpers

Then I run the update script again and all came back to normal. As mentioned by @tobiasbaehr the start script will try docker compose (which is the new way to do it) if docker-compose is not available, so it's crucial to remove the old package.

[spencer2785]

This did not fully solve the problem because as @tobiasbaehr calls out, the shell script looks for the existence of docker-compose (which is the old version) first. I don't remember how that was installed originally, but I simply did a sudo rm /usr/local/bin/docker-compose (path where it was in my case) and re-ran ./bitwarden.sh start and it then started using the new version of docker compose.

I needed to use ./bitwarden.sh rebuild after deleting docker-compose an then start ./bitwarden.sh start.
It would be nice if the script checks that the requirement for docker-compose are right before starting with an error.

[tferic]

Ran into the same issue. Deleting docker-compose manually fixed the issue. Thanks.
The Bitwarden software really should be changed to default to docker compose nowadays.

[Greenderella]

The 2024.5.0 image of the setup container now contains a fix for this issue. Please make sure you are running the latest version of Docker and update Bitwarden again to apply the fix.

Thank you for everybody's patience.