Services

Why compositions.sh?

The multi-staged nature and repetitive commands for setting up compositions with non-standard configuration (e.g., non-root containers, networking across compositions etc.) is the raison d'être for compositions.sh.

For running non-root containers with the host users UID:GID, so as to avoid permission issues with mounted volumes, one must at least create a .env file with variables pointing to their UID:GID, for docker compose to pick them up and replace them in YAML files. But, that's just the start!

One might want to attach some "hooks" when starting and/or stopping services. For instance, since Traefik can't simultaneously accept a static configuration file and some command-line configuration arguments, I have a "start" hook to generate a static configuration file with server-specific details (server's FQDN, ACME email and server etc.).

One might also want to attach containers to externally-created Docker networks, for sharing data across multiple compositions without exposing anything to host. In that case, a separate docker command must be executed before starting services. compositions.sh automatically detects external networks in YAML files, and creates them if they do not already exist.

Finally, I also added several safeguards that docker compose doesn't provide. For instance, missing host directories that are required to be mounted to containers are automatically created by Docker (and are owned by root!), but compositions.sh errors out in such cases requesting the user for explicit action.

Why not use Ansible?

Seemed like an overkill, when I initially wrote compositions.sh.

Usage

$ ./compositions.sh
Usage:
  ./compositions.sh <verb>[,<verb>,...] [flags] <comp_dir> [<comp_dir> ...]

Verbs:
  check                 Check health of a composition
  clean                 Delete '<comp_dir>/data'
  down                  Stop a composition
  overrides             List all override files in a composition
  pull                  Pull all images for a composition
  up                    Start a composition

Flags:
  [-F | --skip-fails]   Ignore verb failures and continue
  [-O | --no-override]  Ignore overrides in scripts, environments, flags etc.
  [-R | --regenerate]   Force generate '.env' and 'generated/'

Options:                { NEVER | auto (default) | ALWAYS }
  [-d | --devices]      Attach devices listed in 'docker-compose.devices.yml'
  [-g | --logging]      Configure logging as specified in 'docker-compose.logging.yml'
  [-h | --hooks]        Run pre and post hook 'docker-compose.*.yml' scripts
  [-l | --labels]       Use labels specified in 'docker-compose.labels.yml'
  [-p | --ports]        Expose ports listed in 'docker-compose.ports.yml'

     NEVER = Never configure the option (and use docker default instead)
      auto = Configure the option unless overridden in options.*.conf
    ALWAYS = Always configure the option as per 'docker-compose.*.{sh,yml}' files

Compositions:
  airdcpp   certbot      docker.sock  gitea      hass       indexarr
  influxdb  kodi.db      monitarr     navidrome  nextcloud  nocodb
  pihole    qbittorrent  tang         telegraf   traefik    unifi

Arch. Support

╭──   Compositions     \     Supported Archs   ───				amd64	386	arm/v6	arm/v7	arm64	ppc64le
AirDC++ :443/airdcpp	A B	openvpn‑client airdcpp	2.6.8 2.12.1	✔️	✔️	✔️ ✖️	✔️	✔️	✔️ ✖️
Certbot :80	A	certbot	2.9.0	✔️	✔️	✔️	✔️	✔️	✔️
docker.sock -	A	docker.sock‑proxy	2.6.14	✔️	✔️	✔️	✔️	✔️	✔️
Gitea :443/gitea	B	gitea	1.21.11-r...s	✔️	✖️	✖️	✖️	✔️	✖️
HAss :443/	A	hass	2024.4.4_1.34.0	✔️	✔️	✔️	✔️	✔️	✖️
Indexarr :443/jackett	A C	openvpn‑client solvarr jackett	2.6.8 v3.3.17 0.21.2042	✔️	✔️ ✔️ ✖️	✔️ ✖️ ✖️	✔️	✔️	✔️ ✖️ ✖️
InfluxDB :8086	B	influxdb	2.7.6-a...e	✔️	✖️	✖️	✖️	✔️	✖️
Kodi.DB :3306	B	mariadb	11.3.2	✔️	✖️	✖️	✖️	✔️	✔️
Monitarr :443/{lid,rad,son}arr	C C C	lidarr radarr sonarr	2.2.5.4141 5.4.6.8723 4.0.4.1515	✔️	✖️ ✖️ ✔️	✖️ ✖️ ✔️	✔️	✔️	✖️ ✖️ ✖️
Navidrome :443/navidrome	B	navidrome	0.51.0	✔️	✔️	✔️	✔️	✔️	✖️
Nextcloud :443/nextcloud	B B B B B	mariadb redis imaginary nextcloud cron push	10.11.6 7.2.4 20240424...t 29.0.0...0 29.0.0...0 29.0.0...0	✔️	✖️ ✔️ ✖️ ✔️ ✔️ ✔️	✖️ ✔️ ✖️ ✔️ ✔️ ✔️	✖️ ✔️ ✖️ ✔️ ✔️ ✔️	✔️	✔️ ✔️ ✖️ ✖️ ✖️ ✖️
NocoDB :443/nocodb	B B	redis nocodb	7.2.4 0.204.9	✔️	✔️ ✖️	✔️ ✖️	✔️	✔️	✔️ ✖️
Pi‑hole :53, :9443/pihole	A B	dnscrypt‑proxy pihole	2.1.5 2024.03.2	✔️	✔️	✔️	✔️	✔️	✔️
qBittorrent :5432, :443/qbittorrent	A A	openvpn‑client qbittorrent‑nox	2.6.8 4.6.4	✔️	✔️	✔️	✔️	✔️	✔️ ✖️
Tang :9080/tang	A	tang	git.05...2a	✔️	✔️	✔️	✔️	✔️	✔️
Telegraf -	B	telegraf	1.30.2	✔️	✖️	✖️	✔️	✔️	✖️
Traefik :443, :9080, :9443	B	traefik	2.11.2	✔️	✖️	✔️	✖️	✔️	✖️
Unifi :3478, :8080, :8843, :10001	B X	mongodb unifi	4.4.18 8.1.113	✔️	✖️ ✖️	✖️ ✖️	✖️ ✔️	✔️	✖️ ✖️

Legend

A	Dockerfiles with application binaries from OS repo / compiled from sources
B	Official open-source Dockerfiles / binaries from authors
C	Dockerfiles / binaries from third parties who publish full source code
X	Dockerfiles containing closed-source binaries (blobs)

Structure & Conventions

When deploying, all changes MUST appear in .gitignored files:

• at the repo root:

  • a static.global.override.env file may store fixed global environment variables, such as WAN FQDN, ACME configs etc.
  • a dynamic.global.override.env.sh script may generate additional server-specific dynamic global evironment variables such as public IP, UID of calling user etc.

• within each composition:

  • a dynamic.override.env.sh script may generate additional service-specific evironment variables
  • docker-compose.override.{yml|yaml} file may contain the usual override stuff for docker compose
  • docker-compose.{up,down,clean}.{pre,post}_hook.override*.sh scripts may contain additional hooks to be run

Subdirectories

• subdirectories under config/, data/, extra/, and generated/ must match the service names within docker-compose.yml
• the directory structure at each of config/X/Y..., data/X/Y..., extra/X/Y..., and generated/X/Y... must match the root directory hierarchy /Y/... in the target container X

	Mount mode	Git commit	Comments
config/	:ro or -	✔️
data/	:rw	✖️
env/	-	✔️	May contain env_files
extra/	-	✔️	Only indirect access, e.g. via generated/
generated/	:ro	✖️