podman opens storage.conf in locations other than XDG_CONFIG_HOME

[lastephey]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Even after a user has set XDG_CONFIG_HOME, it looks like Podman still opens the storage.conf file located in more "standard" locations like /etc/containers/storage.conf. This is an issue for us at NERSC as we try to run Podman at very large scale, and we want make sure Podman only opens the configuration file we have copied into local memory and have specified with XDG_CONFIG_HOME.

Steps to reproduce the issue:

export XDG_CONFIG_HOME=$HOME/.config
strace -e trace=file -f -o /tmp/podman-test.out podman run --rm docker.io/continuumio/miniconda3 python -c "import os"

Part of the strace output file:

39857 newfstatat(AT_FDCWD, "/etc/containers/storage.conf", {st_mode=S_IFREG|0644, st_size=5046, ...}, 0) = 0
39857 newfstatat(AT_FDCWD, "/etc/containers/storage.conf", {st_mode=S_IFREG|0644, st_size=5046, ...}, 0) = 0
39857 openat(AT_FDCWD, "/etc/containers/storage.conf", O_RDONLY|O_CLOEXEC) = 8
39857 newfstatat(AT_FDCWD, "/run/user/75313", {st_mode=S_IFDIR|0700, st_size=180, ...}, 0) = 0
39857 mkdirat(AT_FDCWD, "/run/user/75313/libpod", 01700) = -1 EEXIST (File exists)
39857 fchmodat(AT_FDCWD, "/run/user/75313/libpod", 01700) = 0
39857 newfstatat(AT_FDCWD, "/run/user/75313/containers", {st_mode=S_IFDIR|0700, st_size=140, ...}, 0) = 0
39857 newfstatat(AT_FDCWD, "/global/common/shared/das/podman/bin/fuse-overlayfs", {st_mode=S_IFREG|0775, st_size=85704, ...}, 0) = 0
39857 newfstatat(AT_FDCWD, "/global/homes/s/stephey/.config/containers/storage.conf", {st_mode=S_IFREG|0660, st_size=490, ...}, 0) = 0
39857 newfstatat(AT_FDCWD, "/global/homes/s/stephey/.config/containers/storage.conf", {st_mode=S_IFREG|0660, st_size=490, ...}, 0) = 0
39857 openat(AT_FDCWD, "/global/homes/s/stephey/.config/containers/storage.conf", O_RDONLY|O_CLOEXEC) = 8

It looks like Podman is still opening /etc/containers/storage.conf, even though we have set export XDG_CONFIG_HOME=$HOME/.config.

Our hope is that when a user sets XDG_CONFIG_HOME, Podman will first attempt to open config files in this location. Based on the comments at the top of storage.conf, it does seem like it should start with XDG_CONFIG_HOME. If this fails, Podman could then attempt to open other configuration files in order of precedence.

Output of podman version:

stephey@muller:login01:/tmp> podman --version
podman version 4.0.0-dev

Output of podman info:

host:
  arch: amd64
  buildahVersion: 1.24.0-dev
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.30-150300.8.3.1.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: unknown'
  cpus: 256
  distribution:
    distribution: '"sles"'
    version: "15.3"
  eventLogger: file
  hostname: login01
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 75313
      size: 1
    uidmap:
    - container_id: 0
      host_id: 75313
      size: 1
  kernel: 5.3.18-150300.59.87_11.0.78-cray_shasta_c
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 108847702016
  memTotal: 540249968640
  networkBackend: cni
  ociRuntime:
    name: crun
    package: Unknown
    path: /global/common/shared/das/podman/bin/crun
    version: |-
      crun version 1.4.1.11-38e1b
      commit: 38e1b5e2a3e9567ff188258b435085e329aaba42
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/75313/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: unconfined
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-0.4.7-3.15.1.x86_64
    version: |-
      slirp4netns version 0.4.7
      commit: unknown
      libslirp: 4.3.1-git
  swapFree: 6318092288
  swapTotal: 9999994880
  uptime: 9h 49m 20.29s (Approximately 0.38 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.opensuse.org
  - docker.io
 store:
  configFile: /global/u1/s/stephey/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.ignore_chown_errors: "true"
    overlay.mount_program:
      Executable: /global/common/shared/das/podman/bin/fuse-overlayfs-wrap
      Package: Unknown
      Version: |-
        fusermount3 version: 3.6.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.6.1
        using FUSE kernel interface version 7.29
    overlay.squashmount: "true"
  graphRoot: /tmp/stephey/storage
  graphStatus:
    Backing Filesystem: tmpfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /tmp
  imageStore:
    number: 2
  runRoot: /tmp/stephey
  volumePath: /tmp/stephey/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1643325567
  BuiltTime: Thu Jan 27 15:19:27 2022
  GitCommit: ecf818a1cddd5e3e62b4a2a761d44b2be336317e-dirty
  GoVersion: go1.16.12
  OsArch: linux/amd64
  Version: 4.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

We have a custom installation and can definitely provide more info if needed.

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

No we have not upgraded yet. Please let us know if you'd like us to upgrade.

Thank you all very much,
Laurie @scanon @danfulton

[Luap99]

Can you test with the latest podman version.

[vrothberg]

Can you test with the latest podman version.

It may work for storage.conf but registries.conf and others won't, see containers/image#1647. Changing the behavior is certainly doable but we need to be extremely careful to not break users.

[vrothberg]

@rhatdan tackles storage.conf in containers/storage#1357

[vrothberg]

Reopening as containers/storage#1357 addressed storage.conf only.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[lastephey]

Ah wonderful, thank you!