Pomerium arm64 docker images don't listen on port 443

[therealmik]

What happened?

Pomerium wasn't listening on port 443

What did you expect to happen?

Pomerium should've been listening on port 443 and proxying traffic (I think?)

How'd it happen?

Ran pomerium/pomerium:arm64v8-v0.10.6 docker image

Other recent images had similar/same problem. Building myself had a glibc version conflict with the envoy binary.

What's your environment like?

Docker-compose file:

version: '3'
services:
  pomerium:
    container_name: pomerium
    image: pomerium/pomerium:arm64v8-v0.10.6
    restart: always
    volumes:
      - /etc/pomerium-config.yaml:/pomerium/config.yaml:ro
      - /var/pomerium:/data:rw
    network_mode: host

What's your config.yaml?

# See detailed configuration settings : https://www.pomerium.io/docs/reference/reference/

address: :443

# this is the domain the identity provider will callback after a user authenticates
authenticate_service_url: https://login.XXXX

# certificate settings:  https://www.pomerium.io/docs/reference/certificates.html
autocert: true
autocert_dir: /data/autocert

# REMOVE FOR PRODUCTION
autocert_use_staging: true

# identity provider settings : https://www.pomerium.io/docs/identity-providers.html
idp_provider: google
idp_client_id: XXXX
idp_client_secret: XXXX

# Generate 256 bit random keys  e.g. `head -c32 /dev/urandom | base64`
cookie_secret: XXXX
shared_secret: XXXX

services: all

# https://www.pomerium.io/configuration/#policy
policy:
  - from: https://XXXX
    to: https://XXXX
    allowed_users:
      - XXXX
  - from: XXXX
    to: XXXX
    allowed_users:
      - XXXX

What did you see in the logs?

{"level":"warn","error":"rpc error: code = Canceled desc = latest balancer error: last connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:5443: connect: connection refused\"","time":"2020-12-01T22:53:39Z","message":"sync error"}
{"level":"fatal","error":"envoy exited: fork/exec /tmp/pomerium-embedded-files/envoy: no such file or directory","time":"2020-12-01T22:53:39Z","message":"cmd/pomerium"}

Additional context

I checked the overlayfs and the file /tmp/pomerium-embedded-files/envoy is there and executable from outside of the container.

On the other hand:

# docker exec -it pomerium /tmp/pomerium-embedded-files/envoy
OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "no such file or directory": unknown

[travisgroth]

Hi @therealmik,

I believe this is an issue with glibc support in the alpine image we're based on. I can't quite reproduce that exact error message but there's definitely an issue executing envoy:

{"service":"envoy","name":"envoy","time":"2020-12-02T04:28:16Z","message":"/lib/ld-linux-aarch64.so.1: No such file or directory"}

We're revamping our arm image building presently and should have a fix for this problem included in #1643. Would you mind trying image travisgroth/pomerium:arm64v8-v0.11.0-rc2-23-g48a1e779? It is built from that PR.

[therealmik]

Yep, that docker image works!

[travisgroth]

Great! Thanks @therealmik.

[travisgroth]

FYI, we've republished the pomerium/pomerium:arm64v8-v0.10.6 docker image as well. This should be fixed going forward.