add a HOSTS command to add entries to /etc/hosts

[alexcb]

It's not trivial to modify /etc/hosts; it's currently set as read-only:

FROM alpine

addhost:
    RUN cat /etc/hosts
    RUN echo "myhost 127.0.0.1" >> /etc/hosts
    RUN cat /etc/hosts

results in:

$ earthly +addhost 
           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
              alpine | --> Load metadata linux/amd64
               +base | --> FROM alpine
               +base | [██████████] resolve docker.io/library/alpine@sha256:eb3e4e175ba6d212ba1d6e04fc0782916c08e1c9d7b45892e9796141b1d379ae ... 100%
            +addhost | --> RUN cat /etc/hosts
            +addhost | 127.0.0.1	localhost buildkitsandbox
            +addhost | ::1	localhost ip6-localhost ip6-loopback
            +addhost | --> RUN echo "myhost 127.0.0.1" >> /etc/hosts
            +addhost | /bin/sh: can't create /etc/hosts: Read-only file system
            +addhost | Command /bin/sh -c 'echo "myhost 127.0.0.1" >> /etc/hosts' failed with exit code 1
            +addhost | 
            +addhost | WARN: (RUN echo "myhost 127.0.0.1" >> /etc/hosts) process "/bin/sh -c PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /usr/bin/earth_debugger /bin/sh -c 'echo \"myhost 127.0.0.1\" >> /etc/hosts'" did not complete successfully: exit code: 1
Error: build target: build main: bkClient.Build: failed to solve: process "/bin/sh -c PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /usr/bin/earth_debugger /bin/sh -c 'echo \"myhost 127.0.0.1\" >> /etc/hosts'" did not complete successfully: exit code: 1

The current workaround is to copy /etc/hosts elsewhere, then unmount the read-only version, then copy it back and then modify it:

FROM alpine

addhost:
    RUN --privileged cp /etc/hosts /tmp/hosts && \
        umount /etc/hosts && \
        cp /tmp/hosts /etc/hosts && \
        echo "127.0.0.1 myhost" >> /etc/hosts && \
        cat /etc/hosts && \
        ping -c 1 myhost

pinghostsecondtime:
    FROM +addhost
    RUN cat /etc/hosts && \
        ping -c 1 myhost

However, this must be done in each RUN command. calling earthly +pinghostsecondtime fails as the /etc/hosts file change does not persist.

furthermore, this work around requires priviledged permissions.

This proposal is to see if we can somehow maintain a working list of hosts that will persist between RUNs. It might however require that we run earthly with -P.

This idea came up in:

Originally posted by @dchw in #1154 (comment)

[vladaionescu]

Another question is whether the updated hosts should persist within the resulting image or not.

Precendents in Docker: docker build --add-host option and also docker run --add-host option.

[diegobernardes]

I would love to see this feature too. During my integration tests, I'm trying to use kubefwd to access some services but I'm getting the same error ERRO[12:29:10] Error saving hosts fileopen /etc/hosts: read-only file system.

[wataruian]

Would love to see this too. This will solve the problems in my use case. Thanks!

[cartermckinnon]

I need something like this to support integration tests using HBase.

HBase servers register their hostnames to ZooKeeper, and clients connect to ZooKeeper to look up an HBase server. Therefore, the hostname that HBase registers to ZooKeeper must be addressable by the client, which gets complicated when HBase is running in a container and the client isn't. The easiest workaround I've found for this is to add a loopback entry for HBase to /etc/hosts, like 127.0.0.1 hbase.

I would prefer to bake the /etc/hosts entry into the image, but it's not the end of the world for me if it's a runtime thing.

[alexcb]

Here's a more concrete example of the work-around:

FROM alpine

workaround:
    RUN echo "#!/bin/sh
set -e
cp /etc/hosts /tmp/hosts && \
umount /etc/hosts && \
cp /tmp/hosts /etc/hosts && \
echo \"127.0.0.1 myhost\" >> /etc/hosts" > /bin/hosts-hack && chmod +x /bin/hosts-hack

    RUN --privileged /bin/hosts-hack && ping myhost

Note that each RUN command must call /bin/hosts-hack and that --privileged is always required.

( we currently use this work-around in our own earthly-in-earthly tests: https://github.com/earthly/earthly/blob/main/tests/git-webserver/Earthfile#L85-L90 )