Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New mtime behavior of COPY in 0.6.9 breaks existing mtime-sensitive builds #1712

Closed
rrjjvv opened this issue Mar 3, 2022 · 0 comments · Fixed by #1716
Closed

New mtime behavior of COPY in 0.6.9 breaks existing mtime-sensitive builds #1712

rrjjvv opened this issue Mar 3, 2022 · 0 comments · Fixed by #1716

Comments

@rrjjvv
Copy link
Contributor

rrjjvv commented Mar 3, 2022

Simplest reproduction:

VERSION 0.6
FROM python:3-alpine

verify:
    LOCALLY
    RUN stat -c '%n %y' Earthfile

broke:
    BUILD +verify
    COPY Earthfile .
    RUN stat -c '%n %y' Earthfile && \
        python3 -m zipfile -c foo.zip Earthfile

Behavior in 0.6.8:

$ earthly-0.6.8 -v +broke 
earthly-0.6.8 version v0.6.8 2e49b7dd191a729f6b74286434044af0922e5e2a linux/amd64; Debian testing
$ earthly-0.6.8 --no-cache +broke 
<snip>
     python:3-alpine | --> Load metadata linux/amd64
             +verify *local* | --> RUN stat -c '%n %y' Earthfile
             +verify *local* | Earthfile 2022-03-02 22:20:27.492646623 -0700
               +base | --> FROM python:3-alpine
             context | --> local context .
               +base | [██████████] 100% resolve docker.io/library/python:3-alpine@sha256:a9865ba6472324621e81e1da5cbd02069d528215a4b49d49695eac693c10488a
             ongoing | context (5 seconds ago)
             context | [          ]   0% transferring .:
             context | [██████████] 100% transferring .:
              +broke | --> COPY Earthfile .
              +broke | --> RUN stat -c '%n %y' Earthfile && python3 -m zipfile -c foo.zip Earthfile
              +broke | Earthfile 2020-04-16 12:00:00.000000000 +0000
              output | --> exporting outputs
<snip>
====================== 🌍 Earthly Build  ✅ SUCCESS ==========================

Behavior in 0.6.9:

$ earthly -v
earthly version v0.6.9 92cb87bbd28f806aeb9ba300d22e4bdd38dba18c linux/amd64; Debian testing
$ earthly --no-cache +broke 
<snip>
     python:3-alpine | --> Load metadata linux/amd64
             +verify *local* | --> RUN stat -c '%n %y' Earthfile
             +verify *local* | Earthfile 2022-03-02 22:20:27.492646623 -0700
             context | --> local context .
               +base | --> FROM python:3-alpine
               +base | [██████████] 100% resolve docker.io/library/python:3-alpine@sha256:a9865ba6472324621e81e1da5cbd02069d528215a4b49d49695eac693c10488a
             ongoing | context (5 seconds ago)
             context | [          ]   0% transferring .:
             context | [██████████] 100% transferring .:
              +broke | --> COPY Earthfile .
              +broke | --> RUN stat -c '%n %y' Earthfile && python3 -m zipfile -c foo.zip Earthfile
              +broke | Earthfile 1901-12-13 20:45:52.000000000 +0000
              +broke | Traceback (most recent call last):
              +broke |   File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
              +broke |     return _run_code(code, main_globals, None,
              +broke |   File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
              +broke |     exec(code, run_globals)
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 2463, in <module>
              +broke |     main()
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 2459, in main
              +broke |     addToZip(zf, path, zippath)
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 2443, in addToZip
              +broke |     zf.write(path, zippath, ZIP_DEFLATED)
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 1728, in write
              +broke |     zinfo = ZipInfo.from_file(filename, arcname,
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 518, in from_file
              +broke |     zinfo = cls(arcname, date_time)
              +broke |   File "/usr/local/lib/python3.10/zipfile.py", line 362, in __init__
              +broke |     raise ValueError('ZIP does not support timestamps before 1980')
              +broke | ValueError: ZIP does not support timestamps before 1980
              +broke | Command /bin/sh -c 'stat -c '"'"'%n %y'"'"' Earthfile && python3 -m zipfile -c foo.zip Earthfile' failed with exit code 1
              +broke | 
              +broke | ERROR: Command exited with non-zero code: RUN stat -c '%n %y' Earthfile && python3 -m zipfile -c foo.zip Earthfile
<snip>

I'm guessing this is a result of #1705. The docs don't make any guarantees about what static timestamp is used (and until tonight, it never occurred to me that it wasn't the actual on-disk timestamp), but it breaks a number of my builds. (I wouldn't normally use python for creating zip files, but my actual scenario is building python packages, which utilize that library.)

I did verify that adding --keep-ts is a workaround (though 'workaround' may be a matter of perspective here). I'm guessing that the on-disk timestamps aren't used by default because they degrade performance and/or factor into caching?

Obviously the hope is that COPY will always "just work" as it did before, whether by restoring the previous behavior, picking a different arbitrary date (like epoch + 1), or making it configurable (either a static config-driven timestamp value, or an option allowing --keep-ts to be the default).

alexcb added a commit that referenced this issue Mar 3, 2022
This is to prevent #1712 from reoccurring.

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
@alexcb alexcb mentioned this issue Mar 3, 2022
alexcb added a commit that referenced this issue Mar 3, 2022
This is to prevent #1712 from reoccurring.

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant