-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Poetry install succeeds on macOS but fails in Docker image with error "*.whl not found in known hashes" #7099
Comments
|
I'm unable to immediately reproduce this with the provided pyproject.toml (in a fresh project) or the provided repo, in the My immediate suspicion is corrupted artifacts in a build cache, as your hashes in your lock file match my results locally as well as PyPI. Can you try invalidating all build caching and install again? |
I just noticed the following in your Dockerfile, which I did not use: RUN pip install poetry
ADD pyproject.toml poetry.lock /code/
RUN poetry config virtualenvs.create false && poetry config installer.max-workers 10
RUN poetry install --no-root While I doubt this is the cause of the issues you're having (I see no way this can cause a hash mismatch), it is strongly suggested to not use I'd strongly suggest using a virtual environment for both Poetry and your project (or use one to contain your project at a bare minimum), see #6398. |
Thanks for your prompt response Bjorn.
How can I invalidate the cache in Dockerfile? I've checked (one line before
`poetry install`) that `/root/.cache/pypoetry` is empty
…On Sat, 26 Nov 2022, 3:40 pm Bjorn Neergaard, ***@***.***> wrote:
I just noticed the following in your Dockerfile, which I did not use:
RUN pip install poetryADD pyproject.toml poetry.lock /code/RUN poetry config virtualenvs.create false && poetry config installer.max-workers 10RUN poetry install --no-root
While I doubt this is the cause of the issues you're having (I see no way
this can cause a hash mismatch), it is strongly suggested to *not* use virtualenvs.create
false in container. This instance is worse/straight up going to cause
problems, as since you have installed Poetry in a non-isolated fashion,
Poetry is installing your project into its own environment and will happily
remove its own dependencies from out under itself (or a poetry install
--sync would just straight up uninstall Poetry).
I'd strongly suggest using a virtual environment for both Poetry and your
project (or use one to contain your project at a bare minimum), see #6398
<#6398>.
—
Reply to this email directly, view it on GitHub
<#7099 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALX2WFUSAK7YXH5Q5MZOLTTWKGIEPANCNFSM6AAAAAASLYVZQQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Thanks @neersighted. I've tried the following but it still fails with the same error.
I've pushed these changes to the repo https://github.com/davidtan-tw/poetry-spike. I'll continue exploring what is the root cause of this failure.. |
To simplify the debugging process, I reduced the number of packages in Failure scenario 1: Just installing torch="^1.13.0"In this scenario, The fix was to install Failure scenario 2: Installing torch="^1.12.1" works but adding
|
The hashes that poetry is reporting as "not known" are indeed not values that should be found. Correct hashes for files are available on pypi eg https://pypi.org/project/llvmlite/#copy-hash-modal-63efe853-6770-4515-ad6c-9d40e703b520 Your docker file builds fine for me (every time) That is, it looks as though:
You should investigate whether the distributions that you are obtaining are in fact the same as the distributions that pypi is providing. Presumably the answer is going to be "no they're not", but I don't know what to suggest it is that is different about your environment to make this so. Perhaps you have some sort of proxy somewhere that is interfering?? (You can find the archives that poetry has obtained in the poetry cache.) |
Thanks for that @dimbleby. That gives me more insight on what's going on, though I'm still trying to find the root cause. I'll try to find another machine to see if I can reproduce this error, but in the meantime I thought I'd jot down my observations when
This is not a question necessarily - you folks have already done plenty to help me understand this error that I'm getting. Just jotting some observations before I forget. Will post more updates later when I get access to another Mac |
the 'other' sha that poetry is logging is the calculated value for the file it is trying to install (you can get this value yourself with ie poetry is telling you that the file that it is trying to install is not the file that pypi is distributing (which is a problem, which is why it stops) as for why the file has the wrong checksum only in a docker build - and only on your machine - well, over to you... |
I see. After some further debugging, I think finally found the root cause: insufficient disk space for the Docker runtime. Even though the error doesn't say anything about disk space, the error's gone away after I made some more space available for the Docker daemon:
And Thanks so much for your patience and assistance @dimbleby and @neersighted. |
I wonder if we're silently swallowing a rejected write? @davidtan-tw, feel free to open a new bug if you can come up with a reliable reproduction (e.g. use a VM with 4gb of disk) because I think we should check how many bytes were written as we write files to disk, and report a failure if we don't successfully write. |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Poetry version: 1.2.2
Python version: 3.10.6
OS version and name: Host (macOS 11.3), Docker container (python:3.10.6-slim, which is using Debian GNU/Linux 11 (bullseye)
pyproject.toml: https://github.com/davidtan-tw/poetry-spike/blob/main/pyproject.toml
I am on the latest stable Poetry version, installed using a recommended method.
I have searched the issues of this repo and believe that this is not a duplicate.
I have consulted the FAQ and blog for any relevant entries or release notes.
If an exception occurs when executing a command, I executed it again in debug mode (
-vvv
option) and have included the output below.Issue
Hi there, first of all, thank you for the great work in this awesome library. I'm really enjoying using Poetry.
The issue I've encountered is that
poetry install
fails when I include 2 dependencies (torch and bertopic), but only when I use Poetry in a Docker image/container.Steps to reproduce:
poetry install
): docker build -t poetry-spike:latest .poetry install
fail when an error like the following. (Instead ofllvmlite
, you might also seescipy
,numpy
,pydantic
,torch
. curiously it seems to fail randomly only on these 5 packages)Observations
Poetry install failures
poetry install
with or withoutpoetry.lock
results in the same failurePoetry install succeeds with the following workarounds:
poetry install
on the host (macOS) succeeds with no problempoetry install
in Docker succeeds when I remove 2 dependencies (torch
andbertopic
)Related issues, and why they don't seem to solve this problem
poetry install
The text was updated successfully, but these errors were encountered: