Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

twine: command not found #115

Closed
james-rms opened this issue Dec 7, 2022 · 11 comments
Closed

twine: command not found #115

james-rms opened this issue Dec 7, 2022 · 11 comments
Labels
bug Something isn't working

Comments

@james-rms
Copy link

james-rms commented Dec 7, 2022

Filing separately from #112 , since the $PATH issues outlined there seem separate from what I'm experiencing.

Version: e71a4a4

Example failure:
https://github.com/foxglove/mcap/actions/runs/3634377846/jobs/6132519798#step:11:20

with:
    user: __token__
    password: ***
    packages_dir: python/mcap/dist
    repository_url: https://test.pypi.org/legacy/
    skip_existing: true
    verify_metadata: true
    verbose: false
    print_hash: false
  env:
    pythonLocation: /opt/hostedtoolcache/Python/[3](https://github.com/foxglove/mcap/actions/runs/3634377846/jobs/6132519798#step:11:3).7.15/x6[4](https://github.com/foxglove/mcap/actions/runs/3634377846/jobs/6132519798#step:11:4)
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.7.1[5](https://github.com/foxglove/mcap/actions/runs/3634377846/jobs/6132519798#step:11:5)/x[6](https://github.com/foxglove/mcap/actions/runs/3634377846/jobs/6132519798#step:11:6)4/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.7.15/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.7.15/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.7.15/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.7.15/x64/lib
/usr/bin/docker run --name ac8f0bed99144e79bbebae175f60826_b0a163 --label 290506 --workdir /github/workspace --rm -e "pythonLocation" -e "PKG_CONFIG_PATH" -e "Python_ROOT_DIR" -e "Python2_ROOT_DIR" -e "Python3_ROOT_DIR" -e "LD_LIBRARY_PATH" -e "INPUT_USER" -e "INPUT_PASSWORD" -e "INPUT_PACKAGES_DIR" -e "INPUT_REPOSITORY_URL" -e "INPUT_SKIP_EXISTING" -e "INPUT_VERIFY_METADATA" -e "INPUT_VERBOSE" -e "INPUT_PRINT_HASH" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/mcap/mcap":"/github/workspace" 290506:6ac8f0bed99144e79bbebae175f60826  "__token__" "***" "https://test.pypi.org/legacy/" "python/mcap/dist" "true" "true" "false" "false"
/app/twine-upload.sh: line 50: twine: command not found

Action configuration:
https://github.com/foxglove/mcap/blob/0b5fcf170aa8cb265a38fa2ffc03cb9b53fc131c/.github/workflows/ci.yml#L256:L278

  python:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: python
    steps:
      - uses: actions/checkout@v3
        with:
          lfs: true
      - uses: actions/setup-python@v4
        with:
          python-version: 3.7
      - uses: satackey/action-docker-layer-caching@v0.0.11
        continue-on-error: true
      - run: pip install pipenv==2022.7.24
      - run: make lint
      - run: make test
      - run: make examples
      - run: make build
      - name: Publish mcap to TestPyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        if: ${{ !github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' }}
        with:
          user: __token__
          password: ${{ secrets.TESTPYPI_API_TOKEN }}
          packages_dir: python/mcap/dist
          repository_url: https://test.pypi.org/legacy/
          skip_existing: true
@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

That is super weird. The smoke test that we now have in GHA shows that a broken PATH does not influence this. I wonder if you could try to shed some light by adding

env:
   DEBUG: true

to the corresponding step.

It could also be useful to verify that this does not happen if the action is invoked in a dedicated job that only downloads the artifacts before calling it.

@woodruffw
Copy link
Member

I'm also seeing this error:

Run pypa/gh-action-pypi-publish@v1.6.3
/usr/bin/docker run --name def1238cfe3e4753b6b01329ccdbdf6c_b41197 --label 290506 --workdir /github/workspace --rm -e "pythonLocation" -e "PKG_CONFIG_PATH" -e "Python_ROOT_DIR" -e "Python2_ROOT_DIR" -e "Python3_ROOT_DIR" -e "LD_LIBRARY_PATH" -e "INPUT_USER" -e "INPUT_PASSWORD" -e "INPUT_REPOSITORY_URL" -e "INPUT_PACKAGES_DIR" -e "INPUT_VERIFY_METADATA" -e "INPUT_SKIP_EXISTING" -e "INPUT_VERBOSE" -e "INPUT_PRINT_HASH" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_ID_TOKEN_REQUEST_URL" -e "ACTIONS_ID_TOKEN_REQUEST_TOKEN" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/abi3info/abi3info":"/github/workspace" 290506:def1238cfe3e4753b6b01329ccdbdf6c  "__token__" "***" "" "dist" "true" "false" "false" "false"
/app/twine-upload.sh: line 50: twine: command not found

Logs: https://github.com/woodruffw/abi3info/actions/runs/3634987726/jobs/6133656272

@suizhihao
Copy link

/app/twine-upload.sh: line 50: twine: command not found

@suizhihao
Copy link

@woodruffw
Copy link
Member

It could also be useful to verify that this does not happen if the action is invoked in a dedicated job that only downloads the artifacts before calling it.

My action uses the artifacts from a previous step, so it shouldn't be that:

    - name: deps
      run: python -m pip install -U setuptools build wheel

    - name: build
      run: python -m build

    - name: publish
      uses: pypa/gh-action-pypi-publish@v1.6.3
      with:
        user: __token__
        password: ${{ secrets.PYPI_TOKEN }}

    - name: sign
      uses: sigstore/gh-action-sigstore-python@v0.1.0
      with:
        inputs: ./dist/*.tar.gz ./dist/*.whl
        release-signing-artifacts: true

I'll try a debug run in a moment.

@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

The smoke test that we now have in GHA shows that a broken PATH does not influence this.

I was wrong: it does fail — https://github.com/pypa/gh-action-pypi-publish/actions/runs/3634959022.

@woodruffw
Copy link
Member

I'll try a debug run in a moment.

Logs: https://github.com/woodruffw/abi3info/actions/runs/3635033169/jobs/6133748800

In particular:

Run pypa/gh-action-pypi-publish@v1.6.3
/usr/bin/docker run --name a356773daf846d783af59338dc33a74_ef732c --label 290506 --workdir /github/workspace --rm -e "pythonLocation" -e "PKG_CONFIG_PATH" -e "Python_ROOT_DIR" -e "Python2_ROOT_DIR" -e "Python3_ROOT_DIR" -e "LD_LIBRARY_PATH" -e "DEBUG" -e "INPUT_USER" -e "INPUT_PASSWORD" -e "INPUT_REPOSITORY_URL" -e "INPUT_PACKAGES_DIR" -e "INPUT_VERIFY_METADATA" -e "INPUT_SKIP_EXISTING" -e "INPUT_VERBOSE" -e "INPUT_PRINT_HASH" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_ID_TOKEN_REQUEST_URL" -e "ACTIONS_ID_TOKEN_REQUEST_TOKEN" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/abi3info/abi3info":"/github/workspace" 290506:6a356773daf846d783af59338dc33a74  "__token__" "***" "" "dist" "true" "false" "false" "false"
+ set -Eeuo pipefail
+ export PATH=/usr/bin:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/usr/bin:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ . /etc/profile
+++ id -u
++ '[' 0 -eq 0 ']'
++ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ export PATH
++ '[' '' ']'
++ '[' -d /etc/profile.d ']'
++ for i in /etc/profile.d/*.sh
++ '[' -r '/etc/profile.d/*.sh' ']'
++ unset i
++ python -m site --user-base
+ export PATH=/github/home/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/github/home/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ python -m site --user-site
+ export PYTHONPATH=/github/home/.local/lib/python3.11/site-packages:/root/.local/lib/python3.11/site-packages
+ PYTHONPATH=/github/home/.local/lib/python3.11/site-packages:/root/.local/lib/python3.11/site-packages
+ [[ __token__ == \_\_\t\o\k\e\n\_\_ ]]
+ [[ ! *** =~ ^pypi- ]]
+ ls -A dist/abi3info-2022.12.6.post1.tar.gz
+ [[ true != \f\a\l\s\e ]]
+ twine check dist/abi3info-2022.12.6.post1-py3-none-any.whl dist/abi3info-2022.12.6.post1.tar.gz
/app/twine-upload.sh: line 50: twine: command not found

@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

🤦‍♂️ looks like it's the same problem but with $HOME (python -m site ... invocations probably get the var passed down from the runner).

@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

It's reproducible locally via podman run --rm -v "$(pwd):/app" -e INPUT_REPOSITORY_URL= -e INPUT_PASSWORD= -e INPUT_PRINT_HASH= -e INPUT_VERBOSE= -e INPUT_SKIP_EXISTING= -e INPUT_USER= -e INPUT_PACKAGES_DIR=dist/ -e INPUT_VERIFY_METADATA= -e PATH=utter-nonsense -e DEBUG=true -e HOME= localhost/gh-action-pypi-publish:latest

@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2022

Alright... the CI passes with the hotfix: https://github.com/pypa/gh-action-pypi-publish/actions/runs/3635075803.
I'll plan to replace the user-global installs with a properly managed venv next time I'll get to the feature work here. That should improve the situation in general.

@webknjaz webknjaz added the bug Something isn't working label Dec 7, 2022
@james-rms
Copy link
Author

Thanks! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants