Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip.parse crashes with StopIteration if platform restriction prevents package from install #1868

Closed
sin-ack opened this issue Apr 25, 2024 · 5 comments 路 Fixed by #1885
Closed
Assignees

Comments

@sin-ack
Copy link

sin-ack commented Apr 25, 2024

馃悶 bug report

Affected Rule

The issue is caused by the pip.parse extension.

Is this a regression?

No.

Description

If the given requirement has a platform restriction like platform_system == "Windows" and pip refuses to download the requirements, then Bazel will crash.

馃敩 Minimal Reproduction

https://github.com/sin-ack/rules_python-repro

馃敟 Exception or Error



INFO: Repository rules_python~~pip~pip_311_colorama instantiated at:
  : in 
Repository rule whl_library defined at:
  /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/pip_repository.bzl:932:30: in 
ERROR: An error occurred during the fetch of repository 'rules_python~~pip~pip_311_colorama':
   Traceback (most recent call last):
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/pip_repository.bzl", line 763, column 31, in _whl_library_impl
		repo_utils.execute_checked(
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/private/repo_utils.bzl", line 145, column 29, in _execute_checked
		return _execute_internal(fail_on_error = True, *args, **kwargs)
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/private/repo_utils.bzl", line 86, column 13, in _execute_internal
		fail((
Error in fail: repo.execute: whl_library.ResolveRequirement(rules_python~~pip~pip_311_colorama, colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6): end: failure:
  command: /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/python -m python.pip_install.tools.wheel_installer.wheel_installer --requirement "colorama==0.4.6 ; python_version >= \"3.11\" and python_version < \"4.0\" and platform_system == \"Windows\"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6" --isolated --extra_pip_args "{\"arg\":[]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: 
  timeout: 600
  environment:
PYTHONPATH="/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__build:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__click:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__colorama:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__importlib_metadata:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__installer:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__more_itertools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__packaging:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pep517:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip_tools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pyproject_hooks:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__setuptools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__tomli:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__wheel:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isystem /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/include/python3.11"
===== stdout start =====
Ignoring colorama: markers 'python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"' don't match your environment
===== stdout end =====
===== stderr start =====
Traceback (most recent call last):
  File "", line 198, in _run_module_as_main
  File "", line 88, in _run_code
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in 
    main()
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
===== stderr end =====
ERROR: : fetching whl_library rule //:rules_python~~pip~pip_311_colorama: Traceback (most recent call last):
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/pip_repository.bzl", line 763, column 31, in _whl_library_impl
		repo_utils.execute_checked(
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/private/repo_utils.bzl", line 145, column 29, in _execute_checked
		return _execute_internal(fail_on_error = True, *args, **kwargs)
	File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/private/repo_utils.bzl", line 86, column 13, in _execute_internal
		fail((
Error in fail: repo.execute: whl_library.ResolveRequirement(rules_python~~pip~pip_311_colorama, colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6): end: failure:
  command: /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/python -m python.pip_install.tools.wheel_installer.wheel_installer --requirement "colorama==0.4.6 ; python_version >= \"3.11\" and python_version < \"4.0\" and platform_system == \"Windows\"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6" --isolated --extra_pip_args "{\"arg\":[]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: 
  timeout: 600
  environment:
PYTHONPATH="/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__build:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__click:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__colorama:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__importlib_metadata:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__installer:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__more_itertools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__packaging:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pep517:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip_tools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pyproject_hooks:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__setuptools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__tomli:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__wheel:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isystem /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/include/python3.11"
===== stdout start =====
Ignoring colorama: markers 'python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"' don't match your environment
===== stdout end =====
===== stderr start =====
Traceback (most recent call last):
  File "", line 198, in _run_module_as_main
  File "", line 88, in _run_code
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in 
    main()
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
===== stderr end =====
ERROR: no such package '@@rules_python~~pip~pip_311_colorama//': repo.execute: whl_library.ResolveRequirement(rules_python~~pip~pip_311_colorama, colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6): end: failure:
  command: /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/python -m python.pip_install.tools.wheel_installer.wheel_installer --requirement "colorama==0.4.6 ; python_version >= \"3.11\" and python_version < \"4.0\" and platform_system == \"Windows\"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6" --isolated --extra_pip_args "{\"arg\":[]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: 
  timeout: 600
  environment:
PYTHONPATH="/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__build:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__click:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__colorama:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__importlib_metadata:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__installer:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__more_itertools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__packaging:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pep517:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip_tools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pyproject_hooks:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__setuptools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__tomli:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__wheel:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isystem /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/include/python3.11"
===== stdout start =====
Ignoring colorama: markers 'python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"' don't match your environment
===== stdout end =====
===== stderr start =====
Traceback (most recent call last):
  File "", line 198, in _run_module_as_main
  File "", line 88, in _run_code
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in 
    main()
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
===== stderr end =====
ERROR: /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~pip~pip/colorama/BUILD.bazel:8:6: @@rules_python~~pip~pip//colorama:pkg depends on @@rules_python~~pip~pip_311_colorama//:pkg in repository @@rules_python~~pip~pip_311_colorama which failed to fetch. no such package '@@rules_python~~pip~pip_311_colorama//': repo.execute: whl_library.ResolveRequirement(rules_python~~pip~pip_311_colorama, colorama==0.4.6 ; python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6): end: failure:
  command: /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/python -m python.pip_install.tools.wheel_installer.wheel_installer --requirement "colorama==0.4.6 ; python_version >= \"3.11\" and python_version < \"4.0\" and platform_system == \"Windows\"     --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44     --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6" --isolated --extra_pip_args "{\"arg\":[]}" --pip_data_exclude "{\"arg\":[]}" --environment "{\"arg\":{}}"
  return code: 1
  working dir: 
  timeout: 600
  environment:
PYTHONPATH="/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__build:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__click:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__colorama:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__importlib_metadata:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__installer:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__more_itertools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__packaging:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pep517:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pip_tools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__pyproject_hooks:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__setuptools:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__tomli:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__wheel:/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~internal_deps~pypi__zipp"
CPPFLAGS="-isystem /home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~~python~python_3_11_host/include/python3.11"
===== stdout start =====
Ignoring colorama: markers 'python_version >= "3.11" and python_version < "4.0" and platform_system == "Windows"' don't match your environment
===== stdout end =====
===== stderr start =====
Traceback (most recent call last):
  File "", line 198, in _run_module_as_main
  File "", line 88, in _run_code
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in 
    main()
  File "/home/agni/.cache/bazel/_bazel_agni/ba6071a1d8c8e24a5f08c6b275a4cff8/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
===== stderr end =====
ERROR: Analysis of target '//:repro' failed; build aborted: Analysis failed
INFO: Elapsed time: 2.031s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully

馃實 Your Environment

Operating System:

  
Gentoo Linux 23.0
  

Output of bazel version:

  
Bazelisk version: development
Build label: 7.1.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Mar 21 18:08:37 2024 (1711044517)
Build timestamp: 1711044517
Build timestamp as int: 1711044517
  

Rules_python version:

  
0.31.0
  

Anything else relevant?

I'm using rules_python_poetry, which generates that verbose requirement line.

@sin-ack
Copy link
Author

sin-ack commented Apr 25, 2024

As a workaround, one can exclude that specific requirement from the py_library/py_binary's deps and rules_python will not attempt to install it.

@aignas
Copy link
Collaborator

aignas commented Apr 30, 2024

Given that colorama should be only included on Windows and you run this error on non Windows, I suspect that you are trying to cross-compile, or you are trying to use colorama without including it in your lock file, which should alter the conditional inclusion of the wheel from only on Windows to all platforms.

We at work use a similar lockfile without any issues with rules_python 0.31, so I am perplexed why this is not working in your case.

@vors
Copy link

vors commented May 1, 2024

We switched to platform-agnostic lockfile (meaning that you can use this same lockfile on many platforms, it has all the needed info and all markers).

Consider that we have such lines in it

cupy-cuda12x==13.0.0; sys_platform == "linux" and platform_machine == "x86_64"
cupy-cuda12x==13.1.0; sys_platform == "linux" and platform_machine == "aarch64"

we are running this on linux x86_64 and run into

Error in fail: repo.execute: whl_library.ResolveRequirement(rules_python~~pip~pypi_research_311_cupy_cuda12x, cupy-cuda12x==13.1.0; sys_platform == "linux" and platform_machine == "aarch64"): end: failure:
....
....
....
===== stdout start =====
Ignoring cupy-cuda12x: markers 'sys_platform == "linux" and platform_machine == "aarch64"' don't match your environment
===== stdout end =====
===== stderr start =====
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/.cache/bazel/_bazel_root/00e0182df830644af7af00c92693c660/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 205, in <module>
    main()
  File "/root/.cache/bazel/_bazel_root/00e0182df830644af7af00c92693c660/external/rules_python~/python/pip_install/tools/wheel_installer/wheel_installer.py", line 198, in main
    whl = Path(next(iter(glob.glob("*.whl"))))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

I think one way to fix it would be to add code along the lines of

result_lines = []
for line in lines:
        r = packaging.requirements.Requirement(line)
        if r.marker:
            if not r.marker.evaluate():
                # if we have marker and it's not for our platform of interest, skip
                continue
        result_lines.append(line + "\n")

@aignas
Copy link
Collaborator

aignas commented May 13, 2024

After a little bit of thinking I think the problem is that supporting cross-platform requirements would be something that detracts us from supporting other up and coming or existing cross-platform formats (e.g. pdm, poetry, uv).

I proposed a change in #1885 to unblock people needed different versions per (py_version, os, arch) tuple and it aligns well with the existing codebase.

Technically there is no parser of requirements.txt file and the code that @vors are looking at is about requirements within the METADATA files.

@aignas aignas self-assigned this May 13, 2024
github-merge-queue bot pushed a commit that referenced this issue May 19, 2024
This PR implements a better way of specifying the requirements files for
different (os, cpu) tuples. It allows for more granular specification
than what
is available today and allows for future extension to have all of the
sources
in the select statements in the hub repository.

This is replacing the previous selection of the requirements and there
are a
few differences in behaviour that should not be visible to the external
user.
Instead of selecting the right file which we should then use to create
`whl_library` instances we parse all of the provided requirements files
and
merge them based on the contents. The merging is done based on the
blocks
within the requirement file and this allows the starlark code to
understand if
we are working with different versions of the same package on different
target
platforms.

Fixes #1868
Work towards #1643, #735
@sin-ack
Copy link
Author

sin-ack commented May 23, 2024

@aignas I just tested commit a6cb620 against my project, but it doesn't fix the issue. The problem I have is that I can't really modify the requirements.txt file to not have this requirement, because it is automatically generated by rules_python_poetry from the poetry.lock. What would you recommend in this case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants