Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: --build_python_zip seems to not work in py_binary #236

Open
ChangFu2000 opened this issue Dec 28, 2023 · 3 comments
Open

[Bug]: --build_python_zip seems to not work in py_binary #236

ChangFu2000 opened this issue Dec 28, 2023 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@ChangFu2000
Copy link

ChangFu2000 commented Dec 28, 2023

What happened?

When building a py_binary with --build_python_zip specified, expected .zip artifact is not generated.

Version

Development (host) and target OS/architectures:
Ubuntu 20.04

Output of bazel --version:
bazel 7.0.0 (bazelisk)

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:
aspect_rules_py, version: 0.5.0
rules_python, version: 0.22.0

Language(s) and/or frameworks involved:
Python

How to reproduce

# MODULE.bazel

bazel_dep(name = "aspect_rules_py", version = "0.5.0")

bazel_dep(name = "rules_python", version = "0.22.0", dev_dependency = True)

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    name = "python3",
    configure_coverage_tool = True,
    python_version = "3.11",
)
use_repo(python, "python3_toolchains")

register_toolchains(
    "@python3_toolchains//:all",
)
# BUILD

load("@aspect_rules_py//py:defs.bzl", "py_binary")

py_binary(
    name = "script",
    srcs = ["script.py"],
    main = "script.py",
)

Running with: bazel build //:script --build_python_zip



### Any other information?

_No response_
@ChangFu2000 ChangFu2000 added the bug Something isn't working label Dec 28, 2023
@github-actions github-actions bot added the untriaged Requires traige label Dec 28, 2023
@mattem mattem added enhancement New feature or request and removed bug Something isn't working untriaged Requires traige labels Jan 2, 2024
@thesayyn thesayyn self-assigned this Mar 26, 2024
@alexeagle
Copy link
Member

We had a design discussion about this today, but we don't have enough requirements.

I'm not sure that the Bazel implementation for --build_python_zip produces a file that follows any existing specification. It was written long ago in https://bazel-review.googlesource.com/c/bazel/+/4244 and there's no thread there about how the structure of the file was determined. I'm very hesitant to build an unspecified thing that just emulates this ancient Bazel feature, as it exposes us to a ton of bugs if we make wrong choices. We wouldn't be able to reply to such bugs with "we produce the following well-understood output, so you can reproduce that problem outside of Bazel"

We could start from first-principles and follow https://peps.python.org/pep-0441/ - in this case rules_py could produce a zipapp following that specification that includes only the python files in your application (and the runfiles tree for their data dependencies)
However that spec doesn't include an interpreter, or third-party packages. It's typically intended for deployment to some environment where you already have those things (e.g. a docker container where the base included python and pip install is run as a step, or maybe a remote orchestrator service or lambda environment whose docs say "upload your application code")

We could offer something different - a PEX file is a well-understood "self-extracting executable" for Python. rules_py could produce something identical to https://www.pantsbuild.org/2.20/docs/python/overview/pex and this would be usable in more cases.

@Cartman75
Copy link

If there is a self executing python file, that would seem like a upgrade and much better then zip.. Others can tell me if I am wrong, but currently, the zip is just the best we got right now, sorta deal. I would much prefer to just execute a file like most other code can do.

@ChangFu2000
Copy link
Author

@alexeagle Thanks for the reply. We are using a docker environment so both .pyz and .pex satisfy our use case. I'm not faimliar with Bazel implementation details so please follow your decision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 📋 Backlog
Development

No branches or pull requests

5 participants