Example of using a "src" dir with gazelle #1775

dougthor42 · 2024-02-22T17:15:55Z

N.B.: This is half "example request", half "how do I..." question.

🚀 feature/example request

Relevant Rules

py_*
gazelle

Description

The Python Packaging User Guide recommends using a src dir with tests outside of the package (so they aren't shipped with the distribution/wheel), like so:

packaging_tutorial/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│   └── mypackage/
│       ├── __init__.py
│       └── foo.py
└── tests/
    ├── __init__.py
    └── test_foo.py

pytest also recommends this.

However, none of the examples describe such a use case¹.

Critically, one major aspect of the above dir structure is that the project must be pip-installed²³ before tests can be run because test_foo.py looks like:

import unittest

from mypackage import foo  # here's the problem. Note that it's not `from src.mypackage import foo`

class TestFoo(unittest.testcase):
    def test_add(self) -> None:
        self.assertEqual(foo.add(1, 1), 2)

In addition, the documentation for gazelle is lacking and I haven't been able to figure out a way to get gazelle to work with a src dir.

Notes:

I think that There is no way to specify a root of python's package structure bazel#6903 is similar.
I'm not asking for bazel to be able to do editable installs
- as said in [pip_install] ERROR: File "setup.py" not found. Directory cannot be installed in editable mode #434 (comment), doing so prevents hermeticity

Describe the solution you'd like

What I'd like to see is a new example added that showcases how to configure bazel and gazelle to work with a src dir.

In fact, I've already got a repo for it that we can use as a starting point. General bazel build|test|run works, but I am still struggling with gazelle. I'd be more than happy to build the example, but I'll need help doing so. [Edit 2024-04-11: With recent updates to gazelle, things are now working 😁]

The example would have the following structure (names are just suggestions, of course):

examples/src_dir_with_separate_tests/
├── BUILD
├── MODULE.bazel
├── README.md
├── pyproject.toml
├── requirements.in
├── src
│   └── mypackage
│       ├── BUILD
│       ├── __init__.py
│       ├── foo.py
│       └── subpackage
│           ├── BUILD
│           ├── __init__.py
│           └── subfoo.py
└── tests
    ├── BUILD
    ├── __init__.py
    ├── subpackage
    │   ├── BUILD
    │   ├── __init__.py
    │   └── test_subfoo.py
    └── test_foo.py

Describe alternatives you've considered

I tried looking for other examples on the web, but either my google-fu is failing me or there aren't any 😞.

Note: the bzlmod example appears to do something similar with the libs/my_lib dir, but it's not quite the same because libs/my_lib doesn't need to be pip-installed to run tests. IMO the example also does too much, but that's a separate topic 🙃. ↩
typically as an editable package pip install -e ., but a non-editable install also works. ↩
Really all that's needed is .../packaging_tutorial/src to be in PYTHONPATH. ↩

The text was updated successfully, but these errors were encountered:

While figuring out how to use gazelle with a `src` dir (see bazelbuild#1775) (I figured it out by the way, hence this PR), I noticed that the docs were a little unclear. This PR updates the `gazelle` docs to include details about the `python_root` directive and how to use it. I also fix a the default value for the `python_generation_mode_per_file_include_init` directive, as it was incorrectly listed as "package".

While figuring out how to use gazelle with a `src` dir (see #1775) (I figured it out by the way, hence this PR), I noticed that the docs were a little unclear. This PR updates the `gazelle` docs to include details about the `python_root` directive and how to use it. I also fix a the default value for the `python_generation_mode_per_file_include_init` directive, as it was incorrectly listed as "package".

…nal visibility labels (#1784) Fixes #1783. Add a new gazelle directive, `python_visibility`, that allows users to add labels to the `visibility` attribute of generated targets. out by the way, hence this PR), I noticed that the docs were a little This directive acts similar to[^1] the [`go_visibility` directive](https://github.com/bazelbuild/bazel-gazelle#directives). The primary use case is for python projects that separate unit test files from the python packages/modules that they test, like so: ``` packaging_tutorial/ ├── LICENSE ├── pyproject.toml ├── README.md ├── src/ │ └── mypackage/ │ ├── __init__.py │ └── foo.py └── tests/ ├── __init__.py └── test_foo.py ``` A future PR will add an example to the `./examples` directory (issue #1775). [^1]: At least, similar based on docs. I haven't done any actual comparison.

Asmhoa · 2024-04-16T04:36:03Z

Thanks for raising and putting up the example! Super helpful

While figuring out how to use gazelle with a `src` dir (see bazelbuild/rules_python#1775) (I figured it out by the way, hence this PR), I noticed that the docs were a little unclear. This PR updates the `gazelle` docs to include details about the `python_root` directive and how to use it. I also fix a the default value for the `python_generation_mode_per_file_include_init` directive, as it was incorrectly listed as "package".

This was referenced Feb 27, 2024

docs(gazelle): Add details about python_root directive. #1782

Merged

feat(gazelle): Add "python_visibility" directive that appends additional visibility labels #1784

Merged

dougthor42 linked a pull request Apr 11, 2024 that will close this issue

docs(examples): Add example of using a src dir and separate tests dir with gazelle #1842

Open

aignas added type: documentation gazelle Gazelle plugin related issues labels May 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Example of using a "src" dir with gazelle #1775

Example of using a "src" dir with gazelle #1775

dougthor42 commented Feb 22, 2024 •

edited

Asmhoa commented Apr 16, 2024

Example of using a "src" dir with gazelle #1775

Example of using a "src" dir with gazelle #1775

Comments

dougthor42 commented Feb 22, 2024 • edited

🚀 feature/example request

Relevant Rules

Description

Notes:

Describe the solution you'd like

Describe alternatives you've considered

Footnotes

Asmhoa commented Apr 16, 2024

dougthor42 commented Feb 22, 2024 •

edited