Import could not be resolved with [tool.setuptools.package-dir]

[jakelevi1996]

Environment data

• OS and version: Ubuntu 22.04.1 LTS
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.10.12
• Pylance: v2024.5.1

Repro Steps

I have this pyproject.toml:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "constructions"
version = "0.0.1"

[tool.setuptools.package-dir]
constructions = "src"

And this Python code in src/__init__.py:

def square(x):
    return x*x

def cube(x):
    return x*x*x

I install with python -m pip install -e ., then in a Python script:

import constructions as cn

print(cn.square, cn.square(3), cn.cube(3))

And I get the expected output <function square at 0x7975b5123760> 9 27, BUT I get Import "constructions" could not be resolved Pylance in VS Code and warning squiggles:

If I delete [tool.setuptools.package-dir] constructions = "src" from pyproject.toml and move src/__init__.py to src/constructions/__init__.py and rerun python -m pip install -e ., and rerun the Python script, I get the same output, but now the Pylance message and warning squiggle is removed.

The issue

The Python module works as expected in both cases, but PyLance is not working in the first case src/__init__.py, only in the second case src/constructions/__init__.py.

I would prefer to use the first case, so I don't need the extra unnecessary directory in the repository structure, but in this case PyLance doesn't work, and I expect that it should work, because the Python code itself works fine.

[heejaechang]

I believe you are hitting this issue (https://microsoft.github.io/pyright/#/import-resolution?id=editable-installs)

unfortunately, currently we don't support the new dynamic setup behavior. you need to install using old legacy mode following legacy structure.

[jakelevi1996]

I don't know much about what goes on under the hood with Pylance/Pyright/pip install/setuptools, but the link you posted describes configuring the editable install to use .pth files with file paths instead of executable lines and import hooks, EG by using --config-settings editable_mode=strict.

In my example above, I was able to make Pylance work correctly by changing the location of my source file and removing [tool.setuptools.package-dir] from pyproject.toml, but otherwise not changing anything about the way I install the module (IE in the second case Pylance works fine without having to use --config-settings editable_mode=strict or anything like that), and it's not clear to me that this would change whether pip/setuptools uses file paths vs executable lines?

[jakelevi1996]

Is there a way I can check whether the editable install is using .pth files that contain file paths rather than executable lines? If so I can check in both of the cases I described in the original issue and see if this is indeed the case

[rchiodo]

.pth files should be in the site-packages folder for your package I believe.

I think this is the documentation for .pth on python.org
https://docs.python.org/3/library/site.html

When investigating similar issues, that's where I've seen them.

[jakelevi1996]

@heejaechang you are correct! The .pth file is in /home/username/.local/lib/python3.10/site-packages/__editable__.constructions-0.0.1.pth.

In the first case, with src/__init__.py, it reads:

import __editable___constructions_0_0_1_finder; __editable___constructions_0_0_1_finder.install()

In this second case, with src/constructions/__init__.py, it reads:

/home/username/programming/constructions/src

So you're right, this does change whether the .pth file that contains file paths vs executable lines

[jakelevi1996]

It's not clear to me why specifying [tool.setuptools.package-dir] should change whether file paths are used vs import hooks in the .pth file, but I suppose that's no longer relevant to Pylance

[heejaechang]

it is related to https://peps.python.org/pep-0660/ as @debonte labeled this issue.

Basically, setuptool used to use a static way (file path in pth) to indicate where a static analysis tool can find the source for the editable install package, the Pep660 changed it to use a script as you can see import __editable___constructions_0_0_1_finder; __editable___constructions_0_0_1_finder.install().

problem of the approach is that something needs to run that script to find out where the source of editable install package is, and most of static analysis tools don't run (including us) random code (since tool can't trust what that install script would do) and there is no guarantee that env is set up properly to run the script for the analysis tool and etc.

due to those various reasons, we don't support pep660 way of editable install. but only legacy compat or strict mode.

...

about this,

move src/init.py to src/constructions/init.py and rerun python -m pip install -e . ...

this probably works because we discover them not as editable install but as regular user files. I think your script is probably in the same workspace as that editable install package is?

if you have your package in root1 and put your script in root2 of vscode multi root workspace (*.code-workspace) and do editable install of package in root1 to root2, I bet your approach won't work. but it will work with compat or strict mode.

...

that said, underneath, all the legacy mode does for us is adding the path in the .pth in our python search paths (PYTHONPATH). you can get the same effect by either manually setting PYTHONPATH environment variable or python.analysis.extraPath options in vscode settings (settings.json).

if you don't want to deal with editable install mechanism and all you want is resolving imports to it in edit time, you can just use python.analysis.extraPath approach for pylance and use the new approach for runtime (when you actually run the script)

if you set PYTHONPATH manually, that should work for both run time and edit time as well.

otherwise, the compat/strict mode.

hope this helps.

[ThiefMaster]

It would be nice if this could be fixed properly. Especially with people starting to use the much faster uv pip instead of pip, there's a much higher chance of getting a PEP660 installation (even if setuptools is available).

[heejaechang]

@ThiefMaster I will tag some people @debonte @luabud but I doubt it will be supported in static analysis tool anytime soon. I don't think other tools such as mypy support it for the same reason.

[heejaechang]

related issue #3473

[jakelevi1996]

@ThiefMaster correct me if I'm wrong, but the impression I got from @heejaechang is that this is really an issue with other tools/standards, EG PEP660, setuptools, pip, and that actually this shouldn't be fixed by PyLance (because it would require running external code which is undesirable in a static analysis tool like PyLance), but rather it should be fixed elsewhere (IE in those other tools/standards)

[ThiefMaster]

Sure, ideally it'll eventually be fixed in the standard, but this isn't a battle that should be fought on the backs of the users/developers who just want to use good tooling (e.g. uv) without losing useful features in their IDE. In fact I posted a comment in the linked issue (which I found after commenting in here) which, while probably not a perfect solution, provides a decent workaround that could easily be adapted to work without executing any external code.

[erictraut]

Pylance is built on top of the pyright type checker. Pyright is a standards-compliant type checker.

The way to address this issue is to fix the Python standards so PEP 660 is compatible with static analysis tools and vice versa. The pylance issue tracker is not the right place to discuss this, so you're unlikely to make any headway here.

If you would like to propose a standard way to make this work across the ecosystem (all packaging tools and static analysis tools), you can start a discussion in the Python forums. This issue spans both typing and packaging, so you could post in either and then cross-post to the other. This will ultimately require a PEP, but a public discussion in the appropriate forum is a good place to start.

[ThiefMaster]

Fair enough, I just commented on what I believe is the corresponding issue on the pyright repo: microsoft/pyright#3880 (comment)

And sure, coming up with a proper solution backed by a PEP is clearly the desired long-term solution. But why not be pragmatic and have a way that fixes an obvious problem pretty quickly, instead of waiting for a process that's certainly going to last many months if not over a year...

[erictraut]

Short-term hacks that work in some cases and for some combinations of tools is not the right answer here. Any solution to this problem will require a well-defined and sound design. If you think you have such a proposal and others in the community agree with your approach, you should be able to get to a draft PEP pretty quickly. Once there's a draft PEP that looks like it's likely to be accepted, I can implement provisional support for it in pyright.