setup.py generation for local dependencies appears to be broken.

[jembishop]

• Poetry version: 1.22
• Python version: 3.10.6
• OS version and name: Ubuntu 22.04
• pyproject.toml: https://gist.github.com/jembishop/12a7fdb9b8519cd800fe647f12d408fa

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I am using the build.py feature, necessary as the project is a cython project, and I also want to specify a local dependency.
(The local dependency also happens to be a cython project, but I don't think that is important to the bug)

The stack trace (edited to remove some identifying info)

➜  my-project git:(jb/fix-feed-path) ✗ poetry install -vvv
Using virtualenv: /home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 0 updates, 0 removals, 13 skipped

  • Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Pending...
  • Installing my-other-project (0.1.0 /home/jem/longshot/strategy/my-other-project/python): Skipped for the following reason: Already installed
  • Installing urllib3 (1.26.13): Pending...
  • Installing urllib3 (1.26.13): Skipped for the following reason: Already installed

Installing the current project: my-project (0.1.0)
  - Building package my-project in editable mode
  - Falling back on using a setup.py

  Stack trace:

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1472 in _run
      1470│                 )
      1471│             else:
    → 1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )

  1  /usr/lib/python3.10/subprocess.py:420 in check_output
       418│         kwargs['input'] = empty
       419│ 
    →  420│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       421│                **kwargs).stdout
       422│ 

  CalledProcessError

  Command '['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project']' returned non-zero exit status 1.

  at /usr/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│ 
       528│ 

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:49 in pip_install
       47│ 
       48│     try:
    →  49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
       51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

  2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1435 in run_pip
      1433│         pip = self.get_pip_command()
      1434│         cmd = pip + list(args)
    → 1435│         return self._run(cmd, **kwargs)
      1436│ 
      1437│     def run_python_script(self, content: str, **kwargs: Any) -> int | str:

  1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1712 in _run
      1710│     def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
      1711│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    nip 1712│         return super()._run(cmd, **kwargs)
      1713│ 
      1714│     def get_temp_environ(

  EnvCommandError

  Command ['/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/jem/.cache/pypoetry/virtualenvs/my-project-MImg1Qvf-py3.10', '--upgrade', '--no-deps', '-e', '/home/jem/longshot/strategy/my-project'] errored with the following return code 1, and output: 
  Obtaining file:///home/jem/longshot/strategy/my-project
    Preparing metadata (setup.py): started
    Preparing metadata (setup.py): finished with status 'error'
    error: subprocess-exited-with-error
    
    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [1 lines of output]
        error in my-project setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid URL: ../my-other-project/python
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: metadata-generation-failed
  
  × Encountered error while generating package metadata.
  ╰─> See above for output.
  
  note: This is an issue with the package mentioned above, not pip.
  hint: See above for details.
  

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│ 
      1478│         return decode(output)
      1479│ 
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  10  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:329 in run
       327│ 
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

   9  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
       183│         self._load_plugins(io)
       184│ 
     → 185│         exit_code: int = super()._run(io)
       186│         return exit_code
       187│ 

   8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│ 
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│ 

   7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
       463│ 
       464│         if error is not None:
     → 465│             raise error
       466│ 
       467│         return event.exit_code

   6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
       447│ 
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
        81│ 
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

   3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/install.py:177 in handle
       175│             return 0
       176│ 
     → 177│         builder.build()
       178│ 
       179│         if overwrite:

   2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:63 in build
        61│                     "  - Falling back on using a setup.py"
        62│                 )
     →  63│                 self._setup_build()
        64│                 path: Path = self._path
        65│                 return path

   1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/masonry/builders/editable.py:120 in _setup_build
       118│                 )
       119│                 try:
     → 120│                     pip_install(self._path, self._env, upgrade=True, editable=True)
       121│                 finally:
       122│                     shutil.move(

  PoetryException

  Failed to install /home/jem/longshot/strategy/my-project

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│ 
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

After some debugging it appears that the generated setup.py file has an incorrect reference to the
local dependency. It generates:

install_requires = [
...
"my-other-package @ ../my-other-package",
...
]

When it should generate a file uri:

install_requires = [
...
"my-other-package @ file:///absolute/path/to/my-other-package",
...
]

to be a valid dependency.

I hacked around this by modifying sdist.py in the convert_dependencies function.
I simply matched against the regex (.*) @ ((?:\.|\/).*) to pick up a local path and replace it with the file uri.

Would you be interested in me raising a pull request for this?

[neersighted]

This is not specific to setup.py, but to our PEP 508 metadata generation in general. For some historic context on why this lurked for so long, path dependencies were not originally envisioned as being useful for distributed packages.

In any case, this is a duplicate of #5273 and was solved in python-poetry/poetry-core#512. This fix was shipped in poetry-core 1.4.0. Poetry 1.3.0 will incorporate this version of poetry-core; you may follow the release process at #6972.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.