python: Add python.venv.pythonPackages option

[pinkwah]

This option allows the user to install Nix-native Python packages, much like pkgs.python3.withPackages (ps: ...). The difference is that the packages are inserted into the user's venv as standard .pth files.

See the python-venv example.

[domenkozar]

I'm not sure yet how to support requirements.txt and pkgs.pythonPackages without it being confusing to the users.

Maybe something like:

{
  languages.python.packages = [ "requests" ];
  languages.python.packagesSource = "nix";
}

{
  languages.python.packages = [ "requests" ];
  languages.python.packagesSource = "requirements";
}

Sometimes it might be useful to mix the two, but I think it's a slippery slope.

Just some ideas, it's important to get the interface right.

[pinkwah]

package already has a meaning in devenv (which python package to use), so packages might be confusing.

Maybe:

{
  languages.python.venv.requirements = [ "requests" ];
  languages.python.venv.nativeRequirements = [ "pyqt5" ];
}

Note that this is only meaningful in a context of venv.

[bobvanderlinden]

I'm wondering whether it is actually needed to add these packages to venv. It might be possible to add these packages to PYTHON_PATH so that they can be found regardless of venv.

I use the same mechanism to add pjsip to python path in order to use it in addition to the packages poetry supplies with its venv.

[pinkwah]

PYTHONPATH would interfere with other Python programs that the user may want to run from the devenv shell. For me, the best solution would be the one that works only within the virtualenv, rather than shell-wide such as when setting PYTHONPATH.

[mcdonc]

In reality, given how Nix works, I think mixing of Nix Python packages and pip-installed Python packages probably has to be supported. Rationale: half the world uses databases with binary drivers, and the other half uses numpy/pandas, and approximately none of that works right now (on the python-rewrite branch, at least) if any of it comes from a pip-found package requirement.