Specify default Python(s)

[stephenfin]

What's the problem this feature will solve?

tox defaults to using the Python version that tox itself is installed under (retrieved via sys.executable) for testenvs that do not contain a Python factor (e.g. py38) or have not defined a base_python / basepython setting. This version varies depending on the user's environment: a contributor using Ubuntu 22.04 will have a default Python version of 3.10, while a contributor using Fedora 37 would have a default Python version of 3.11. This means a test run on one environment may result in different results that a test run in another. This harms reproducibility and leads to confusion. Worse, this can result in failures for users in environments with recent Python versions (e.g. Fedora users) as the package under test or one of its dependencies may not yet support newer Python versions.

Currently, there are a few separate ways to resolve this.

1. Monkey patch tox so that the call to sys.executable returns the Python version(s) you want it to default to. This is obviously not something you should do.
2. You can insist that users install tox under a given interpreter version. This is error prone (users need to read documentation), does not allow (Linux) users to take advantage of distro-provided packages that are automatically updated, and generally feels kind of ugly.
3. You can take advantage of factors and create suffixed/prefixed versions of your various testenvs, so functional becomes functional{-py37,-py38,-py39,-py310}. This results in rather ugly tox.ini files and is tedious to use (tox -e functional > tox -e functional-py310).
4. You can define base_python for every environment that does not contain a Python factor. Again, this is rather tedious (particularly for projects with a large number of testenvs) and prone to mistakes when writing your tox.ini file, but at least it's nicer to use for the end user.
5. Finally, you can define a top-level [testenv] base_python and set the [tox] ignore_base_python_conflict setting to true. This results in the simplest tox.ini file and is easy to run.

Currently, 5. provides the nicest blend between tox.ini simplicity and end-user usability. Unfortunately though, there is talk of the [tox] ignore_base_python_conflict setting being removed meaning this option might not be available in the future.

Describe the solution you'd like

I would like to be able to specify a global default Python version to be used when a Python version is not already defined via a factor. Put another way, I want a user-configurable way to override the default version derived sys.executable.

Alternative Solutions

• Do not deprecate ignore_base_python_conflict.
• Remove ignore_base_python_conflict but make its true behaviour the default (so Python version specified via a factor always trumps base_python, without warning, if there's a conflict)
• Don't use tox - use containers (tbc, this is not a realistic solution 😄)

Additional context

• tox 4 breaks generative env def with -pyXXX fragments if basepython is defined #2838
• Only return Python factor on base_python conflict #2840

[gaborbernat]

Thanks for explaining the use case. This is a bit of historic artifice, and not defining base python means I don't care about the python version. They are some cases where this is useful, e.g., when calling black to format your code or pyproject-build. The python version these tools run with in these cases doesn't matter. I'd be interested to hear some concrete use cases where this is not the case. Can you provide some please?

4. You can define base_python for every environment that does not contain a Python factor. Again, this is rather tedious (particularly for projects with a large number of testenvs) and prone to mistakes when writing your tox.ini file, but at least it's nicer to use for the end user.

For those envs where the python version does matter, this was the recommended path ahead; and what I'd prefer people to do until today.

5. Finally, you can define a top-level [testenv] base_python and set the [tox] ignore_base_python_conflict setting to true. This results in the simplest tox.ini file and is easy to run.

This solution happened to work, but not by design, and not our recommended way to do it, nor do we plan to encourage/support this.

With some concrete use cases for this, I'm open to adding a fallback_base_python setting defined here https://github.com/tox-dev/tox/blob/main/src/tox/tox_env/python/api.py#L62, defaults to the host tox python version, and is used here https://github.com/tox-dev/tox/blob/main/src/tox/tox_env/python/api.py#L128 rather than fallback directly to sys.executable.

[stephenfin]

Thanks for explaining the use case. This is a bit of historic artifice, and not defining base python means I don't care about the python version. They are some cases where this is useful, e.g., when calling black to format your code or pyproject-build. The python version these tools run with in these cases doesn't matter. I'd be interested to hear some concrete use cases where this is not the case. Can you provide some please?

Sure. If you're doing typing then it is desirable/necessary to install all dependencies. This isn't possible if one or more of the dependencies doesn't support the recent version of Python found on your host. This is particularly an issue for Fedora users due to the frequent updates there.

4. You can define base_python for every environment that does not contain a Python factor. Again, this is rather tedious (particularly for projects with a large number of testenvs) and prone to mistakes when writing your tox.ini file, but at least it's nicer to use for the end user.

For those envs where the python version does matter, this was the recommended path ahead; and what I'd prefer people to do until today.

5. Finally, you can define a top-level [testenv] base_python and set the [tox] ignore_base_python_conflict setting to true. This results in the simplest tox.ini file and is easy to run.

This solution happened to work, but not by design, and not our recommended way to do it, nor do we plan to encourage/support this.

You say that, but I did explicitly call out the desire for this kind of thing when merging the ignore_basepython_conflict setting way back 😄

Out of curiosity, what is the expected use case of [testenv] base_python (rather than [testenv:foo] base_python if you don't have this kind of flag/feature? Isn't it effectively useless unless you don't want to rely on any of the pyXY factor stuff?

With some concrete use cases for this, I'm open to adding a fallback_base_python setting defined here main/src/tox/tox_env/python/api.py#L62, defaults to the host tox python version, and is used here main/src/tox/tox_env/python/api.py#L128 rather than fallback directly to sys.executable.

That sounds exactly like base_python but with a longer name 😉 If you think that's necessary then sure, but base_python is already there.

If I was going to redefine this, I'd add a new [tox] default_base_python. This would default to sys.executable. I'd put it into [tox] rather than [testenv] since it's effectively a global flag. Also, I would make this affect everything including the packaging environments. That's just me though.

Edit One additional bonus of the new global value is that it provides an opportunity to explicitly document how tox determines Python versions absent a pyXY factor.

[gaborbernat]

Out of curiosity, what is the expected use case of [testenv] base_python (rather than [testenv:foo] base_python if you don't have this kind of flag/feature? Isn't it effectively useless unless you don't want to rely on any of the pyXY factor stuff?

Not entirely; I have many projects, especially web applications, where I'm using a single python version and don't use pyxy, which is only needed for libraries, in which case you can do:

[tox] 
env_list = test,format,lint,type_check
[testenv]
basepython = python3.11

That sounds exactly like base_python but with a longer name 😉 If you think that's necessary then sure, but base_python is already there.

There's a semantic difference; one is used if base_python is not specified, and the other is always on. This solution also keeps backwards compatibility nice, without needing to ignore the option selectively for some envs (like py38, py39, etc).

If I was going to redefine this, I'd add a new [tox] default_base_python. This would default to sys.executable. I'd put it into [tox] rather than [testenv] since it's effectively a global flag. Also, I would make this affect everything including the packaging environments. That's just me though.

My first thought was also the core section, but then I thought you might want groups of envs where the default is different. E.g. for a,b,c default should be 3.11 but for c,d,e you don't care and happy to remain sys.executable. So putting it in env offers the config file more flexibility.

Edit One additional bonus of the new global value is that it provides an opportunity to explicitly document how tox determines Python versions absent a pyXY factor.

This should happen here https://tox.wiki/en/latest/config.html#base_python, not sure why it's not done today. It really should say that if not will try to extract the spec from a factor in the env name, and if that fails fallbacks to tox host pythons spec.

[stephenfin]

Out of curiosity, what is the expected use case of [testenv] base_python (rather than [testenv:foo] base_python if you don't have this kind of flag/feature? Isn't it effectively useless unless you don't want to rely on any of the pyXY factor stuff?

Not entirely; I have many projects, especially web applications, where I'm using a single python version and don't use pyxy, which is only needed for libraries, in which case you can do:

[tox] 
env_list = test,format,lint,type_check
[testenv]
basepython = python3.11

That sounds exactly like base_python but with a longer name 😉 If you think that's necessary then sure, but base_python is already there.

There's a semantic difference; one is used if base_python is not specified, and the other is always on. This solution also keeps backwards compatibility nice, without needing to ignore the option selectively for some envs (like py38, py39, etc).

There's still an implied priority though. fallback_base_python is used unless base_python or a Python factor are present. This contrasts with the current situation with [tox] ignore_base_python = true, where base_python is used unless a Python factor is present [*]. These do seem exceedingly similar...

[*] Admittedly it's not quite a simple as this. base_python will still be used if a Python factor is present if they both point to the same MAJOR.MINOR version of the same interpreter implementation, e.g.

[testenv:py311]
base_python = python3.11-64

If I was going to redefine this, I'd add a new [tox] default_base_python. This would default to sys.executable. I'd put it into [tox] rather than [testenv] since it's effectively a global flag. Also, I would make this affect everything including the packaging environments. That's just me though.

My first thought was also the core section, but then I thought you might want groups of envs where the default is different. E.g. for a,b,c default should be 3.11 but for c,d,e you don't care and happy to remain sys.executable. So putting it in env offers the config file more flexibility.

YAGNI? As I noted, my main use case for this is to insist that no environment use a new version of Python if the project under test doesn't support it. Personally, I think if I could do this:

[tox]
default_base_python = python3.10,python3.9,python3.8,python3.7

I'd be happy, since that would allow us to run all envs against some version of Python that the project explicitly supports without stating a specific version (which could cause conflicts between different environments).

Edit One additional bonus of the new global value is that it provides an opportunity to explicitly document how tox determines Python versions absent a pyXY factor.

This should happen here tox.wiki/en/latest/config.html#base_python, not sure why it's not done today. It really should say that if not will try to extract the spec from a factor in the env name, and if that fails fallbacks to tox host pythons spec.

Yeah, definitely need docs here.

[gaborbernat]

YAGNI? As I noted, my main use case for this is to insist that no environment use a new version of Python if the project under test doesn't support it. Personally, I think if I could do this:

[tox]
 default_base_python = 3.10, 3.9, 3.8, 3.7

It would suffice, but I'll need to insist on adding this to testenv rather than the core tox section. There's value in being able to do it at the env level for other use cases than yours.

I consider default_base_python because is more along the lines of what users would expect rather than ignoring settings 😊 the ignore_base_python does.

[gaborbernat]

@stephenfin do you plan to put in a PR for this?

[stephenfin]

Can do, but I'm on PTO this week so it'll be a while