Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modifying tox_root does not propagate to dependent fields. #2933

Closed
kdestin opened this issue Mar 4, 2023 · 3 comments
Closed

Modifying tox_root does not propagate to dependent fields. #2933

kdestin opened this issue Mar 4, 2023 · 3 comments
Labels
enhancement help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.

Comments

@kdestin
Copy link
Contributor

kdestin commented Mar 4, 2023

Issue

Changing the value of tox_root does not propagate to fields whose default value also depends on tox_root (i.e work_dir, temp_dir, env_dir, etc...).

This reproducible in tox 4.4.6 and in the latest commit (which is 177a227 at time of writing) when trying to modify tox_root using the CLI interface (e.g tox --root ~).

I assume this would also be the case when using a plugin to modify tox_root, but the following (using an approach based on this discussion) doesn't even change the value of tox_root, so I assume that tox_root might be getting cached before this hook can run:

@impl
def tox_add_core_config(core_conf: CoreConfigSet, state: State):
    core_conf.loaders.insert(0, MemoryLoader(tox_root=Path.home())

Environment

Provide at least:

  • OS:
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.1 LTS
Release:	22.04
Codename:	jammy
  • pip list of the host Python where tox is installed:
Package               Version
--------------------- -------
cachetools            5.3.0
chardet               5.1.0
colorama              0.4.6
distlib               0.3.6
docstring-to-markdown 0.11
filelock              3.9.0
jedi                  0.18.2
packaging             23.0
parso                 0.8.3
pip                   22.0.2
platformdirs          3.0.0
pluggy                1.0.0
pylsp-rope            0.1.11
pyproject_api         1.5.0
python-lsp-jsonrpc    1.0.0
python-lsp-server     1.7.1
pytoolconfig          1.2.5
rope                  1.7.0
setuptools            59.6.0
tomli                 2.0.1
tox                   4.4.6
ujson                 5.7.0
virtualenv            20.20.0

Output of running tox

Provide the output of tox -rvv:

py: 603 W remove tox env folder /home/kd/Documents/tox/repro/.tox/py [tox/tox_env/api.py:322]
py: 683 I find interpreter for spec PythonSpec(path=/home/kd/Documents/tox/venv/bin/python3) [virtualenv/discovery/builtin.py:56]
py: 683 D discover exe for PythonInfo(spec=CPython3.10.6.final.0-64, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:437]
py: 685 D filesystem is case-sensitive [virtualenv/info.py:24]
py: 687 D got python info of /usr/bin/python3.10 from /home/kd/.local/share/virtualenv/py_info/1/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:129]
py: 688 I proposed PythonInfo(spec=CPython3.10.6.final.0-64, system=/usr/bin/python3.10, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
py: 689 D accepted PythonInfo(spec=CPython3.10.6.final.0-64, system=/usr/bin/python3.10, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
py: 762 I create virtual environment via CPython3Posix(dest=/home/kd/Documents/tox/repro/.tox/py, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
py: 763 D create folder /home/kd/Documents/tox/repro/.tox/py/bin [virtualenv/util/path/_sync.py:9]
py: 764 D create folder /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages [virtualenv/util/path/_sync.py:9]
py: 766 D write /home/kd/Documents/tox/repro/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py: 767 D 	home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
py: 767 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py: 767 D 	version_info = 3.10.6.final.0 [virtualenv/create/pyenv_cfg.py:34]
py: 768 D 	virtualenv = 20.20.0 [virtualenv/create/pyenv_cfg.py:34]
py: 768 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py: 768 D 	base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 768 D 	base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 768 D 	base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
py: 769 D symlink /usr/bin/python3.10 to /home/kd/Documents/tox/repro/.tox/py/bin/python [virtualenv/util/path/_sync.py:28]
py: 770 D create virtualenv import hook file /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
py: 771 D create /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
py: 772 D ============================== target debug ============================== [virtualenv/run/session.py:50]
py: 772 D debug via /home/kd/Documents/tox/repro/.tox/py/bin/python /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:193]
py: 772 D {
  "sys": {
    "executable": "/home/kd/Documents/tox/repro/.tox/py/bin/python",
    "_base_executable": "/home/kd/Documents/tox/repro/.tox/py/bin/python",
    "prefix": "/home/kd/Documents/tox/repro/.tox/py",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/home/kd/Documents/tox/repro/.tox/py",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib/python310.zip",
      "/usr/lib/python3.10",
      "/usr/lib/python3.10/lib-dynload",
      "/home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]",
  "makefile_filename": "/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/usr/lib/python3.10/os.py'>",
  "site": "<module 'site' from '/usr/lib/python3.10/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib/python3.10/datetime.py'>",
  "math": "<module 'math' (built-in)>",
  "json": "<module 'json' from '/usr/lib/python3.10/json/__init__.py'>"
} [virtualenv/run/session.py:51]
py: 847 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/kd/.local/share/virtualenv) [virtualenv/run/session.py:55]
py: 863 D got embed update of distribution wheel from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
py: 865 D got embed update of distribution pip from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
py: 876 D got embed update of distribution setuptools from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
py: 878 D install pip from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-23.0.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 880 D install wheel from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 884 D install setuptools from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/setuptools-67.4.0-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 891 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip [virtualenv/util/path/_sync.py:36]
py: 903 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools-67.4.0.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools-67.4.0.virtualenv [virtualenv/util/path/_sync.py:36]
py: 904 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel [virtualenv/util/path/_sync.py:36]
py: 909 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/_distutils_hack to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
py: 927 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools-67.4.0.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools-67.4.0.dist-info [virtualenv/util/path/_sync.py:36]
py: 966 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
py: 994 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
py: 998 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
py: 1047 D generated console scripts wheel3 wheel3.10 wheel-3.10 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1360 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/pkg_resources to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
py: 1480 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/distutils-precedence.pth to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
py: 1484 D generated console scripts  [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1656 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip-23.0.1.dist-info [virtualenv/util/path/_sync.py:36]
py: 1670 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip-23.0.1.virtualenv [virtualenv/util/path/_sync.py:36]
py: 1674 D generated console scripts pip3 pip-3.10 pip3.10 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1675 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
py: 1682 D write /home/kd/Documents/tox/repro/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py: 1682 D 	home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D 	version_info = 3.10.6.final.0 [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D 	virtualenv = 20.20.0 [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D 	base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D 	base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 1685 D 	base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
py: 1702 W commands[0]> python -c 'print('"'"'\n'"'"'.join(['"'"'tox_root:\t/home/kd/Documents/tox/repro'"'"', '"'"'work_dir:\t/home/kd/Documents/tox/repro/.tox'"'"', '"'"'temp_dir:\t/home/kd/Documents/tox/repro/.tox/.tmp'"'"', '"'"'env_dir:\t/home/kd/Documents/tox/repro/.tox/py'"'"', '"'"'env_tmp_dir:\t/home/kd/Documents/tox/repro/.tox/py/tmp'"'"', '"'"'env_log_dir:\t/home/kd/Documents/tox/repro/.tox/py/log'"'"']))' [tox/tox_env/api.py:428]
tox_root:	/home/kd/Documents/tox/repro
work_dir:	/home/kd/Documents/tox/repro/.tox
temp_dir:	/home/kd/Documents/tox/repro/.tox/.tmp
env_dir:	/home/kd/Documents/tox/repro/.tox/py
env_tmp_dir:	/home/kd/Documents/tox/repro/.tox/py/tmp
env_log_dir:	/home/kd/Documents/tox/repro/.tox/py/log
py: 1809 I exit 0 (0.10 seconds) /home/kd/Documents/tox/repro> python -c 'print('"'"'\n'"'"'.join(['"'"'tox_root:\t/home/kd/Documents/tox/repro'"'"', '"'"'work_dir:\t/home/kd/Documents/tox/repro/.tox'"'"', '"'"'temp_dir:\t/home/kd/Documents/tox/repro/.tox/.tmp'"'"', '"'"'env_dir:\t/home/kd/Documents/tox/repro/.tox/py'"'"', '"'"'env_tmp_dir:\t/home/kd/Documents/tox/repro/.tox/py/tmp'"'"', '"'"'env_log_dir:\t/home/kd/Documents/tox/repro/.tox/py/log'"'"']))' pid=3033 [tox/execute/api.py:275]
  py: OK (1.21=setup[1.10]+cmd[0.10] seconds)
  congratulations :) (1.41 seconds)

Minimal example

If possible, provide a minimal reproducer for the issue:

  1. Reproduce minimal tox project

      $ tree
 .
 ├── foo
 └── tox.ini

    Where

    # tox.ini
[testenv]
commands=
python -c "print('\n'.join(\[\
     'tox_root:\t{tox_root}', \
     'work_dir:\t{work_dir}', \
     'env_dir:\t{env_dir}', \
     'env_tmp_dir:\t{env_tmp_dir}', \
     'env_log_dir:\t{env_log_dir}'\]))"

  2. Run tox -qq --root foo

    Expected Output:

    tox_root:	/home/kd/Documents/tox/repro/foo
work_dir:	/home/kd/Documents/tox/repro/foo/.tox
temp_dir:	/home/kd/Documents/tox/repro/foo/.tox/.tmp
env_dir:	/home/kd/Documents/tox/repro/foo/.tox/py
env_tmp_dir:	/home/kd/Documents/tox/repro/foo/.tox/py/tmp
env_log_dir:	/home/kd/Documents/tox/repro/foo/.tox/py/log

    Actual Output:

    tox_root:	/home/kd/Documents/tox/repro/foo
work_dir:	/home/kd/Documents/tox/repro/.tox
temp_dir:	/home/kd/Documents/tox/repro/.tox/.tmp
env_dir:	/home/kd/Documents/tox/repro/.tox/py
env_tmp_dir:	/home/kd/Documents/tox/repro/.tox/py/tmp
env_log_dir:	/home/kd/Documents/tox/repro/.tox/py/log

Miscelleanous

  • --workdir functions as expected. With the same setup as above running tox -qq --workdir foo/.tox does correctly propagate the changed workdir to dependent config values:

    tox_root:	/home/kd/Documents/tox/repro
work_dir:	/home/kd/Documents/tox/repro/foo/.tox
temp_dir:	/home/kd/Documents/tox/repro/foo/.tox/.tmp
env_dir:	/home/kd/Documents/tox/repro/foo/.tox/py
env_tmp_dir:	/home/kd/Documents/tox/repro/foo/.tox/py/tmp
env_log_dir:	/home/kd/Documents/tox/repro/foo/.tox/py/log
@gaborbernat gaborbernat added help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted. enhancement labels Mar 12, 2023
@gaborbernat
Copy link
Member

PR welcome.

@Tbruno25
Copy link
Contributor

Ran into this issue today myself -- going to take a shot at a PR

@Tbruno25 Tbruno25 mentioned this issue Mar 29, 2023
Merged
5 tasks
@kdestin
Copy link
Contributor Author

kdestin commented Apr 5, 2023

Resolved by #2962

@kdestin kdestin closed this as completed Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gaborbernat @Tbruno25 @kdestin