Make xonsh tolerant to inaccessible xonsh data paths #5319

inmaldrerah · 2024-04-02T10:50:00Z

xonfig

My distro is actually NixOS.

$ xonfig
+------------------+--------------------+
| xonsh            | 0.15.1             |
| Python           | 3.11.8             |
| PLY              | 3.11               |
| have readline    | True               |
| prompt toolkit   | 3.0.43             |
| shell type       | prompt_toolkit     |
| history backend  | json               |
| pygments         | 2.17.2             |
| on posix         | True               |
| on linux         | True               |
| distro           | unknown            |
| on wsl           | False              |
| on darwin        | False              |
| on windows       | False              |
| on cygwin        | False              |
| on msys2         | False              |
| is superuser     | False              |
| default encoding | utf-8              |
| xonsh encoding   | utf-8              |
| encoding errors  | surrogateescape    |
| xontrib          | []                 |
| RC file 1        | /etc/xonsh/xonshrc |
+------------------+--------------------+

Expected Behavior

Xonsh runs, though giving up logging command history to file.

People need a shell for data rescue if they mess up their filesystem and can only mount it read only.

Current Behavior

Xonsh keeps throwing errors on an open(...) call and I cannot input anything except pressing <c-C> to exit the shell.

Traceback (if applicable)

I havn't yet tried readonly ~/.local/share/xonsh, but I set $XONSH_DATA_DIR to p"~/ro" where I build up a readonly overlayfs, and got the following output (and now I understand why it is generating infinite logs: I set xonsh as my login shell and it keeps falling back to itself on error)

/etc/xonsh/xonshrc:16:16 -     source-bash "/nix/store/ml2wn9nd7p5912ynfxq27dwlkh77qx1m-set-environment"
/etc/xonsh/xonshrc:16:16 +     ![source-bash "/nix/store/ml2wn9nd7p5912ynfxq27dwlkh77qx1m-set-environment"]
Traceback (most recent call last):
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/main.py", line 469, in main
    args = premain(argv)
           ^^^^^^^^^^^^^
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/main.py", line 409, in premain
    env = start_services(shell_kwargs, args, pre_env=pre_env)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/main.py", line 355, in start_services
    _load_rc_files(shell_kwargs, args, env, execer, ctx)
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/main.py", line 310, in _load_rc_files
    XSH.rc_files = xonshrc_context(
                   ^^^^^^^^^^^^^^^^
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/environ.py", line 2492, in xonshrc_context
    status = xonsh_script_run_control(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/environ.py", line 2553, in xonsh_script_run_control
    exc_info = run_script_with_cache(filename, execer, ctx)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/codecache.py", line 169, in run_script_with_cache
    update_cache(ccode, cachefname)
  File "/nix/store/3sgwcszkn0c5ab5nwm0k1iga4shj2ps0-python3-3.11.8-env/lib/python3.11/site-packages/xonsh/codecache.py", line 98, in update_cache
    with open(cache_file_name, "wb") as cfile:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 30] Read-only file system: 'ro/xonsh_script_cache/nix/store/hy8zfhlx9rjwws4w62bjdk5ix9x550q1-etc-xonsh-xonshrc.cpython-311'
Xonsh encountered an issue during launch
Failback to /run/current-system/sw/bin/xonsh

Steps to Reproduce

make ~/.local read only (maybe through mounting an overlayfs)
start xonsh

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

The text was updated successfully, but these errors were encountered:

anki-code · 2024-04-02T12:07:27Z

We need to avoid stopping the work in this case and show the warning about setting $XONSH_DATA_DIR manually.
PR is welcome!

inmaldrerah · 2024-04-02T18:24:43Z

It seems whether to use cache is not checked before updating cache on boot, but is checked in BaseShell. Maybe the former should also check use_cache before updating cache?

The warning should be given directly from the update_cache function directly, I think. Is it the right way calling is_writable_file to see if the log file is writable, and calling print_warning to show the warning?

BTW, I found some bad practice here.

anki-code · 2024-04-02T21:51:19Z

@inmaldrerah thanks for open this!

UPD: let's implement simpler solution with mitigations.

I see this bigger: make xonsh tolerant to inaccessible xonsh data paths starting from $XONSH_DATA_DIR and $XONSH_CACHE_DIR. After this maybe others in env. There are not so many uses of these variables in the code.

So for achieving this awesome thing I have in mind this draft:

For $XONSH_DATA_DIR we have str ensurer. Probably we can create new env_path_strict ensurer based on env_path ensurer that will check the directory access and show warning (may be switched off by additional env var) in is_env_path_strict that based on is_env_path. Than replace str to env_path_strict for $XONSH_DATA_DIR and $XONSH_CACHE_DIR. Thus we will control the path on first get and alert about issue as soon as possible and just once.
I noticed that it's possible to run from bash XONSH_DATA_DIR= xonsh --no-rc without any warnings about $XONSH_DATA_DIR == '' and I don't like this. I expect replacing '' to None in this case and expect crashes and exceptions from the code that aren't tolerant to the lack of this variable. It's pretty good to motivate developers support the cases without accessible paths and gives the way to test this manually.
In the xonsh core code and core xontribs we need to use is_writable_file to check the access to $XONSH_DATA_DIR before usage and do nothing (show warning in XONSH_DEBUG mode).
Finally we need test for this e.g. in test_update_cache.

…che (#5430) Closes #5319 ### Before ```xsh mkdir -p /tmp/noaccess chmod 000 /tmp/noaccess echo 'print(1)' > /tmp/1.xsh xonsh --no-rc --no-env -DXONSH_DATA_DIR='/tmp/noaccess' # Json History error xonsh --no-rc --no-env -DXONSH_DATA_DIR='/tmp/noaccess' /tmp/1.xsh # Script cache error ``` ### After ```xsh mkdir -p /tmp/noaccess chmod 000 /tmp/noaccess echo 'print(1)' > /tmp/1.xsh xonsh --no-rc --no-env -DXONSH_DATA_DIR='/tmp/noaccess' # Error during load <class 'xonsh.history.json.JsonHistory'>: [Errno 13] Permission denied: '/tmp/noaccess/history_json' # Set $XONSH_HISTORY_BACKEND='dummy' to disable history. # History disabled. # @ xonsh --no-rc --no-env -DXONSH_DATA_DIR=/tmp/noaccess /tmp/1.xsh # xonsh: For full traceback set: $XONSH_SHOW_TRACEBACK = True # update_cache: Cache file is not writable: /tmp/noaccess/xonsh_script_cache/private/tmp/1_.xsh.cpython-311 # Set $XONSH_CACHE_SCRIPTS=0, $XONSH_CACHE_EVERYTHING=0 to disable cache. # 1 ``` And no warnings when everything is switched off: ```xsh xonsh % python -m xonsh --no-rc --no-env -DXONSH_DATA_DIR=/tmp/noaccess -DXONSH_HISTORY_BACKEND=dummy # @ xonsh --no-rc --no-env -DXONSH_DATA_DIR=/tmp/noaccess -DXONSH_CACHE_SCRIPTS=0 -DXONSH_CACHE_EVERYTHING=0 /tmp/1.xsh # 1 ``` ## For community ⬇️ **Please click the 👍 reaction instead of leaving a `+1` or 👍 comment** --------- Co-authored-by: a <1@1.1> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

anki-code added good first issue cache case priority-high labels Apr 2, 2024

inmaldrerah changed the title ~~Xonsh history panics about ~/.local being read only~~ Xonsh codecache panics about ~/.local being read only Apr 2, 2024

inmaldrerah changed the title ~~Xonsh codecache panics about ~/.local being read only~~ Xonsh code cache panics about ~/.local being read only Apr 2, 2024

anki-code changed the title ~~Xonsh code cache panics about ~/.local being read only~~ Make xonsh tolerant to inaccessible xonsh data paths Apr 2, 2024

anki-code added the environ label Apr 2, 2024

anki-code mentioned this issue May 20, 2024

Make xonsh tolerant to inaccessible paths: history backend, script cache #5430

Merged

anki-code added error-handling and removed environ cache case labels May 20, 2024

anki-code closed this as completed in #5430 May 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make xonsh tolerant to inaccessible xonsh data paths #5319

Make xonsh tolerant to inaccessible xonsh data paths #5319

inmaldrerah commented Apr 2, 2024 •

edited

anki-code commented Apr 2, 2024 •

edited

inmaldrerah commented Apr 2, 2024

anki-code commented Apr 2, 2024 •

edited

Make xonsh tolerant to inaccessible xonsh data paths #5319

Make xonsh tolerant to inaccessible xonsh data paths #5319

Comments

inmaldrerah commented Apr 2, 2024 • edited

xonfig

Expected Behavior

Current Behavior

Traceback (if applicable)

Steps to Reproduce

For community

anki-code commented Apr 2, 2024 • edited

inmaldrerah commented Apr 2, 2024

anki-code commented Apr 2, 2024 • edited

inmaldrerah commented Apr 2, 2024 •

edited

anki-code commented Apr 2, 2024 •

edited

anki-code commented Apr 2, 2024 •

edited