Unhelpful "Operation timed out" when failing to load history

[jaraco]

There's some corruption in my xonsh history, probably because I've symlinked the history to store on Google Drive, leading to corruption:

 ~ @ ls -la ~/.local/share/xonsh/history_json
lrwxr-xr-x@ 1 jaraco  staff  57 Jan 14 13:37 /Users/jaraco/.local/share/xonsh/history_json -> /Users/jaraco/Google Drive/My Drive/AppData/xonsh history

Now when I start a new xonsh window and wait a short time for the history to load (I have a 1G limit for history), I see this traceback:

 ~ @ Exception in thread Thread-17 (_in_load_thread):
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/Cellar/python@3.12/3.12.1_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.12/site-packages/prompt_toolkit/history.py", line 189, in _in_load_thread
    for item in self.history.load_history_strings():
  File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.12/site-packages/xonsh/ptk_shell/history.py", line 28, in load_history_strings
    for cmd in hist.all_items(newest_first=True):
  File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.12/site-packages/xonsh/history/json.py", line 555, in all_items
    json_file = xlj.LazyJSON(f, reopen=False)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.12/site-packages/xonsh/lazyjson.py", line 209, in __init__
    self._load_index()
  File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.12/site-packages/xonsh/lazyjson.py", line 239, in _load_index
    locs = f.read(48)
           ^^^^^^^^^^
TimeoutError: [Errno 60] Operation timed out

When this happens, no history is loaded.

When I first migrated my history into Google Drive, I encountered this error and there were several files of zero length in the folder... and files with a -2 appended. And indeed, when I look at the history directory, there's one file there with zero length:

 ~ @ ls -laS ~/.local/share/xonsh/history_json/ | tail -n 3
-rw-------@    1 jaraco  staff    2677 Jan 14 13:47 xonsh-4fc5b669-5c75-40af-9f14-f28f53d8cb0f.json
drwx------@    6 jaraco  staff     192 Jan 14 17:14 ..
-rw-------@    1 jaraco  staff       0 Jan 21 17:02 xonsh-609cf859-1d56-4045-a7d1-0444ae63d2bc.json

Something about the interaction between xonsh and Google Drive has caused that file to be truncated to zero.

Deleting that file causes the error to go away.

xonfig

 @ xonfig
+------------------+-----------------+
| xonsh            | 0.14.3          |
| Python           | 3.12.1          |
| PLY              | 3.11            |
| have readline    | True            |
| prompt toolkit   | 3.0.43          |
| shell type       | prompt_toolkit  |
| history backend  | json            |
| pygments         | None            |
| on posix         | True            |
| on linux         | False           |
| on darwin        | True            |
| on windows       | False           |
| on cygwin        | False           |
| on msys2         | False           |
| is superuser     | False           |
| default encoding | utf-8           |
| xonsh encoding   | utf-8           |
| encoding errors  | surrogateescape |
| xontrib 1        | vox             |
| xontrib 2        | voxapi          |
| xontrib 3        | jaraco.xonsh    |
| RC file          | []              |
+------------------+-----------------+

Expected Behavior

Ideally, xonsh would avoid creating these corrupted history files by operating more robustly when the content is stored in Google Drive (or other cloud provider). I understand, however, that xonsh may be taking every reasonable attempt to write out the content and if the macOS or FileProvider or Google Drive can't save that content accurately, it may be out of the hands of xonsh.

At the very least, it would be nice if xonsh could report a more helpful error message and maybe even be tolerant to corrupt history files, such as by logging a warning about the file (or files) that couldn't be loaded, but load the rest.

Steps to Reproduce

1. Create a zero-length .json file in the history directory and launch xonsh.

For community

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment