Cache xonshrc and imports

[micahdlamb]

xonfig

+------------------+-----------------+
| xonsh | 0.9.24 |
| Python | 3.8.5 |
| PLY | 3.11 |
| have readline | True |
| prompt toolkit | 3.0.8 |
| shell type | prompt_toolkit |
| pygments | 2.7.2 |
| on posix | False |
| on linux | False |
| on darwin | False |
| on windows | True |
| on cygwin | False |
| on msys2 | False |
| is superuser | False |
| default encoding | utf-8 |
| xonsh encoding | utf-8 |
| encoding errors | surrogateescape |
| on jupyter | True |
| jupyter kernel | None |
+------------------+-----------------+

Problem

In my .xonshrc I'm importing a .xsh file that is 700 lines long. Its taking about 20 seconds which seems excessive.

I profiled and its spending all the time compiling the code.

Is this expected?

I guess its pretty slow to do all the tokenizing, parsing, ... in python, but I was hoping it could at least be cached so my terminal is fast to startup the next time.

I asked ChatGPT and it gave me this code which is working well. I put it at the beginning of my .xonshrc. So maybe something like this could be done automatically?

import marshal
from xonsh.imphooks import XonshImportHook
from pathlib import Path

class CachedXonshImportHook(XonshImportHook):
    def get_code(self, fullname):
        filename = Path(self.get_filename(fullname))
        cache_dir = filename.parent / '__xonshcache__'
        cache_dir.mkdir(parents=True, exist_ok=True)
        cache_filename = filename.stem + '.xshc'
        cache_path = cache_dir / cache_filename
        if cache_path.exists() and cache_path.stat().st_mtime >= filename.stat().st_mtime:
            with open(cache_path, 'rb') as cache_file:
                return marshal.load(cache_file)
        else:
            code = super().get_code(fullname)
            with open(cache_path, 'wb') as cache_file:
                marshal.dump(code, cache_file)
            return code

import sys
sys.meta_path = [CachedXonshImportHook()] + sys.meta_path

[anki-code]

Basically xonsh has codecache. PR with improvements is welcome!

[micahdlamb]

@anki-code Is there an existing way to turn on the code caching for .xsh files? Or are you suggesting I create a Pull Request that updates XonshImportHook to utilize codecache?

[anki-code]

Xonsh already cashes xsh files and parts of code (i.e. ls -d $XONSH_DATA_DIR/xonsh_*_cache) when you run it. I think it will be cool if you investigate the situation around xonshrc and imports and if there is no background issues with caching this create a PR with improvements. May be it should be an option like $XONSH_CACHE_SCRIPTS or $XONSH_CACHE_EVERYTHING.