Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many open files OSError during initialisation #60

Open
bragradon opened this issue Nov 6, 2022 · 1 comment
Open

Too many open files OSError during initialisation #60

bragradon opened this issue Nov 6, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@bragradon
Copy link

Describe the bug
For the Django project I work on, I would like to use reloadium with the python repl.
Usually to run the repl, I would type on the commandline python manage.py shell
But when using reloadium: reloadium run manage.py shell I get the following traceback

Traceback (most recent call last):
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/ll1l1lll1l1lll1lIl1l1.py", line 138, in l11l1llll1llll11Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 326, in llllll1111llll11Il1l1
  File "<string>", line 4, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 242, in __post_init__
  File "<string>", line 5, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l1llll11lll1Il1l1.py", line 170, in __post_init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l1llll11lll1Il1l1.py", line 248, in l1lll1llllll11l1Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/lllllll111l1l1llIl1l1.py", line 389, in l11l1llll1lll111Il1l1
  File "<string>", line 12, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/lllllll111l1l1llIl1l1.py", line 332, in __post_init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/observers/api.py", line 291, in schedule
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/observers/kqueue.py", line 450, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/utils/dirsnapshot.py", line 251, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/utils/dirsnapshot.py", line 283, in walk
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/utils/dirsnapshot.py", line 283, in walk
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/utils/dirsnapshot.py", line 283, in walk
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/watchdog/utils/dirsnapshot.py", line 259, in walk
OSError: [Errno 24] Too many open files: '/Users/me/Work/code/project/node_modules/queue-microtask'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/me/.virtualenvs/python/bin/reloadium", line 8, in <module>
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/__init__.py", line 20, in start
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 612, in start
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 596, in l1llll11ll11ll1lIl1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 502, in l1l1lll11ll1ll1lIl1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1111l1l111ll1llIl1l1.py", line 325, in llllll1111llll11Il1l1
  File "/Users/me/.pyenv/versions/3.10.1/lib/python3.10/contextlib.py", line 153, in __exit__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/ll1l1lll1l1lll1lIl1l1.py", line 143, in l11l1llll1llll11Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/ll1l1lll1l1lll1lIl1l1.py", line 107, in l11ll1111lll11llIl1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l111l1l111l1Il1l1.py", line 15, in l111ll11lll11l11Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l111l1l111l1Il1l1.py", line 43, in ll1ll1l1111l1ll1Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l111l1l111l1Il1l1.py", line 15, in l111ll11lll11l11Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/corium/l1l1l111l1l111l1Il1l1.py", line 24, in lll11ll1l1l111l1Il1l1
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/hub.py", line 105, in _init
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/client.py", line 85, in __init__
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/client.py", line 119, in _init_impl
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/integrations/__init__.py", line 93, in setup_integrations
  File "/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/integrations/__init__.py", line 41, in iter_default_integrations
  File "/Users/me/.pyenv/versions/3.10.1/lib/python3.10/importlib/__init__.py", line 126, in import_module
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1430, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1402, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1539, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1582, in _fill_cache
OSError: [Errno 24] Too many open files: '/Users/me/.virtualenvs/python/lib/python3.10/site-packages/reloadium/vendored/sentry_sdk/integrations'

Desktop (please complete the following information):

  • OS: MacOS
  • OS version: 12.6 (Monterey)
  • Reloadium package version: 0.9.4
  • PyCharm plugin version: None
  • Editor: commandline
  • Run mode: reloadium run

Additional context
It appears that reloadium (through watchdog) is not only scanning all my node_modules but opening the files too.
This is happening regardless if I set either the RELOADIUMPATH or RELOADIUMIGNORE environment variables.
I assumed I could configure it via reloadium_config.py since the Config object has ignored_paths and watched_paths but I think this scanning occurs prior to the config being loaded(?).

I believe this is more a problem with the vendored watchdog library, which also seems to also try limit you to 300 files in the pycharm plugin. This would be a problem for me since the django project I have as several thousand files.

Maybe you could use an alternate filesystem watcher like: https://pypi.org/project/watchfiles/

@boxed
Copy link

boxed commented Jan 27, 2023

Maybe respecting .gitignore by default is also a good idea. I have some good experiences with gitignorefile.

@dkrystki dkrystki added the bug Something isn't working label Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants