Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: module 'select' has no attribute 'epoll' #2848

Open
kushalmraut opened this issue Nov 1, 2023 · 20 comments · May be fixed by #2928
Open

AttributeError: module 'select' has no attribute 'epoll' #2848

kushalmraut opened this issue Nov 1, 2023 · 20 comments · May be fixed by #2928

Comments

@kushalmraut
Copy link

kushalmraut commented Nov 1, 2023

version : trio==0.22.2

[2023-10-10 12:06:56 +0000] [4254] [INFO] Worker exiting (pid: 4254)
[2023-10-10 12:07:00 +0000] [4232] [INFO] Shutting down: Master
[2023-10-10 12:07:00 +0000] [4232] [INFO] Reason: Worker failed to boot.
[2023-10-10 12:07:04 +0000] [4264] [DEBUG] Current configuration:
 config: None
 bind: ['unix:/run/anahit.sock']
 backlog: 2048
 workers: 1
 worker_class: gevent
 threads: 1
 worker_connections: 1000
 max_requests: 0
 max_requests_jitter: 0
 timeout: 120
 graceful_timeout: 30
 keepalive: 2
 limit_request_line: 4094
 limit_request_fields: 100
 limit_request_field_size: 8190
 reload: False
 reload_engine: auto
 reload_extra_files: []
 spew: False
 check_config: False
 preload_app: False
 sendfile: None
 reuse_port: False
 chdir: /home/anahit_web/anahit_website/app
 daemon: False
 raw_env: []
 pidfile: None
 worker_tmp_dir: None
 user: 1000
 group: 33
 umask: 0
 initgroups: False
 tmp_upload_dir: None
 secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
 forwarded_allow_ips: ['127.0.0.1']
 accesslog: /home/anahit_web/logs/access.log
 disable_redirect_access_to_syslog: False
 access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
 errorlog: /home/anahit_web/logs/anahit_gunicorn.log
 loglevel: debug
 capture_output: True
 logger_class: gunicorn.glogging.Logger
 logconfig: None
 logconfig_dict: {}
 syslog_addr: udp://localhost:514
 syslog: False
 syslog_prefix: None
 syslog_facility: user
 enable_stdio_inheritance: True
 statsd_host: None
 dogstatsd_tags: 
 statsd_prefix: 
 proc_name: None
 default_proc_name: anahit.wsgi:application
 pythonpath: None
 paste: None
 on_starting: <function OnStarting.on_starting at 0x7f93d1e15440>
 on_reload: <function OnReload.on_reload at 0x7f93d1e15580>
 when_ready: <function WhenReady.when_ready at 0x7f93d1e156c0>
 pre_fork: <function Prefork.pre_fork at 0x7f93d1e15800>
 post_fork: <function Postfork.post_fork at 0x7f93d1e15940>
 post_worker_init: <function PostWorkerInit.post_worker_init at 0x7f93d1e15a80>
 worker_int: <function WorkerInt.worker_int at 0x7f93d1e15bc0>
 worker_abort: <function WorkerAbort.worker_abort at 0x7f93d1e15d00>
 pre_exec: <function PreExec.pre_exec at 0x7f93d1e15e40>
 pre_request: <function PreRequest.pre_request at 0x7f93d1e15f80>
 post_request: <function PostRequest.post_request at 0x7f93d1e16020>
 child_exit: <function ChildExit.child_exit at 0x7f93d1e16160>
 worker_exit: <function WorkerExit.worker_exit at 0x7f93d1e162a0>
 nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7f93d1e163e0>
 on_exit: <function OnExit.on_exit at 0x7f93d1e16520>
 proxy_protocol: False
 proxy_allow_ips: ['127.0.0.1']
 keyfile: None
 certfile: None
 ssl_version: 2
 cert_reqs: 0
 ca_certs: None
 suppress_ragged_eofs: True
 do_handshake_on_connect: False
 ciphers: None
 raw_paste_global_conf: []
 strip_header_spaces: False
[2023-10-10 12:07:04 +0000] [4264] [INFO] Starting gunicorn 20.0.2
[2023-10-10 12:07:04 +0000] [4264] [DEBUG] Arbiter booted
[2023-10-10 12:07:04 +0000] [4264] [INFO] Listening at: unix:/run/anahit.sock (4264)
[2023-10-10 12:07:04 +0000] [4264] [INFO] Using worker: gevent
[2023-10-10 12:07:04 +0000] [4285] [INFO] Booting worker with pid: 4285
[2023-10-10 12:07:04 +0000] [4264] [DEBUG] 1 workers
[2023-10-10 12:07:21 +0000] [4285] [ERROR] Exception in worker process
Traceback (most recent call last):
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
  worker.init_process()
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/workers/ggevent.py", line 162, in init_process
  super().init_process()
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/workers/base.py", line 119, in init_process
  self.load_wsgi()
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
  self.wsgi = self.app.wsgi()
        ^^^^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/app/base.py", line 66, in wsgi
  self.callable = self.load()
          ^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
  return self.load_wsgiapp()
      ^^^^^^^^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
  return util.import_app(self.app_uri)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/util.py", line 358, in import_app
  mod = importlib.import_module(module)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
  return _bootstrap._gcd_import(name[level:], package, level)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
 File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
 File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 940, in exec_module
 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
 File "/home/anahit_web/anahit_website/app/anahit/wsgi.py", line 16, in <module>
  application = get_wsgi_application()
         ^^^^^^^^^^^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
  django.setup(set_prefix=False)
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
  apps.populate(settings.INSTALLED_APPS)
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/django/apps/registry.py", line 124, in populate
  app_config.ready()
 File "/home/anahit_web/anahit_website/app/apps/users/apps.py", line 9, in ready
  from . import signals
 File "/home/anahit_web/anahit_website/app/apps/users/signals.py", line 11, in <module>
  from common.utils import send_email_notification
 File "/home/anahit_web/anahit_website/app/common/utils.py", line 28, in <module>
  from discord_webhook import DiscordWebhook
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/discord_webhook/__init__.py", line 5, in <module>
  from .async_webhook import AsyncDiscordWebhook
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/discord_webhook/async_webhook.py", line 13, in <module>
  import httpx # noqa
  ^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpx/__init__.py", line 2, in <module>
  from ._api import delete, get, head, options, patch, post, put, request, stream
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpx/_api.py", line 4, in <module>
  from ._client import Client
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpx/_client.py", line 30, in <module>
  from ._transports.default import AsyncHTTPTransport, HTTPTransport
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpx/_transports/default.py", line 30, in <module>
  import httpcore
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpcore/__init__.py", line 1, in <module>
  from ._api import request, stream
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpcore/_api.py", line 5, in <module>
  from ._sync.connection_pool import ConnectionPool
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpcore/_sync/__init__.py", line 1, in <module>
  from .connection import HTTPConnection
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 12, in <module>
  from .._synchronization import Lock
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/httpcore/_synchronization.py", line 13, in <module>
  import trio
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/__init__.py", line 19, in <module>
  from ._core import TASK_STATUS_IGNORED as TASK_STATUS_IGNORED # isort: skip
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/_core/__init__.py", line 21, in <module>
  from ._local import RunVar
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/_core/_local.py", line 5, in <module>
  from . import _run
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/_core/_run.py", line 2543, in <module>
  from ._io_epoll import EpollIOManager as TheIOManager
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/_core/_io_epoll.py", line 189, in <module>
  class EpollIOManager:
 File "/home/anahit_web/anahit_311/lib/python3.11/site-packages/trio/_core/_io_epoll.py", line 190, in EpollIOManager
  _epoll = attr.ib(factory=select.epoll)
               ^^^^^^^^^^^^
AttributeError: module 'select' has no attribute 'epoll'

I am getting above error if I specify the workers in gunicorn config.

ExecStart= gunicorn_path \
          --log-level debug \
          --enable-stdio-inheritance \
          --capture-output \
          --access-logfile access_log_path \
          --error-logfile error_log_path\
          --worker-class gevent \
          --workers 2 \
          --timeout 120 \
          --bind unix: <socket> \
          project.wsgi:application

gunicorn service file

@altendky
Copy link
Member

altendky commented Nov 1, 2023

Are you running Linux? It appears so and https://docs.python.org/3.11/library/select.html#select.epoll reads like it should be fairly available. Maybe the wrong select module is getting loaded. Could you add raise Exception(str(select)) right before the failing line? That will tell where the loaded select module is located.

@kushalmraut
Copy link
Author

@altendky thank you for reply, Yes I am running linux based operating system.
I have added raise Exception(str(select)) right before the failing line Exception: <module 'select' from '/usr/local/lib/python3.11/lib-dynload/select.cpython-311-x86_64-linux-gnu.so'> is the exception I am getting.

@altendky
Copy link
Member

altendky commented Nov 2, 2023

So Trio does assume that Python will have select.epoll() in Linux.

https://github.com/python-trio/trio/blob/v0.22.2/trio/_core/_run.py#L2541-L2543

elif sys.platform == "linux" or (not TYPE_CHECKING and hasattr(select, "epoll")):
    from ._generated_io_epoll import *
    from ._io_epoll import EpollIOManager as TheIOManager

But Python checks to see if it is available on the system at compile time. It looks like this is set via the configure script.

https://github.com/python/cpython/blob/v3.11.6/Modules/selectmodule.c#L1230

#ifdef HAVE_EPOLL

Could you review this 'script' to make sure you are comfortable running it, and if so, share back the output? (after reviewing the output for anything you might be concerned about sharing :]) Consider the path to python and if that is wrong for the python you are using, correct it.

echo ---- python select && /usr/bin/python3.11 -c 'import select; print(select); print(getattr(select, "epoll", None))' && echo ---- python config && python -c 'import sysconfig; config = sysconfig.get_config_vars(); print({key: value for key, value in config.items() if "EPOLL" in key})' &&echo ---- uname && uname -a && echo ---- lsb_release && lsb_release -a && echo ---- boot config && grep -r EPOLL /boot/config* && echo ---- proc && grep -ri epoll /proc/sys/kernel/

For example, I get this output (for my Python 3.10 instead) showing that epoll seems to be enabled in both the kernel and in Python.

$ echo ---- python select && /usr/bin/python3.10 -c 'import select; print(select); print(getattr(select, "epoll", None))' && echo ---- python config && python -c 'import sysconfig; config = sysconfig.get_config_vars(); print({key: value for key, value in config.items() if "EPOLL" in key})' &&echo ---- uname && uname -a && echo ---- lsb_release && lsb_release -a && echo ---- boot config && grep -r EPOLL /boot/config* && echo ---- proc && grep -ri epoll /proc/sys/kernel/
---- python select
<module 'select' (built-in)>
<class 'select.epoll'>
---- python config
{'HAVE_EPOLL': 1, 'HAVE_EPOLL_CREATE1': 1, 'HAVE_SYS_EPOLL_H': 1}
---- uname
Linux p1 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
---- lsb_release
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy
---- boot config
/boot/config-5.19.0-46-generic:CONFIG_EPOLL=y
/boot/config-6.2.0-35-generic:CONFIG_EPOLL=y
/boot/config-6.2.0-36-generic:CONFIG_EPOLL=y
---- proc
grep: /proc/sys/kernel/apparmor_display_secid_mode: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns_complain: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns_force: Permission denied
grep: /proc/sys/kernel/cad_pid: Permission denied
grep: /proc/sys/kernel/unprivileged_userns_apparmor_policy: Permission denied
grep: /proc/sys/kernel/usermodehelper/bset: Permission denied
grep: /proc/sys/kernel/usermodehelper/inheritable: Permission denied

@jakkdl
Copy link
Member

jakkdl commented Nov 6, 2023

grep -ri epoll /proc/sys/kernel/

I get this output (for my Python 3.10 instead) showing that epoll seems to be enabled in both the kernel and in Python.

grep: /proc/sys/kernel/apparmor_display_secid_mode: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns_complain: Permission denied
grep: /proc/sys/kernel/apparmor_restrict_unprivileged_userns_force: Permission denied
grep: /proc/sys/kernel/cad_pid: Permission denied
grep: /proc/sys/kernel/unprivileged_userns_apparmor_policy: Permission denied
grep: /proc/sys/kernel/usermodehelper/bset: Permission denied
grep: /proc/sys/kernel/usermodehelper/inheritable: Permission denied

no, you're just getting a bunch of permission errors. Running it on my system with sudo I'm not getting any output, and the boot config similarly doesn't work on my system. (Arch Linux)

zcat /proc/config.gz | grep -i EPOLL

Was suggested by the internet for checking EPOLL availability though, and works on my system - so @kushalmraut might want to run that in addition and check the output in case their system differs from altkendy.

@kushalmraut
Copy link
Author

@altendky

Output for the script is:

$ echo ---- python select && /home/anahit_web/venv_311/bin/python -c 'import select; print(select); print(getattr(select, "epoll", None))' && echo ---- python config && python -c 'import sysconfig; config = sysconfig.get_config_vars(); print({key: value for key, value in config.items() if "EPOLL" in key})' &&echo ---- uname && uname -a && echo ---- lsb_release && lsb_release -a && echo ---- boot config && grep -r EPOLL /boot/config* && echo ---- proc && grep -ri epoll /proc/sys/kernel/
---- python select
<module 'select' from '/usr/local/lib/python3.11/lib-dynload/select.cpython-311-x86_64-linux-gnu.so'>
<class 'select.epoll'>
---- python config
{'HAVE_EPOLL': 1, 'HAVE_EPOLL_CREATE1': 1, 'HAVE_SYS_EPOLL_H': 1}
---- uname
Linux ip-172-31-26-228 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
---- lsb_release
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.6 LTS
Release:	18.04
Codename:	bionic
---- boot config
/boot/config-5.4.0-1060-aws:CONFIG_EPOLL=y
/boot/config-5.4.0-1094-aws:CONFIG_EPOLL=y
/boot/config-5.4.0-1101-aws:CONFIG_EPOLL=y
/boot/config-5.4.0-1103-aws:CONFIG_EPOLL=y
---- proc
grep: /proc/sys/kernel/cad_pid: Permission denied
grep: /proc/sys/kernel/unprivileged_userns_apparmor_policy: Permission denied
grep: /proc/sys/kernel/usermodehelper/bset: Permission denied
grep: /proc/sys/kernel/usermodehelper/inheritable: Permission denied

@kushalmraut
Copy link
Author

@jakkdl
I couldn't find /proc/config.gz a file.

@A5rocks
Copy link
Contributor

A5rocks commented Nov 7, 2023

I'm struggling to reconcile:

  _epoll = attr.ib(factory=select.epoll)
               ^^^^^^^^^^^^
AttributeError: module 'select' has no attribute 'epoll'

And yet getattr(select, "epoll", None) seems to work...

(I don't have anything productive to add, just found this puzzling).

@jakkdl
Copy link
Member

jakkdl commented Nov 10, 2023

yeah this is very strange. It feels like there might be a bug in gunicorn or something being misconfigured somehow such that the python/select.epoll being invoked by trio is not the same python as the one that could access it np from the command line. I think you'll have to do further debugging on your end, e.g. liberally adding print(getattr(select, "epoll")) to your code and/or the installed library code of gunicorn/trio.
It does not really sound like a problem on trios end though, you could try asking folks from gunicorn.

@kushalmraut
Copy link
Author

Please let me elaborate in detail.
I started getting this error when I upgraded python version for my project from 3.9 to 3.11 and my django version 3.2 to 4.2.

This is my os

Linux #111~18.04.1-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

when i specify workers in gunicorn configurations i get this error.

--worker-class gevent \
--workers 2 \

I search for this issue and followed following links but unable to fix it.
I have also raised the issue on gunicorn github.

followed references.
https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1918516.html
urllib3/urllib3#1104

Please let me know if you find solution or root of cause.

@jakkdl
Copy link
Member

jakkdl commented Nov 24, 2023

I started getting this error when I upgraded python version for my project from 3.9 to 3.11 and my django version 3.2 to 4.2.

hm, is it a problem with virtualenvs or similar? Some questions to help you narrow down the problem: Do you also get it in CI? Tried a clean virtualenv? What distro? How did you do the python version upgrade? If you create a new project can you reproduce the problem? Does the problem disappear if you revert the upgrade?

@kushalmraut
Copy link
Author

kushalmraut commented Nov 27, 2023

I started getting this error when I upgraded python version for my project from 3.9 to 3.11 and my django version 3.2 to 4.2.

hm, is it a problem with virtualenvs or similar? Some questions to help you narrow down the problem: Do you also get it in CI? Tried a clean virtualenv? What distro? How did you do the python version upgrade? If you create a new project can you reproduce the problem? Does the problem disappear if you revert the upgrade?

I don't have CL if you're referring to the continuous integration.

I have created a new virtualenv with python 3.11 while upgrading the packages instate of upgrading previous virtualenv.

distro

VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

I have installed a 3.11 separately used this reference for python installation

The issue is reproduced when new project is created. I have two servers stage and main and issue is for both.

The problem is disappear when i switch to the old virtualenv.

@jakkdl
Copy link
Member

jakkdl commented Nov 27, 2023

I think you'll have to do further debugging on your end, e.g. liberally adding print(getattr(select, "epoll")) to your code and/or the installed library code of gunicorn/trio.

did you ever try this?
Clarifying the last part, open up files like /home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/arbiter.py in a text editor and modify them to add print statements and the like. The backtrace is pretty huge so you might have to modify a lot of places to figure out if/when the getattr call breaks.

@kushalmraut
Copy link
Author

I think you'll have to do further debugging on your end, e.g. liberally adding print(getattr(select, "epoll")) to your code and/or the installed library code of gunicorn/trio.

did you ever try this? Clarifying the last part, open up files like /home/anahit_web/anahit_311/lib/python3.11/site-packages/gunicorn/arbiter.py in a text editor and modify them to add print statements and the like. The backtrace is pretty huge so you might have to modify a lot of places to figure out if/when the getattr call breaks.

Yes I have tried adding print(getattr(select, "epoll")) before _epoll = attr.ib(factory=select.epoll) it is giving the same error.

File "/home/anahit_web/venv_311/lib/python3.11/site-packages/trio/_core/_io_epoll.py", line 190, in EpollIOManager
  print('getattr(select, "epoll")*****************', getattr(select, "epoll"))
                            ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'select' has no attribute 'epoll'

I have also deleted the python 3.11 and old vertualenv and installed python 3.11 again and created new vertualenv and tried it again but getting the same error.

I have printed dir(select) before the line causing the error. below is the output this might help.

['EPOLLERR', 'EPOLLET', 'EPOLLEXCLUSIVE', 'EPOLLHUP', 'EPOLLIN', 'EPOLLMSG', 'EPOLLONESHOT', 'EPOLLOUT', 'EPOLLPRI', 'EPOLLRDBAND', 'EPOLLRDHUP', 'EPOLLRDNORM', 'EPOLLWRBAND', 'EPOLLWRNORM', 'EPOLL_CLOEXEC', 'PIPE_BUF', 'POLLERR', 'POLLHUP', 'POLLIN', 'POLLMSG', 'POLLNVAL', 'POLLOUT', 'POLLPRI', 'POLLRDBAND', 'POLLRDHUP', 'POLLRDNORM', 'POLLWRBAND', 'POLLWRNORM', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'error', 'poll', 'select']

@jakkdl
Copy link
Member

jakkdl commented Nov 28, 2023

Yes I have tried adding print(getattr(select, "epoll")) before _epoll = attr.ib(factory=select.epoll) it is giving the same error.

and then add it to every other location in the backtrace, to figure out where it breaks. The big question mark is why you can run it in the python prompt w/o error but it breaking deep down in the call stack when trio tries to do it

@jeongilpark
Copy link

jeongilpark commented Jan 17, 2024

I've dug this problem and got this result.

I add print statement into patch_all() in site-packages/gevent/monkey.py, like this

    if select:
        import select
        print('flag1', getattr(select, 'epoll'))
        patch_select(aggressive=aggressive)
        print('flag2', getattr(select, 'epoll'))
        patch_selectors(aggressive=aggressive)
        print('flag3', getattr(select, 'epoll'))

And got this output

[2024-01-17 15:25:16 +0900] [97818] [INFO] Starting gunicorn 21.2.0
[2024-01-17 15:25:16 +0900] [97818] [DEBUG] Arbiter booted
[2024-01-17 15:25:16 +0900] [97818] [INFO] Listening at: http://0.0.0.0:5000 (97818)
[2024-01-17 15:25:16 +0900] [97818] [INFO] Using worker: gevent
[2024-01-17 15:25:16 +0900] [97847] [INFO] Booting worker with pid: 97847
flag1 <class 'select.epoll'>
[2024-01-17 15:25:16 +0900] [97818] [DEBUG] 1 workers
[2024-01-17 15:25:16 +0900] [97847] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/gunicorn/arbiter.py", line 610, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.12/site-packages/gunicorn/workers/ggevent.py", line 145, in init_process
    self.patch()
  File "/usr/local/lib/python3.12/site-packages/gunicorn/workers/ggevent.py", line 39, in patch
    monkey.patch_all()
  File "/usr/local/lib/python3.12/site-packages/gevent/monkey.py", line 1235, in patch_all
    print('flag2', getattr(select, 'epoll'))
                   ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'select' has no attribute 'epoll'. Did you mean: 'poll'?

But I'm not good at gevent and what happens in patch_select() and why...
Do you have any idea?


Exactly same code works with Python 3.9.16. But not with 3.11.7 (nor 3.12.1). Same gevent version (23.9.1) is used.

@kushalmraut
Copy link
Author

@jeongilpark

select module has been monkey patched so if you are using gevent and gunicorn this error will occure in runtime.

gevent/gevent#2008

@jeongilpark
Copy link

Got it.
The problem is that Gunicorn, gevent and trio do not work together. Correct?
I just found that upgrading selenium version to 4.16.0, which requires trio, makes my problem.

@A5rocks
Copy link
Contributor

A5rocks commented Jan 17, 2024

Oh huh I see the problem now.

I think a solution would be to use a lambda and reference select.epoll inside (assuming you don't actually end up using trio?)

But a workaround as a user might be to just import trio first thing before gevent's monkey patching and then accept that it'll have bad state? I don't know gunicorn well enough to know if that's possible.

@matt-vianai
Copy link

Running: pip uninstall -y httpcore fixed the issue for me

@Robert-Beier
Copy link

I solved my problem by uninstalling selenium that included trio. I was lucky that my flask app didn't actually need these. They were included in my requirements.txt file for other scripts in my repo.

The comment from @jeongilpark sparked the thought. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants