gevent patching order is documented incorrectly #8763
Comments
|
@matjam As pointed out in the documentation, we recommend importing and report whether that solved your problem? Thanks 🙏 . |
|
@P403n1x87 What you're saying is, we need to do the following? If so, the docs are not clear on this at all. Patching in this order results in the following startup errors: And then later when our config class tries to access KMS we get We're using 3.9.18. The gevent docs are clear in that if you see these errors its because you're patching things too late. The above happens with the following startup code: @click.command()
@click.option("--entry-name", default="app", type=click.STRING)
@click.argument("gunicorn_args", nargs=-1, type=click.UNPROCESSED)
def run_app(entry_name, gunicorn_args):
# we need to patch gevent before anything at all is imported
log.info("Starting Redacted ...")
config = SimpleConfig(log)
gevent_enabled: bool = config.get("GUNICORN.worker_class") == "gevent"
datadog_apm_enabled: bool = config.get("DATADOG.APM_ENABLED")
datadog_profiling_enabled: bool = config.get("DATADOG.PROFILING_ENABLED")
if datadog_apm_enabled:
log.info("datadog APM enabled, doing datadog monkey patching ...")
import ddtrace.auto # noqa
from ddtrace import patch_all
if datadog_profiling_enabled:
log.info("Enabling Datadog Profiling...")
import ddtrace.profiling.auto # noqa
patch_all(gevent=gevent_enabled, psycopg=True, elasticsearch=True, urllib3=True, botocore=True)
if gevent_enabled:
log.info("gevent worker enabled, doing gevent monkey patching...")
from gevent import monkey
import psycogreen.gevent
monkey.patch_all()
log.info("Using psycogreen to monkey patch psycopg2.")
psycogreen.gevent.patch_psycopg()
try:
from . import RedactedApplication
application = RedactedApplication(
config.get("_meta.environment"),
entry_name=entry_name,
gunicorn_args=gunicorn_args,
)
except Exception as e:
print(f"ERROR: exception starting RedactedAppliction: {e}")
sys.exit(1)
application.run()note the I've tried moving the |
|
So looks like there were some importsin import logging
import os
import sys
import click
from .simpleconfig import SimpleConfig
# in our quest to ensure NOTHING but gevent gets to patch things first,
# we implement a very simple logger that only logs to stdout. This is
# replaced by the custom logger later.
formatter = logging.Formatter(
fmt="[%(asctime)s] [%(process)d] [%(levelname)s] %(message)s", datefmt="%Y-%m-%d %H:%M:%S %z"
)
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(formatter)
log = logging.getLogger(__name__)
log.addHandler(handler)
log.setLevel(logging.INFO)
@click.command()
@click.option("--entry-name", default="app", type=click.STRING)
@click.argument("gunicorn_args", nargs=-1, type=click.UNPROCESSED)
def run_app(entry_name, gunicorn_args):
# based on a discussion in https://github.com/DataDog/dd-trace-py/issues/8763
#
# We import and patch with ddtrace.auto first, then we patch with the
# ddtrace.patch function. This is because ddtrace.auto does not patch
# urllib3 and botocore, which are used by the application.
log.info("Starting Redacted ...")
config = SimpleConfig(log)
gevent_enabled: bool = config.get("GUNICORN.worker_class") == "gevent"
datadog_apm_enabled: bool = config.get("DATADOG.APM_ENABLED")
datadog_profiling_enabled: bool = config.get("DATADOG.PROFILING_ENABLED")
if datadog_apm_enabled:
log.info("Datadog APM enabled, doing datadog monkey patching ...")
import ddtrace.auto # noqa
from ddtrace import patch
patch(urllib3=True, botocore=True)
if gevent_enabled:
log.info("gevent worker enabled, doing gevent monkey patching...")
import gevent.monkey
import psycogreen.gevent
gevent.monkey.patch_all(thread=False, socket=False)
log.info("gevent monkey patching psycopg2 ...")
psycogreen.gevent.patch_psycopg()
if datadog_profiling_enabled:
log.info("Enabling Datadog Profiling...")
import ddtrace.profiling.auto # noqa
try:
from .application import RedactedApplication
application = RedactedApplication(
config.get("_meta.environment"),
entry_name=entry_name,
gunicorn_args=gunicorn_args,
)
except Exception as e:
print(f"ERROR: exception starting RedactedAppliction: {e}")
sys.exit(1)
log.info("Running application.")
application.run()This seems to work, though we do see one warning now: When I added tracing to the gevent class GeventWorker(AsyncWorker):
server_class = None
wsgi_handler = None
def patch(self):
monkey.patch_all()I believe this is harmless. This change does remove an exception we were seeing at startup: |
|
Oh, its also important to note gevent.monkey.patch_all(thread=False, socket=False)this was required, or the application would hang on startup. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In an earlier issue
#3595 (comment)
it is noted that
yet there is a note in the documentation that states
However, when we do this, we get this error:
clearly, patching ddtrace BEFORE gevent WILL NOT WORK as there are dependencies inside ddtrace that are importing the
ssllibrary.If the correct order is indeed supposed to be ddtrace before gevent, then that means ddtrace needs to be smarter.
if I reverse the above to patch gevent first, the above error goes away. Additionally the app starts.
The text was updated successfully, but these errors were encountered: