Celery Import Error

[marimuthuei]

Not able to import Celery module when creating simple app.

from celery import Celery
ImportError: cannot import name 'Celery' from 'celery'

Additonal info:
This issue doesn't occur when we downgrade importlib-metadata to 4.12.0
Env details

[open-collective-bot]

Hey @marimuthuei 👋,
Thank you for opening an issue. We will get back to you as soon as we can.
Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors.
If you require immediate assistance please consider sponsoring us.

[woutdenolf]

What python version? Seems to occur only with python 3.7 #7784

[woutdenolf]

Ok I guess because importlib-metadata is not installed when you install celery in a python 3.8 environment.

[ppanero]

similar imports are failing (e.g. current_app) with the same behavior (works when downgrading importlib-metadata). Seems to only be in python 3.7.

Tests with py3.8 and importlib-metadata works for me.

[marimuthuei]

its python3.7 env

[woutdenolf]

This seems to be the cause: python/importlib_metadata#405

Successfully installed importlib_metadata-4.13.0
$ python
Python 3.7.9 (default, Jul  5 2021, 22:17:33) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from importlib_metadata import entry_points
>>> entry_points().get("celery.result_backends", list())
__main__:1: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
[]

So I guess we should fix

celery/celery/utils/imports.py

Lines 142 to 145 in 59263b0

	def load_extension_class_names(namespace):
	for ep in entry_points().get(namespace, []):
	yield ep.name, ep.value

[woutdenolf]

The kombu package probably has the same problem

https://github.com/celery/kombu/blob/97e887605b21a110e0ec304913930a8115987c6a/kombu/utils/compat.py#L93

[woutdenolf]

Made a fix for celery (#7785) and kombu (celery/kombu#1601).

[woutdenolf]

importlib-metadata<5.0 is an alternative solution: #7782 and celery/kombu#1602

[marimuthuei]

@auvipy Can this issue will be resolved in celery ==4.4.7 version?

[auvipy]

nope

[jaraco]

@auvipy Can this issue will be resolved in celery ==4.4.7 version?

As mentioned above, pinning to importlib_metadata < 5 in your environment should work around the issue while keeping celery == 4.4.7.

[Nusnus]

Hey guys, the PR associated with this issue introduced a bug and the documentation cannot be built.
Relevant commit: f77e98a
Doc build error from this PR: https://readthedocs.org/projects/celery/builds/18287948/

Please revert until a proper fix is available, we cannot have breaking code in master.
@woutdenolf @auvipy

[auvipy]

Hey nusnus, the actual PR was squashed merged a4545db , the commit you are refering should have the fix with the commit merged a4545db . but if it still have the element of failure, we will revert to fix it.

[woutdenolf]

https://readthedocs.org/projects/celery/builds/18287948/

  File "/home/docs/checkouts/readthedocs.org/user_builds/celery/envs/7785/lib/python3.7/site-packages/kombu/utils/compat.py", line 82, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
AttributeError: 'EntryPoints' object has no attribute 'get'

But we capture AttributeError

try:
    _entry_points = entry_points().get(namespace, [])
except AttributeError:
    _entry_points = entry_points().select(group=namespace)

So maybe you're looking at an old CI run?

[auvipy]

https://readthedocs.org/projects/celery/builds/18287948/

  File "/home/docs/checkouts/readthedocs.org/user_builds/celery/envs/7785/lib/python3.7/site-packages/kombu/utils/compat.py", line 82, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
AttributeError: 'EntryPoints' object has no attribute 'get'

But we capture AttributeError

try:
    _entry_points = entry_points().get(namespace, [])
except AttributeError:
    _entry_points = entry_points().select(group=namespace)

So maybe you're looking at an old CI run?

i have opened a revert pr, there the doc build is still failing, can you cross check please?

[woutdenolf]

Ah wait:

  File "/home/docs/checkouts/readthedocs.org/user_builds/celery/envs/7785/lib/python3.7/site-packages/kombu/utils/compat.py", line 82, in entrypoints
    for ep in importlib_metadata.entry_points().get(namespace, [])
AttributeError: 'EntryPoints' object has no attribute 'get'

This is an error from kombu. So it won't pass until we release kombu (fixed in celery/kombu#1601).

[auvipy]

oh i see! a new beta of kombu should be released, I will do that

[auvipy]

#7834 testing

[woutdenolf]

Apart from kombu, it seems readthedocs was not using requirements/docs.txt. This fixes the doc building: #7835

[Nusnus]

Thank you @woutdenolf for investigating this and coming up with a solution quickly! Appreciate it. Hope it will get merged soon :)

[titusfx]

@jaraco downgrading importlib_metadata didn't work for me :(

In my case, I had a file that was called queue.py