Fixes for Django 5.0. #1606

carltongibson · 2023-08-31T15:37:07Z

b035bf2 is needed to surface the ChoiceIterator issue, otherwise we hit apps not loaded setting up the test run.

Found 77 test(s).
Creating test database for alias 'default'...
Traceback (most recent call last):
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 107, in handle
    for app_config in apps.get_app_configs():
                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/apps/registry.py", line 147, in get_app_configs
    self.check_apps_ready()
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/apps/registry.py", line 138, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/trans_real.py", line 215, in _add_installed_apps_translations
    app_configs = reversed(apps.get_app_configs())
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/apps/registry.py", line 147, in get_app_configs
    self.check_apps_ready()
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/apps/registry.py", line 138, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/carlton/Projects/Django/django-filter/runtests.py", line 15, in <module>
    runtests()
  File "/Users/carlton/Projects/Django/django-filter/runtests.py", line 11, in runtests
    execute_from_command_line(argv)
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/commands/test.py", line 24, in run_from_argv
    super().run_from_argv(argv)
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/commands/test.py", line 68, in handle
    failures = test_runner.run_tests(test_labels)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/test/runner.py", line 1056, in run_tests
    old_config = self.setup_databases(
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/test/runner.py", line 955, in setup_databases
    return _setup_databases(
           ^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/test/utils.py", line 203, in setup_databases
    connection.creation.create_test_db(
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/db/backends/base/creation.py", line 78, in create_test_db
    call_command(
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/__init__.py", line 194, in call_command
    return command.execute(*args, **defaults)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/core/management/base.py", line 109, in wrapper
    translation.activate(saved_locale)
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/__init__.py", line 181, in activate
    return _trans.activate(language)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/trans_real.py", line 303, in activate
    _active.value = translation(language)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/trans_real.py", line 292, in translation
    _translations[language] = DjangoTranslation(language)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/trans_real.py", line 169, in __init__
    self._add_installed_apps_translations()
  File "/Users/carlton/Projects/Django/django-filter/.tox/py311-latest/lib/python3.11/site-packages/django/utils/translation/trans_real.py", line 217, in _add_installed_apps_translations
    raise AppRegistryNotReady(
django.core.exceptions.AppRegistryNotReady: The translation infrastructure cannot be initialized before the apps registry is ready. Check that you don't make non-lazy gettext calls at import time.
py311-latest: exit 1 (0.35 seconds) /Users/carlton/Projects/Django/django-filter> coverage run --parallel-mode --source django_filters ./runtests.py --testrunner xmlrunner.extra.djangotestrunner.XMLTestRunner pid=84393

(@felixxm don't suppose you know when this snuck in do you? Ta! 😊)

codecov-commenter · 2023-08-31T15:38:33Z

Codecov Report

Merging #1606 (f279fb7) into main (e5fc05d) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1606      +/-   ##
==========================================
+ Coverage   98.58%   98.59%   +0.01%     
==========================================
  Files          15       15              
  Lines        1271     1285      +14     
==========================================
+ Hits         1253     1267      +14     
  Misses         18       18

Files Changed	Coverage Δ
django_filters/fields.py	`100.00% <100.00%> (ø)`

carltongibson · 2023-09-01T09:38:07Z

Hey @ngnpope 👋 Don't suppose I could ask for your opinion here could I please? 🎁

carltongibson · 2023-09-01T09:46:44Z

b035bf2 is needed to surface the ChoiceIterator issue, otherwise we hit apps not loaded setting up the test run.

This only comes up in tox. (Grrr.) Calling django.setup() is unobjectionable, and probably should have always been there… 🤔, so unless someone wants to input I'm just going to leave it.

ngnpope · 2023-09-01T10:06:21Z

Hey @ngnpope 👋 Don't suppose I could ask for your opinion here could I please? 🎁

Will take a look... 👀

ngnpope · 2023-09-01T10:44:34Z

runtests.py

 from django.core.management import execute_from_command_line


 def runtests():
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "tests.settings")
+    django.setup()


I believe this was due to a deep circular import issue that didn't flag up when running Django's test suite.

See django/django#17215 for where this was fixed.

ngnpope · 2023-09-01T10:46:17Z

django_filters/fields.py

+# Compat for Django >= 5.0
+try:
+    from django.utils.choices import normalize_choices
+except ImportError:
+    normalize_choices = None
+
+DJANGO_50 = normalize_choices is not None
+


I think this could be simplified:

Suggested change

# Compat for Django >= 5.0

try:

from django.utils.choices import normalize_choices

except ImportError:

normalize_choices = None

DJANGO_50 = normalize_choices is not None

try:

from django.utils.choices import normalize_choices

except ImportError:

DJANGO_50 = False

else:

DJANGO_50 = True

ngnpope · 2023-09-01T11:44:52Z

django_filters/fields.py

+        @forms.ChoiceField.choices.setter
+        def choices(self, value):
+            self._choices = self.widget.choices = self.iterator(
+                self, normalize_choices(value)


There are a few interesting things here:

Ideally we'd not rehash the Django internals here, i.e. self._choices = self.widget.choices = ...

For equivalent behaviour as in Django 5.0 it ought to be ... = normalize_choices(self.iterator(...))

To make the normalization lazy, ChoiceIterator should extend django.utils.choices.ChoiceIterator

Maybe I should have called that django.utils.choices.BaseChoiceIterator? 🤔

We should push the normalize_choices() call into ChoiceIterator.__iter__()

It's a shame that we have to implement this twice rather than once in the mixin

I think all of this can be solved. Will open an alternate PR soon...

ngnpope · 2023-09-01T12:25:16Z

See #1607 for an alternative 🙂

carltongibson · 2023-09-01T12:28:21Z

Thanks @ngnpope 🦸‍♀️ Looks good. I'll give it a proper read over the weekend. 🎁

carltongibson · 2023-09-04T09:03:25Z

Closing in favour of #1607

carltongibson added 2 commits September 1, 2023 08:57

Add django.setup() call to runtests.py.

d03ba6e

Initial pass at 5.0 compat.

0b42447

carltongibson force-pushed the fixes-for-django-5.0 branch from b035bf2 to 0b42447 Compare September 1, 2023 09:32

Re-require passing tests against Django main.

f279fb7

carltongibson mentioned this pull request Sep 1, 2023

Fix CallableChoiceIterator import on Django 5.0+ wagtail/wagtail#10846

Merged

carltongibson linked an issue Sep 1, 2023 that may be closed by this pull request

Choice iterator breakage against Django pre-5.0 #1605

Closed

ngnpope reviewed Sep 1, 2023

View reviewed changes

ngnpope mentioned this pull request Sep 1, 2023

Added support for Django 5.0. #1607

Merged

carltongibson closed this Sep 4, 2023

carltongibson deleted the fixes-for-django-5.0 branch September 4, 2023 09:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes for Django 5.0. #1606

Fixes for Django 5.0. #1606

carltongibson commented Aug 31, 2023 •

edited

codecov-commenter commented Aug 31, 2023 •

edited

carltongibson commented Sep 1, 2023

carltongibson commented Sep 1, 2023 •

edited

ngnpope commented Sep 1, 2023

ngnpope Sep 1, 2023

ngnpope Sep 1, 2023

ngnpope Sep 1, 2023

ngnpope commented Sep 1, 2023

carltongibson commented Sep 1, 2023

carltongibson commented Sep 4, 2023

Fixes for Django 5.0. #1606

Fixes for Django 5.0. #1606

Conversation

carltongibson commented Aug 31, 2023 • edited

codecov-commenter commented Aug 31, 2023 • edited

Codecov Report

carltongibson commented Sep 1, 2023

carltongibson commented Sep 1, 2023 • edited

ngnpope commented Sep 1, 2023

ngnpope Sep 1, 2023

Choose a reason for hiding this comment

ngnpope Sep 1, 2023

Choose a reason for hiding this comment

ngnpope Sep 1, 2023

Choose a reason for hiding this comment

ngnpope commented Sep 1, 2023

carltongibson commented Sep 1, 2023

carltongibson commented Sep 4, 2023

carltongibson commented Aug 31, 2023 •

edited

codecov-commenter commented Aug 31, 2023 •

edited

carltongibson commented Sep 1, 2023 •

edited