Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons #8209

alexmojaki · 2023-11-22T19:12:16Z

(previously #7958)

Replaces all calls (outside v1) to typing._eval_type with the new typing_extra.eval_type_backport which falls back to https://github.com/alexmojaki/eval_type_backport if installed.

… properly, clean up type ignore comments

…ckport

codspeed-hq · 2023-11-22T19:15:40Z

CodSpeed Performance Report

Merging #8209 will not alter performance

_{Comparing alexmojaki:eval_type_backport (60aa70f) with main (5de67f7)}

Summary

✅ 10 untouched benchmarks

alexmojaki · 2023-11-22T19:21:31Z

please review

pydantic/_internal/_typing_extra.py

pyproject.toml

tests/test_types.py

dmontagu

Suggestions made above, I think at least the ImportError should be handled, if you disagree or I'm misunderstanding something though let me know.

…port function docstring

…t_recursive_models_union passes without backport

…icRecursiveRef.__or__

…skipping one more test

bswck · 2023-12-23T13:04:55Z

You might want to check out the current discussion on built-in generic's subclasses:
alexmojaki/eval_type_backport#9

…ckport

samuelcolvin · 2024-01-16T16:37:34Z

pydantic/_internal/_typing_extra.py

+        try:
+            from eval_type_backport import eval_type_backport
+        except ImportError:
+            raise RuntimeError(


Suggested change

raise RuntimeError(

raise ImportError(

???

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [pydantic](https://github.com/pydantic/pydantic) ([changelog](https://docs.pydantic.dev/latest/changelog/)) | dependencies | minor | `2.5.3` -> `2.6.1` | --- ### Release Notes <details> <summary>pydantic/pydantic (pydantic)</summary> ### [`v2.6.1`](https://github.com/pydantic/pydantic/blob/HEAD/HISTORY.md#v261-2024-02-05) [Compare Source](pydantic/pydantic@v2.6.0...v2.6.1) [GitHub release](https://github.com/pydantic/pydantic/releases/tag/v2.6.1) ##### What's Changed ##### Packaging - Upgrade to `pydantic-core` 2.16.2 by [@sydney-runkle](https://github.com/sydney-runkle) in [#8717](pydantic/pydantic#8717) ##### Fixes - Fix bug with `mypy` plugin and `no_strict_optional = True` by [@dmontagu](https://github.com/dmontagu) in [#8666](pydantic/pydantic#8666) - Fix `ByteSize` error `type` change by [@sydney-runkle](https://github.com/sydney-runkle) in [#8681](pydantic/pydantic#8681) - Fix inheriting `Field` annotations in dataclasses by [@sydney-runkle](https://github.com/sydney-runkle) in [#8679](pydantic/pydantic#8679) - Fix regression in core schema generation for indirect definition references by [@dmontagu](https://github.com/dmontagu) in [#8702](pydantic/pydantic#8702) - Fix unsupported types bug with `PlainValidator` by [@sydney-runkle](https://github.com/sydney-runkle) in [#8710](pydantic/pydantic#8710) - Reverting problematic fix from 2.6 release, fixing schema building bug by [@sydney-runkle](https://github.com/sydney-runkle) in [#8718](pydantic/pydantic#8718) - Fix warning for tuple of wrong size in `Union` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1174](pydantic/pydantic-core#1174) - Fix `computed_field` JSON serializer `exclude_none` behavior by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1187](pydantic/pydantic-core#1187) ### [`v2.6.0`](https://github.com/pydantic/pydantic/blob/HEAD/HISTORY.md#v260-2024-01-23) [Compare Source](pydantic/pydantic@v2.5.3...v2.6.0) [GitHub release](https://github.com/pydantic/pydantic/releases/tag/v2.6.0) The code released in v2.6.0 is practically identical to that of v2.6.0b1. ##### What's Changed ##### Packaging - Check for `email-validator` version >= 2.0 by [@commonism](https://github.com/commonism) in [#6033](pydantic/pydantic#6033) - Upgrade \`ruff\`\` target version to Python 3.8 by [@Elkiwa](https://github.com/Elkiwa) in [#8341](pydantic/pydantic#8341) - Update to `pydantic-extra-types==2.4.1` by [@yezz123](https://github.com/yezz123) in [#8478](pydantic/pydantic#8478) - Update to `pyright==1.1.345` by [@Viicos](https://github.com/Viicos) in [#8453](pydantic/pydantic#8453) - Update pydantic-core from 2.14.6 to 2.16.1, significant changes from these updates are described below, full changelog [here](pydantic/pydantic-core@v2.14.6...v2.16.1) ##### New Features - Add `NatsDsn` by [@ekeew](https://github.com/ekeew) in [#6874](pydantic/pydantic#6874) - Add `ConfigDict.ser_json_inf_nan` by [@davidhewitt](https://github.com/davidhewitt) in [#8159](pydantic/pydantic#8159) - Add `types.OnErrorOmit` by [@adriangb](https://github.com/adriangb) in [#8222](pydantic/pydantic#8222) - Support `AliasGenerator` usage by [@sydney-runkle](https://github.com/sydney-runkle) in [#8282](pydantic/pydantic#8282) - Add Pydantic People Page to docs by [@sydney-runkle](https://github.com/sydney-runkle) in [#8345](pydantic/pydantic#8345) - Support `yyyy-MM-DD` datetime parsing by [@sydney-runkle](https://github.com/sydney-runkle) in [#8404](pydantic/pydantic#8404) - Added bits conversions to the `ByteSize` class [#8415](pydantic/pydantic#8415) by [@luca-matei](https://github.com/luca-matei) in [#8507](pydantic/pydantic#8507) - Enable json schema creation with type `ByteSize` by [@geospackle](https://github.com/geospackle) in [#8537](pydantic/pydantic#8537) - Add `eval_type_backport` to handle union operator and builtin generic subscripting in older Pythons by [@alexmojaki](https://github.com/alexmojaki) in [#8209](pydantic/pydantic#8209) - Add support for `dataclass` fields `init` by [@dmontagu](https://github.com/dmontagu) in [#8552](pydantic/pydantic#8552) - Implement pickling for `ValidationError` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1119](pydantic/pydantic-core#1119) - Add unified tuple validator that can handle "variadic" tuples via PEP-646 by [@dmontagu](https://github.com/dmontagu) in [pydantic/pydantic-core#865](pydantic/pydantic-core#865) ##### Changes - Drop Python3.7 support by [@hramezani](https://github.com/hramezani) in [#7188](pydantic/pydantic#7188) - Drop Python 3.7, and PyPy 3.7 and 3.8 by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1129](pydantic/pydantic-core#1129) - Use positional-only `self` in `BaseModel` constructor, so no field name can ever conflict with it by [@ariebovenberg](https://github.com/ariebovenberg) in [#8072](pydantic/pydantic#8072) - Make `@validate_call` return a function instead of a custom descriptor - fixes binding issue with inheritance and adds `self/cls` argument to validation errors by [@alexmojaki](https://github.com/alexmojaki) in [#8268](pydantic/pydantic#8268) - Exclude `BaseModel` docstring from JSON schema description by [@sydney-runkle](https://github.com/sydney-runkle) in [#8352](pydantic/pydantic#8352) - Introducing `classproperty` decorator for `model_computed_fields` by [@Jocelyn-Gas](https://github.com/Jocelyn-Gas) in [#8437](pydantic/pydantic#8437) - Explicitly raise an error if field names clashes with types by [@Viicos](https://github.com/Viicos) in [#8243](pydantic/pydantic#8243) - Use stricter serializer for unions of simple types by [@alexdrydew](https://github.com/alexdrydew) [pydantic/pydantic-core#1132](pydantic/pydantic-core#1132) ##### Performance - Add Codspeed profiling Actions workflow by [@lambertsbennett](https://github.com/lambertsbennett) in [#8054](pydantic/pydantic#8054) - Improve `int` extraction by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1155](pydantic/pydantic-core#1155) - Improve performance of recursion guard by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1156](pydantic/pydantic-core#1156) - `dataclass` serialization speedups by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1162](pydantic/pydantic-core#1162) - Avoid `HashMap` creation when looking up small JSON objects in `LazyIndexMaps` by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/jiter#55](pydantic/jiter#55) - use hashbrown to speedup python string caching by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/jiter#51](pydantic/jiter#51) - Replace `Peak` with more efficient `Peek` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/jiter#48](pydantic/jiter#48) ##### Fixes - Move `getattr` warning in deprecated `BaseConfig` by [@tlambert03](https://github.com/tlambert03) in [#7183](pydantic/pydantic#7183) - Only hash `model_fields`, not whole `__dict__` by [@alexmojaki](https://github.com/alexmojaki) in [#7786](pydantic/pydantic#7786) - Fix mishandling of unions while freezing types in the `mypy` plugin by [@dmontagu](https://github.com/dmontagu) in [#7411](pydantic/pydantic#7411) - Fix `mypy` error on untyped `ClassVar` by [@vincent-hachin-wmx](https://github.com/vincent-hachin-wmx) in [#8138](pydantic/pydantic#8138) - Only compare pydantic fields in `BaseModel.__eq__` instead of whole `__dict__` by [@QuentinSoubeyranAqemia](https://github.com/QuentinSoubeyranAqemia) in [#7825](pydantic/pydantic#7825) - Update `strict` docstring in `model_validate` method. by [@LukeTonin](https://github.com/LukeTonin) in [#8223](pydantic/pydantic#8223) - Fix overload position of `computed_field` by [@Viicos](https://github.com/Viicos) in [#8227](pydantic/pydantic#8227) - Fix custom type type casting used in multiple attributes by [@ianhfc](https://github.com/ianhfc) in [#8066](pydantic/pydantic#8066) - Fix issue not allowing `validate_call` decorator to be dynamically assigned to a class method by [@jusexton](https://github.com/jusexton) in [#8249](pydantic/pydantic#8249) - Fix issue `unittest.mock` deprecation warnings by [@ibleedicare](https://github.com/ibleedicare) in [#8262](pydantic/pydantic#8262) - Added tests for the case `JsonValue` contains subclassed primitive values by [@jusexton](https://github.com/jusexton) in [#8286](pydantic/pydantic#8286) - Fix `mypy` error on free before validator (classmethod) by [@sydney-runkle](https://github.com/sydney-runkle) in [#8285](pydantic/pydantic#8285) - Fix `to_snake` conversion by [@jevins09](https://github.com/jevins09) in [#8316](pydantic/pydantic#8316) - Fix type annotation of `ModelMetaclass.__prepare__` by [@slanzmich](https://github.com/slanzmich) in [#8305](pydantic/pydantic#8305) - Disallow `config` specification when initializing a `TypeAdapter` when the annotated type has config already by [@sydney-runkle](https://github.com/sydney-runkle) in [#8365](pydantic/pydantic#8365) - Fix a naming issue with JSON schema for generics parametrized by recursive type aliases by [@dmontagu](https://github.com/dmontagu) in [#8389](pydantic/pydantic#8389) - Fix type annotation in pydantic people script by [@shenxiangzhuang](https://github.com/shenxiangzhuang) in [#8402](pydantic/pydantic#8402) - Add support for field `alias` in `dataclass` signature by [@NeevCohen](https://github.com/NeevCohen) in [#8387](pydantic/pydantic#8387) - Fix bug with schema generation with `Field(...)` in a forward ref by [@dmontagu](https://github.com/dmontagu) in [#8494](pydantic/pydantic#8494) - Fix ordering of keys in `__dict__` with `model_construct` call by [@sydney-runkle](https://github.com/sydney-runkle) in [#8500](pydantic/pydantic#8500) - Fix module `path_type` creation when globals does not contain `__name__` by [@hramezani](https://github.com/hramezani) in [#8470](pydantic/pydantic#8470) - Fix for namespace issue with dataclasses with `from __future__ import annotations` by [@sydney-runkle](https://github.com/sydney-runkle) in [#8513](pydantic/pydantic#8513) - Fix: make function validator types positional-only by [@pmmmwh](https://github.com/pmmmwh) in [#8479](pydantic/pydantic#8479) - Fix usage of `@deprecated` by [@Viicos](https://github.com/Viicos) in [#8294](pydantic/pydantic#8294) - Add more support for private attributes in `model_construct` call by [@sydney-runkle](https://github.com/sydney-runkle) in [#8525](pydantic/pydantic#8525) - Use a stack for the types namespace by [@dmontagu](https://github.com/dmontagu) in [#8378](pydantic/pydantic#8378) - Fix schema-building bug with `TypeAliasType` for types with refs by [@dmontagu](https://github.com/dmontagu) in [#8526](pydantic/pydantic#8526) - Support `pydantic.Field(repr=False)` in dataclasses by [@tigeryy2](https://github.com/tigeryy2) in [#8511](pydantic/pydantic#8511) - Override `dataclass_transform` behavior for `RootModel` by [@Viicos](https://github.com/Viicos) in [#8163](pydantic/pydantic#8163) - Refactor signature generation for simplicity by [@sydney-runkle](https://github.com/sydney-runkle) in [#8572](pydantic/pydantic#8572) - Fix ordering bug of PlainValidator annotation by [@Anvil](https://github.com/Anvil) in [#8567](pydantic/pydantic#8567) - Fix `exclude_none` for json serialization of `computed_field`s by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1098](pydantic/pydantic-core#1098) - Support yyyy-MM-DD string for datetimes by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1124](pydantic/pydantic-core#1124) - Tweak ordering of definitions in generated schemas by [@StrawHatDrag0n](https://github.com/StrawHatDrag0n) in [#8583](pydantic/pydantic#8583) ##### New Contributors ##### `pydantic` - [@ekeew](https://github.com/ekeew) made their first contribution in [#6874](pydantic/pydantic#6874) - [@lambertsbennett](https://github.com/lambertsbennett) made their first contribution in [#8054](pydantic/pydantic#8054) - [@vincent-hachin-wmx](https://github.com/vincent-hachin-wmx) made their first contribution in [#8138](pydantic/pydantic#8138) - [@QuentinSoubeyranAqemia](https://github.com/QuentinSoubeyranAqemia) made their first contribution in [#7825](pydantic/pydantic#7825) - [@ariebovenberg](https://github.com/ariebovenberg) made their first contribution in [#8072](pydantic/pydantic#8072) - [@LukeTonin](https://github.com/LukeTonin) made their first contribution in [#8223](pydantic/pydantic#8223) - [@denisart](https://github.com/denisart) made their first contribution in [#8231](pydantic/pydantic#8231) - [@ianhfc](https://github.com/ianhfc) made their first contribution in [#8066](pydantic/pydantic#8066) - [@eonu](https://github.com/eonu) made their first contribution in [#8255](pydantic/pydantic#8255) - [@amandahla](https://github.com/amandahla) made their first contribution in [#8263](pydantic/pydantic#8263) - [@ibleedicare](https://github.com/ibleedicare) made their first contribution in [#8262](pydantic/pydantic#8262) - [@jevins09](https://github.com/jevins09) made their first contribution in [#8316](pydantic/pydantic#8316) - [@cuu508](https://github.com/cuu508) made their first contribution in [#8322](pydantic/pydantic#8322) - [@slanzmich](https://github.com/slanzmich) made their first contribution in [#8305](pydantic/pydantic#8305) - [@jensenbox](https://github.com/jensenbox) made their first contribution in [#8331](pydantic/pydantic#8331) - [@szepeviktor](https://github.com/szepeviktor) made their first contribution in [#8356](pydantic/pydantic#8356) - [@Elkiwa](https://github.com/Elkiwa) made their first contribution in [#8341](pydantic/pydantic#8341) - [@parhamfh](https://github.com/parhamfh) made their first contribution in [#8395](pydantic/pydantic#8395) - [@shenxiangzhuang](https://github.com/shenxiangzhuang) made their first contribution in [#8402](pydantic/pydantic#8402) - [@NeevCohen](https://github.com/NeevCohen) made their first contribution in [#8387](pydantic/pydantic#8387) - [@zby](https://github.com/zby) made their first contribution in [#8497](pydantic/pydantic#8497) - [@patelnets](https://github.com/patelnets) made their first contribution in [#8491](pydantic/pydantic#8491) - [@edwardwli](https://github.com/edwardwli) made their first contribution in [#8503](pydantic/pydantic#8503) - [@luca-matei](https://github.com/luca-matei) made their first contribution in [#8507](pydantic/pydantic#8507) - [@Jocelyn-Gas](https://github.com/Jocelyn-Gas) made their first contribution in [#8437](pydantic/pydantic#8437) - [@bL34cHig0](https://github.com/bL34cHig0) made their first contribution in [#8501](pydantic/pydantic#8501) - [@tigeryy2](https://github.com/tigeryy2) made their first contribution in [#8511](pydantic/pydantic#8511) - [@geospackle](https://github.com/geospackle) made their first contribution in [#8537](pydantic/pydantic#8537) - [@Anvil](https://github.com/Anvil) made their first contribution in [#8567](pydantic/pydantic#8567) - [@hungtsetse](https://github.com/hungtsetse) made their first contribution in [#8546](pydantic/pydantic#8546) - [@StrawHatDrag0n](https://github.com/StrawHatDrag0n) made their first contribution in [#8583](pydantic/pydantic#8583) ##### `pydantic-core` - [@mariuswinger](https://github.com/mariuswinger) made their first contribution in [pydantic/pydantic-core#1087](pydantic/pydantic-core#1087) - [@adamchainz](https://github.com/adamchainz) made their first contribution in [pydantic/pydantic-core#1090](pydantic/pydantic-core#1090) - [@akx](https://github.com/akx) made their first contribution in [pydantic/pydantic-core#1123](pydantic/pydantic-core#1123) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.apud.pl/jacek/adventofcode/pulls/57 Co-authored-by: Renovate <renovate@apud.pl> Co-committed-by: Renovate <renovate@apud.pl>

alexmojaki added 16 commits November 10, 2023 16:46

Add eval_type_backport to handle union operator in older Pythons

75986a9

Use modified get_type_hints in test_config

ca60caa

unskip a couple more tests in older pythons

a163ea3

Use pipe operator in a bunch of tests

8033556

various misc tidying up: use default localns=None, handle None values…

0495f0f

… properly, clean up type ignore comments

explain asserts

780b46a

type hints

4c536b6

inline node_to_ref

d7f874b

is_unsupported_types_for_union_error

09d2e93

tidying

d7b1462

remove more type: ignore comments

c65ae3d

docstrings and tidying

3aa88da

fix and tighten test_is_union

ca09a03

Merge branch 'main' of github.com:pydantic/pydantic into eval_type_ba…

4890fc8

…ckport

Use eval_type_backport package

c2d9d22

Add test dependency

40a41fb

alexmojaki marked this pull request as ready for review November 22, 2023 19:21

pydantic-hooky bot added the ready for review label Nov 22, 2023

pydantic-hooky bot assigned davidhewitt Nov 22, 2023

alexmojaki mentioned this pull request Nov 22, 2023

Add eval_type_backport to handle union operator in older Pythons #7958

Closed

5 tasks

dmontagu reviewed Nov 27, 2023

View reviewed changes

pydantic/_internal/_typing_extra.py Outdated Show resolved Hide resolved

dmontagu reviewed Nov 27, 2023

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

dmontagu reviewed Nov 27, 2023

View reviewed changes

tests/test_types.py Outdated Show resolved Hide resolved

dmontagu requested changes Nov 27, 2023

View reviewed changes

pydantic-hooky bot added awaiting author revision and removed ready for review labels Nov 27, 2023

pydantic-hooky bot assigned alexmojaki and unassigned davidhewitt Nov 27, 2023

alexmojaki added 11 commits December 16, 2023 21:28

upgrade eval_type_backport to handle fussy typing._type_check

71ca7cf

update is_backport_fixable_error and move down, update eval_type_back…

f060876

…port function docstring

raise helpful error if eval_type_backport isn't installed. ensure tes…

2b42d65

…t_recursive_models_union passes without backport

Restore skip, add another test for combination of backport and Pydant…

dcbdd28

…icRecursiveRef.__or__

Test that eval_type_backport is being called in the right places

959c755

test calling backport from get_type_hints

71c912e

upgrade eval_type_backport to handle working union operator

9847058

unskip tests that can now pass in 3.8

6beab5b

revert scattered test changes

b79692b

unskip more tests

6bc0ee4

upgrade eval_type_backport to copy ForwardRef attributes, allowing un…

f423659

…skipping one more test

alexmojaki force-pushed the eval_type_backport branch from cd8b955 to f423659 Compare December 17, 2023 18:17

alexmojaki marked this pull request as ready for review December 17, 2023 18:52

alexmojaki requested review from samuelcolvin and dmontagu December 17, 2023 18:52

samuelcolvin mentioned this pull request Dec 23, 2023

Adopt PEP 585 and PEP 604 #8434

Closed

13 tasks

bswck mentioned this pull request Dec 23, 2023

Backport PEP 585 + PEP 604 to Python 3.8 and 3.9 pydantic/FastUI#99

Closed

bswck mentioned this pull request Jan 15, 2024

How to use it alexmojaki/eval_type_backport#11

Open

Merge branch 'main' of github.com:pydantic/pydantic into eval_type_ba…

d3d5584

…ckport

alexmojaki changed the title ~~Add eval_type_backport to handle union operator in older Pythons~~ Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons Jan 15, 2024

revert moving part of pyproject.toml

aa21092

samuelcolvin mentioned this pull request Jan 15, 2024

🚀 2.6.0b1 checklist ✅ #8549

Closed

10 tasks

samuelcolvin added the relnotes-feature label Jan 16, 2024

samuelcolvin approved these changes Jan 16, 2024

View reviewed changes

alexmojaki added 2 commits January 16, 2024 19:22

Refine and test error raised when eval_type_backport isn't installed

8da4294

use a type annotation that's unsupported in 3.9, not just 3.8

60aa70f

dmontagu approved these changes Jan 16, 2024

View reviewed changes

dmontagu merged commit 8060fa1 into pydantic:main Jan 16, 2024
54 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons #8209

Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons #8209

alexmojaki commented Nov 22, 2023 •

edited

codspeed-hq bot commented Nov 22, 2023 •

edited

alexmojaki commented Nov 22, 2023

dmontagu left a comment

bswck commented Dec 23, 2023

samuelcolvin Jan 16, 2024

Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons #8209

Add eval_type_backport to handle union operator and builtin generic subscripting in older Pythons #8209

Conversation

alexmojaki commented Nov 22, 2023 • edited

codspeed-hq bot commented Nov 22, 2023 • edited

CodSpeed Performance Report

Merging #8209 will not alter performance

Summary

alexmojaki commented Nov 22, 2023

dmontagu left a comment

Choose a reason for hiding this comment

bswck commented Dec 23, 2023

samuelcolvin Jan 16, 2024

Choose a reason for hiding this comment

alexmojaki commented Nov 22, 2023 •

edited

codspeed-hq bot commented Nov 22, 2023 •

edited