Field metadata from Annotated is lost after a call to update_forward_refs

[xx55tt]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.8.2
            pydantic compiled: False
                 install path: /usr/lib/python3.9/site-packages/pydantic
               python version: 3.9.7 (default, Aug 31 2021, 13:28:12)  [GCC 11.1.0]
                     platform: Linux-5.14.8-arch1-1-x86_64-with-glibc2.33
     optional deps. installed: ['email-validator', 'typing-extensions']

from __future__ import annotations
from typing import Annotated

from pydantic import BaseModel, Field


class Works(BaseModel):
    foo: list[Foo] = Field(..., alias="bar")


class Fails(BaseModel):
    foo: Annotated[list[Foo], Field(alias="bar")]


class Foo(BaseModel):
    a: int


Works.update_forward_refs()
Fails.update_forward_refs()

json = '{"bar":[{"a":0}]}'

Works.parse_raw(json)
Fails.parse_raw(json)

With the postponed evaluation of annotations enabled, the Field metadata seems to be lost after the update_forward_refs call. Apart from that it seemed to be working fine, it parsed the JSON if the field name was foo. I also tried the allow_mutation argument, it was not working as well.

Without the from __future__ import annotations import, all of the following worked:

foo: "list[Foo]" = Field(..., alias="bar")
foo: List["Foo"] = Field(..., alias="bar")
foo: list[ForwardRef("Foo")] = Field(..., alias="bar")
foo: Annotated["list[Foo]", Field(alias="bar")]
foo: Annotated[List["Foo"], Field(alias="bar")]
foo: Annotated[list[ForwardRef("Foo")], Field(alias="bar")]

[klaa97]

I reproduced your bug and debugged a little, and I 100% agree: the root of the problem is that the update_forward_refs method doesn't correctly parse the information from the FieldInfo of Annoted types.

I tried to propose a fix here: #3283

[dmontagu]

It looks like this is now fixed in both v1 and v2

[mshonichev]

Whoa! Why was this issue closed without merging the fix?
I find it is still reproducing at

pydantic==2.4.2
pydantic_core==2.10.1

reproducer is exactly the same:

from __future__ import annotations

class Foo(BaseModel):
    bar: Annotated[Bar | None, Field(alias="_bar")] = None

class Bar(BaseModel):
    foo: int | None = None


def test_model():
    data= '{"_bar": {"foo": 42}}'
    model = Foo.model_validate_json(data)
    assert model.bar.foo == 42

throws AttributeError: 'NoneType' object has no attribute 'foo'

but if we change the order of declarations, validation successfully loads the alias and test succeeds:

class Bar(BaseModel):
    foo: int | None = None

class Foo(BaseModel):
    bar: Annotated[Bar | None, Field(alias="_bar")] = None


def test_model():
    data= '{"_bar": {"foo": 42}}'
    model = Foo.model_validate_json(data)
    assert model.bar.foo == 42

[mshonichev]

@dmontagu could you please reopen this issue, or should I probably file a new one?

[dmontagu]

I'm not sure why I closed it, clearly it's still an issue. Apologies for that.

[dmontagu]

@mshonichev I think I've got a fix for it ready in #7698 (at least for pydantic v2), thanks for bringing this to my attention again, and apologies for closing it before when it wasn't actually fixed.

[dmontagu]

Also @mshonichev, I added a different test in #7698, but I can confirm that your snippet above does not error with the change in that PR.