Issue with schema generation when using forward refs

[jsmith]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

There is an issue when merging fields in the _common_field_schema function in pydantic/_internal/_generate_schema.py. The issue is that field_info._attributes_set includes both FieldInfo attributes in addition to metadata so when the function checks if if k not in field_info._attributes_set (where k is a key from _attributes_set) and then uses setattr(field_info, k, v), it will fail for anything that isn't actually an attribute of the object (i.e. anything that is in the metadata). Sorry if this is a poor explanation, I have a test which fails below.

This bug was introduced in #7698. cc @dmontagu I see you authored the original PR. I wish I could put up a fix myself but I don't fully understand the purpose of the code.

Here is my branch with a test that reproduces the bug:
main...jsmith:pydantic:jsmith/fix-generate-schema-annotation-merge

Example Code

# Test in tests/test_annotated


def test_min_length_field_info_not_lost():
    from pydantic import BaseModel

    class AnnotatedFieldModel(BaseModel):
        foo: 'Annotated[String, Field(min_length=3)]' = Field(description='hello')

    String = str

    AnnotatedFieldModel.model_rebuild()

    assert AnnotatedFieldModel(foo='000').foo == '000'

    with pytest.raises(ValidationError) as exc_info:
        AnnotatedFieldModel(foo='00')

    assert exc_info.value.errors(include_url=False) == [
        {
            'loc': ('foo',),
            'input': '00',
            'ctx': {'min_length': 3},
            'msg': 'String should have at least 3 characters',
            'type': 'string_too_short',
        }
    ]

Python, Pydantic & OS Version

pydantic version: 2.6.0a1
        pydantic-core version: 2.14.5
          pydantic-core build: profile=release pgo=false
               python version: 3.9.6 (default, Nov 10 2023, 13:38:27)  [Clang 15.0.0 (clang-1500.1.0.2.5)]
                     platform: macOS-14.1.2-arm64-arm-64bit
             related packages: pydantic-extra-types-2.1.0 pyright-1.1.344 typing_extensions-4.8.0 pydantic-settings-2.1.0 mypy-1.1.1 email-validator-2.1.0.post1
                       commit: 5178bcc1

[dmontagu]

@jsmith Thanks for reporting. I believe I have fixed the issue in #8494. (I stole your test, thanks very much for that.)

It would be great if you could confirm whether that fixes your issues; it at least made the test you provided pass (with some additional validation that it behaves correctly when the metadata should not be overwritten).