Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeAdapter.json_schema drops description from Annotated Fields when self.type is a function #9404

Open
1 task done
cisaacstern opened this issue May 7, 2024 · 6 comments · May be fixed by #9481
Open
1 task done
Labels
bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation

Comments

@cisaacstern
Copy link

cisaacstern commented May 7, 2024

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

I was very happy to learn about TypeAdapter via reading #7133.

In experimenting with generation of JSON Schema from plain python functions, I found that the description attribute of an Annotated Field is dropped from the default generated JSON Schema if the type passed to TypeAdapter is a plain function.

If the type passed to TypeAdapter is a type alias (the Annotated Field itself) or a BaseModel using that type, the description attribute is preserved.

The MRE provided below demonstrates this issue and a workaround using a subclass of GenerateJsonSchema.

Is it reasonable to expect that the default schema generator would produce the same JSON for the same Annotated Field used in these three different settings? If so, would the team be open to a PR upstreaming the behavior of this workaround into the default?

Thank you all very much, Pydantic is truly remarkable.

Example Code

import json
from typing import Annotated

from pydantic import BaseModel, Field, TypeAdapter

# An annotated type with a `description`
FooType = Annotated[int, Field(default=1, description="bar")]

class Model(BaseModel):
    foo: FooType

def func(foo: FooType): ...

# Description is preserved 😃 
from_type_alias = TypeAdapter(FooType).json_schema()
print(json.dumps(from_type_alias, indent=4))
# {
#     "default": 1,
#     "description": "bar",
#     "type": "integer"
# }

# Description is preserved 😃
from_model = TypeAdapter(Model).json_schema()
print(json.dumps(from_model, indent=4))
# {
#     "properties": {
#         "foo": {
#             "default": 1,
#             "description": "bar",
#             "title": "Foo",
#             "type": "integer"
#         }
#     },
#     "title": "Model",
#     "type": "object"
# }

# Description is lost 🤔
from_func = TypeAdapter(func).json_schema()
print(json.dumps(from_func, indent=4))
# {
#     "additionalProperties": false,
#     "properties": {
#         "foo": {
#             "default": 1,
#             "title": "Foo",
#             "type": "integer"
#         }
#     },
#     "type": "object"
# }


# Workaround ----------------------------------------------

from inspect import signature
from typing import get_args

from pydantic.fields import FieldInfo
from pydantic.json_schema import GenerateJsonSchema


class SurfacesDescriptionSchema(GenerateJsonSchema):
    def generate(self, schema, mode='validation'):
        json_schema = super().generate(schema, mode=mode)
        if "function" in schema:
            for p in json_schema["properties"]:
                annotation_args = get_args(signature(schema["function"]).parameters[p].annotation)
                if any([isinstance(arg, FieldInfo) for arg in annotation_args]):
                    Field: FieldInfo = [arg for arg in annotation_args if isinstance(arg, FieldInfo)][0]
                    if Field.description:
                        json_schema["properties"][p]["description"] = Field.description
        return json_schema
    

# Description is preserved 😃
from_func_custom_generator = TypeAdapter(func).json_schema(schema_generator=SurfacesDescriptionSchema)
print(json.dumps(from_func_custom_generator, indent=4))
# {
#     "additionalProperties": false,
#     "properties": {
#         "foo": {
#             "default": 1,
#             "title": "Foo",
#             "type": "integer",
#             "description": "bar"
#         }
#     },
#     "type": "object"
# }

Python, Pydantic & OS Version

pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=false
                 install path: /Users/charlesstern/miniconda3/envs/ecoscope/lib/python3.10/site-packages/pydantic
               python version: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:39:40) [Clang 15.0.7 ]
                     platform: macOS-14.4.1-x86_64-i386-64bit
             related packages: typing_extensions-4.11.0
                       commit: unknown
@cisaacstern cisaacstern added bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels May 7, 2024
@cisaacstern cisaacstern changed the title TypeAdapter.json_schema drops description from Annotated Fields TypeAdapter.json_schema drops description from Annotated Fields when self.type is a function May 7, 2024
@sydney-runkle
Copy link
Member

sydney-runkle commented May 16, 2024

Hi @cisaacstern,

Thanks for your kind words. We really appreciate the in-depth description and MRE :).

Thanks for reporting this - definitely looks like a bug, and is something we should fix. Are you interested in opening a PR to address this issue?

@cisaacstern
Copy link
Author

@sydney-runkle, thanks for your reply!

Yes, I would be happy to make a PR.

I will ping back here when I have one for review and/or if I need further guidance. 🙏

@sydney-runkle
Copy link
Member

@cisaacstern,

Fantastic! Sorry for the delayed response last week, was out for vacation.

@datalogics-kam
Copy link

Yes, I would be happy to make a PR.

I thank you as well. I have put your workaround to good use.

@sydney-runkle
Copy link
Member

Working on this now. It's quite messy, lots of core schema issues for parameters. Should have a fix done by the end of the week.

@cisaacstern
Copy link
Author

Thanks @sydney-runkle! It would be quite some time before I have enough bandwidth to work on this, so very happy to know you are working on it! TYSM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants