Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle constraints being applied to schemas that don't accept it #6951

Merged
merged 14 commits into from Aug 10, 2023

Conversation

adriangb
Copy link
Member

@adriangb adriangb commented Jul 30, 2023

This handles the particular case of pattern not working when using a custom GenerateSchema (reported in #6045 (comment)) but also tries to be more general to handle other similar issues.

Selected Reviewer: @samuelcolvin

@cloudflare-pages
Copy link

cloudflare-pages bot commented Jul 30, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2a88570
Status: ✅  Deploy successful!
Preview URL: https://72f8bed9.pydantic-docs2.pages.dev
Branch Preview URL: https://annotate-schema-update.pydantic-docs2.pages.dev

View logs

@adriangb
Copy link
Member Author

adriangb commented Jul 30, 2023

Open questions:

  • Should we try to error fast if we know there's going to be an issue, e.g. Annotated[int, Field(pattern=...)]. The only way I see to do this is essentially special case by checking constraint = 'pattern' and schema['type'] in ('int', 'float', etc.). I think it’s a lot of work and not worth it.
  • Can we get rid of the "merging" of FieldInfo? I think for validation yes, for JSON schema we may still need to merge them, but not even sure about that. Seems like the answer is yes
  • Do we still want to make a more performant version of this in pydantic-core?
  • Can we get rid of the whole __prepare_pydantic_annotations__ thing and just say that customizing checks is not supported?

@adriangb
Copy link
Member Author

adriangb commented Jul 30, 2023

please review

Most tests are passing, a couple are failing, in part because of the questions above.

@adriangb adriangb marked this pull request as ready for review July 30, 2023 17:02
@adriangb
Copy link
Member Author

This is now working and allowing constraint re-ordering. The cost is running things in Python when they can't be "inlined" into Rust. There's also a slight inconsistency I belive since when we do "inline" it the order is not preserved (it's whatever the order of the if statements is in Rust). I guess that's fine.

tests/test_types.py Outdated Show resolved Hide resolved
@adriangb
Copy link
Member Author

adriangb commented Aug 4, 2023

These two give different results:

from typing_extensions import Annotated

from pydantic import BaseModel, Field, AfterValidator

from pydantic.type_adapter import TypeAdapter


Tp = Annotated[int, AfterValidator(lambda x: x * 2), Field(gt=3)]

class A(BaseModel):
    x: Tp

print(A.__pydantic_core_schema__['definitions'][0]['schema']['fields']['x']['schema'])

ta = TypeAdapter(Tp)
print(ta.core_schema)

It seems like this is because BaseModel merges field infos when it's constructed. We'd have to get rid of all field-info merging to get the right behavior w.r.t. ordering of constraints. I'm inclined to leave fixing that for a future PR. WDYT @dmontagu?

Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise LGTM.

pydantic/_internal/_validators.py Outdated Show resolved Hide resolved
for constraint in URL_CONSTRAINTS:
CONSTRAINTS_TO_ALLOWED_SCHEMAS[constraint].update(('url', 'multi-host-url'))
for constraint in BOOL_CONSTRAINTS:
CONSTRAINTS_TO_ALLOWED_SCHEMAS[constraint].update(('bool',))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this get called on from pydantic import X? If so we might need to worry about import time, if it's later/when require it shouldn't be a problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it is imported immediately. It's not all that complex though, just assigning times to a dict. I could hide it behind an lru_cache function or something, but that would probably make schema generation slower (one more dict lookup), and that seems to be a bigger problem right now than import time

@samuelcolvin
Copy link
Member

please update.

@adriangb
Copy link
Member Author

adriangb commented Aug 8, 2023

please review

adriangb and others added 2 commits August 9, 2023 07:33
Co-authored-by: Samuel Colvin <s@muelcolvin.com>
@adriangb
Copy link
Member Author

adriangb commented Aug 9, 2023

please review

maybe_as_any: Optional[SerializeAsAny[User]]
maybe_as_any: Optional[SerializeAsAny[User]] = None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly necessary for this PR but the insert_assert() below doesn't work without it

Comment on lines +275 to +278
def forbid_inf_nan_check(x: Any) -> Any:
if not math.isfinite(x):
raise PydanticKnownError('finite_number')
return x
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test for the error case here

pydantic/_internal/_known_annotated_metadata.py Outdated Show resolved Hide resolved
@adriangb adriangb merged commit 998a6ba into main Aug 10, 2023
49 checks passed
@adriangb adriangb deleted the annotate-schema-update branch August 10, 2023 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants