strip_whitespace after coercion, but before constraint checks on custom data types

[prryplatypus]

Initial Checks

• I have searched Google & GitHub for similar requests and couldn't find anything
• I have read and followed the docs and still think this feature is missing

Description

Note: See #6469 for the full discussion.

In Pydantic V1, it was possible to do something as simple as this, which resulted in the following operations happening in this order:

1. Type checking/coercion
2. Whitespace stripping
3. Length constraint checks

from pydantic import StrictStr

class Username(StrictStr):
    min_length = 3
    max_length = 20
    strip_whitespace = True

However, in Pydantic V2 this case is particularly tricky because the min_length constraint gets applied after coercion but before the Python string is created, and hence before the AfterValidator. So there is really no way to insert custom logic between these two steps.

The closest one can get is by using something like this, but this is a lot of mental gymnastics for something that was trivial in V1.

from typing_extensions import Annotated
from pydantic import BaseModel, Field, BeforeValidator

Username = Annotated[str, BeforeValidator(lambda x: str.strip(str(x))), Field(min_length=3, max_length=20)]

@adriangb suggested the easiest thing to do would probably be to just add back str_strip_whitespace to Field or create some sort of ConStr() marker for annotated (Annotated[str, ConStr(strip_whitespace=True), Field(max_length=10, ...)] or something like that), and that is why I am creating this feature request.

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .model_dump() and .model_dump_json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.

Selected Assignee: @lig

[zakstucke]

Not much simpler at the minute, but this as_bv() util allows you to use existing validatory types such as constr, rather than rolling your own unnecessarily, by wrapping the type in a BeforeValidator.

tp.Annotated[str, Field(min_length=3, max_length=10), as_bv(constr(strip_whitespace=True))]

I'm hoping the equivalent of as_bv() is done implicitly pydantic-side at some point as per #6506 (comment)

import typing as tp
from pydantic import BeforeValidator, TypeAdapter, constr, Field

def as_bv(t: type) -> BeforeValidator:
    """Converts a type into a BeforeValidator"""
    ta = TypeAdapter(t)
    return BeforeValidator(lambda v: ta.validate_python(v))

MyT = tp.Annotated[str, Field(min_length=3, max_length=10), as_bv(constr(strip_whitespace=True))]

res = TypeAdapter(MyT).validate_python("   123456789   ")

print(f"res: '{res}'")

res: '123456789'

[adriangb]

This comment relates more to #6506 but I'm keeping it here since this is where I originally posted it.

There are fundamental problems with this approach that will not allow us to include it in Pydantic directly.

Practically speaking, we've worked very hard to make the entirety of validation happen within a single Rust call (with callbacks to Python when necessary, yes, but avoiding these as much as possible for performance reasons). Not only would inserting a call to a TypeAdapter be problematic performance-wise, but it's also going to wreak havoc with exceptions (the exception from that validation call will end up just being a repr(ValueError()) instead of being collected into the top level ValueError) and probably other things I'm not even thinking of that we won't figure out until it's too late. It's just fundamentally an approach that Pydantic is not designed for.

On a perhaps more philosophical note the metadata in Annotated is not intended to be types, and while there are no runtime errors by putting a type in there it does seem just wrong to do.

That is not to say you can't use this utility function if it works for you, I just don't think it's as simple, universal, and cheap of a win as it seems on the surface.

[prryplatypus]

@lig why is this "not planned"? It's a step back from Pydantic V1.

@adriangb suggested the easiest thing to do would probably be to just add back str_strip_whitespace to Field or create some sort of ConStr() marker for annotated (Annotated[str, ConStr(strip_whitespace=True), Field(max_length=10, ...)] or something like that), and his last comment in this thread related more to the other issue than to this one, as he said.

[lig]

@prryplatypus At the moment solving this is too complicated as it would require some serious work in pydantic-core and it looks tricky to implement without affecting overall Pydantic performance.

This is not a step back but a cost of significant performance improvement of Pydantic v2 comparing to Pydantic v1.

That said, PRs are always welcome. In this particular case it's important that the change wouldn't affect the performance.

[adriangb]

#6951 maybe solves this