Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding validators package #258

Open
sobolevn opened this issue Feb 7, 2020 · 16 comments · May be fixed by #584
Open

Consider adding validators package #258

sobolevn opened this issue Feb 7, 2020 · 16 comments · May be fixed by #584
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@sobolevn
Copy link
Member

sobolevn commented Feb 7, 2020

Currently a lot of python tools just throw exceptions at us or even uses: .is_valid() and .errors properties. It is not type-safe at all. Sometimes it is hard to explain to people how to really use type-driven design and make invalid states unreachable.

My idea is that we need to write thin wrappers around popular typed validation packages: like pydantic, cerebrus, schema, etc to return really typed things.

We also need to think about API to be expressive and recognisable enough.

Related: https://gist.github.com/maksbotan/14b4bebda2acab98cdd158f85a970855
CC @maksbotan

@sobolevn sobolevn added enhancement New feature or request help wanted Extra attention is needed labels Feb 7, 2020
@sobolevn
Copy link
Member Author

sobolevn commented Feb 7, 2020

Shame on me! Not validators but parsers!

See https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/

@stereobutter
Copy link

there is also https://validators.readthedocs.io/en/latest/ which imho fortunately returns errors instead of raising them

@sobolevn
Copy link
Member Author

sobolevn commented Jun 1, 2020

Or we can add validate function to the pipeline.py.
Probably in the next release.

@sobolevn
Copy link
Member Author

sobolevn commented Jun 5, 2020

@sobolevn
Copy link
Member Author

sobolevn commented Jun 9, 2020

@sobolevn
Copy link
Member Author

sobolevn commented Jun 12, 2020

So, it looks like Validated should be defined like so:

class Validated(Generic[_ValueType, _ErrorType]):
    _inner_value: Iterable[
        Callable[[_ValueType], Result[_ValueType, _ErrorType]],
    ]

It should work with all types: Maybe, FutureResult, IOResult, and Result, somehow. Union?

@sobolevn
Copy link
Member Author

@maksbotan
Copy link

@sobolevn
Copy link
Member Author

I am using These in fp-ts, it is quite useful when working with components in vue.
But, I don't know how one can use it in Python? Can you please share some usecases?

@astynax
Copy link
Collaborator

astynax commented Jun 13, 2020

Here is another nice lib for validation: https://github.com/typeable/validationt
This one collects warnings but stops on errors - a pretty convenient behavior IMHO. Also you can get more than one warning for each field - it's nice too.

@sobolevn
Copy link
Member Author

@sobolevn
Copy link
Member Author

sobolevn commented Aug 1, 2020

Current validation prototype:

from returns.primitives.hkt import KindN, kinded, Kinded
from returns.interfaces.specific.result import ResultLikeN
from returns.primitives.container import BaseContainer
from typing import TypeVar, Sequence, Callable, Iterable, TYPE_CHECKING
from returns._generated.iterable import iterable_kind

# Definitions:
_FirstType = TypeVar('_FirstType')
_SecondType = TypeVar('_SecondType')
_ThirdType = TypeVar('_ThirdType')

_ResultKind = TypeVar('_ResultKind', bound=ResultLikeN)

if not TYPE_CHECKING:
    reveal_type = print



# TODO:
def validate(
    items: Iterable[
        Callable[
            [_FirstType],
            KindN[_ResultKind, _FirstType, _SecondType, _ThirdType],
        ]
    ],
) -> Kinded[Callable[
    [_FirstType],
    KindN[_ResultKind, _FirstType, Sequence[_SecondType], _ThirdType],
]]:
    @kinded
    def factory(instance: _FirstType) -> KindN[
        _ResultKind,
        _FirstType,
        Sequence[_SecondType],
        _ThirdType,
    ]:
        swapped = [item.swap() for item in items]
        return iterable_kind(type(swapped[0]), swapped).swap()
    return factory

What can be changed / improved?

  1. We need to add Better helpers for working with collections #526 support
  2. We also need to change the function's signature to be validate(items, *, strategy: _ValidationStrategy = ...). Where _ValidationStrategy is some kind of a callable protocol. There can be multiple strategies, including: collecting all erorrs for a field vs collecting only a single (first / last) error for each field. So, the code would change like so: returns strategy(swapped).swap()
  3. We also need to provide a higher-level abstraction to create validation pipelines. Because, currently there's no way to define that in a reasonable manner. Why do we need this? Because each validation item must have similar KindN[_ResultKind, ...] types. So, we cannot combine IOResult and Result validations in a single call:
import attr
from returns.result import Result
from returns.io import IOResult
from returns.context import ReaderResult

@attr.dataclass
class Person(object):
    fullname: str
    age: int
    passport: str

def validate_fullname(person: Person) -> Result[Person, str]:
    if not person.fullname:
        return Result.from_failure('No fullname specified')
    return Result.from_value(person)

def validate_age(person: Person) -> Result[Person, str]:
    if person.age < 0:
        return Result.from_failure('Negative age')
    return Result.from_value(person)

def validate_passport(person: Person) -> IOResult[Person, str]:
    """Impures, calls 3rd party API."""
    if not person.passport:  # this is not an IO action, just an example
        return IOResult.from_failure('Missing passort')
    return IOResult.from_value(person)

def validate_with_context(person: Person) -> ReaderResult[Person, str, int]:
    """Requires ``int`` context to actually validate anything."""
    def factory(deps: int) -> Result[Person, str]:
        if person.age < deps:
            return Result.from_failure('Less than minimal {0} age'.format(deps))
        return Result.from_value(person)
    return ReaderResult(factory)

person = Person('', 28, '')

simple = validate([
    validate_fullname,
    validate_age,
])(person)

hard = validate([
    validate_passport,
])(person)

context = validate([
    validate_with_context
])(person)

reveal_type(simple)
reveal_type(hard)
reveal_type(context(35))

Outputs:

experiments/validate.py:94: note: Revealed type is 'returns.result.Result*[validate.Person*, typing.Sequence[builtins.str*]]'
experiments/validate.py:95: note: Revealed type is 'returns.io.IOResult*[validate.Person*, typing.Sequence[builtins.str*]]'
experiments/validate.py:96: note: Revealed type is 'returns.result.Result[validate.Person*, typing.Sequence*[builtins.str*]]'

So, we need to somehow turn simple, hard, and context into a single call.

@sobolevn
Copy link
Member Author

sobolevn commented Aug 1, 2020

We can also add simple types to make validation more meaningful:

from returns.validation import Valid, Invalid, Validated

def register_user(user: Valid[User]) -> None:
     api.register_user_call(user.value)

Where Validated = Union[Valid[_ValueType], Invalid[_ValueType]]

@thepabloaguilar thepabloaguilar linked a pull request Sep 10, 2020 that will close this issue
4 tasks
@sobolevn
Copy link
Member Author

sobolevn commented Oct 3, 2020

@sobolevn sobolevn linked a pull request Apr 28, 2021 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Development

Successfully merging a pull request may close this issue.

4 participants