Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modernize data validation #1674

Open
BryceGattis opened this issue Mar 3, 2024 · 0 comments
Open

Modernize data validation #1674

BryceGattis opened this issue Mar 3, 2024 · 0 comments

Comments

@BryceGattis
Copy link
Contributor

BryceGattis commented Mar 3, 2024

We should consider switching to more modern data validation libraries such as "pydantic".

Motivation
The schema library we are using is pretty outdated these days. We are using a very old version of schema that was released in 2014. We have a couple of complaints with the current system:

  • We currently are relying on "evil" metaclasses that implicitly inject attributes to classes at runtime. This both doesn't work very well with IDEs and also is just hard to make sense of from a developers point of view.
  • Additionally, all of the attributes we are defining in these schemas have no docstring, and therefore their expected usage and data is completely unknown to new users.
  • The current schema library can have some pretty awful output for users, and can be difficult to parse and determine what went wrong. Here's an example output:
  File "D:\Code\rez\venv\lib\site-packages\rez\vendor\schema\schema.py", line 235, in validate
    raise SchemaError([None] + x.autos, [e] + x.errors)
rez.vendor.schema.schema.SchemaError: Or(<class 'rez.utils.sourcecode.SourceCode'>, Schema({Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.util
s.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})})) did not validate 'None'  
'None' should be instance of {Optional(<class 'str'>): Or(Or(<class 'str'>, [<class 'str'>]), {Optional(<class 'str'>): <class 'object'>, 'command': Or(<class 'str'>, [<class 'str'>]), Optional('requires'): [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))], Optional
('run_on'): Or(<class 'str'>, [<class 'str'>]), Optional('on_variants'): Or(<class 'bool'>, {'type': 'requires', 'value': [Or(<class 'str'>, And(<class 'rez.utils.formatting.PackageRequest'>, Use(<class 'str'>)))]})})}

Related Conversations

#1662 (comment)

Note that if we use pydantic, we must use pydantic < 2.6, as 2.6 introduced a breaking change with Python 3.7 which is our lowest support Python version as of now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant