Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add caches for collecting definitions and invalid schemas from a CoreSchema #7527

Merged
merged 2 commits into from Sep 20, 2023
Merged

Add caches for collecting definitions and invalid schemas from a CoreSchema #7527

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
32b2977
Add caches for collecting definitions and invalid schemas from a Core…
adriangb Sep 20, 2023
f0487f6
Revert change to collect_invalid_schemas
adriangb Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 18 additions & 2 deletions pydantic/_internal/_core_utils.py
View file
Open in desktop
Expand Up @@ -116,6 +116,12 @@ def collect_definitions(schema: core_schema.CoreSchema) -> dict[str, core_schema
defs: dict[str, CoreSchema] = {}

def _record_valid_refs(s: core_schema.CoreSchema, recurse: Recurse) -> core_schema.CoreSchema:
if 'metadata' in s:
definitions_cache: _DefinitionsState | None = s['metadata'].get(_DEFINITIONS_CACHE_METADATA_KEY, None)
if definitions_cache is not None:
defs.update(definitions_cache['definitions'])
return s

ref = get_ref(s)
if ref:
defs[ref] = s
Expand Down Expand Up @@ -157,11 +163,21 @@ def _record_refs(s: core_schema.CoreSchema, recurse: Recurse) -> core_schema.Cor

def collect_invalid_schemas(schema: core_schema.CoreSchema) -> list[core_schema.CoreSchema]:
invalid_schemas: list[core_schema.CoreSchema] = []
child_invalid = False # track recursion

def _is_schema_valid(s: core_schema.CoreSchema, recurse: Recurse) -> core_schema.CoreSchema:
if s.get('metadata', {}).get('invalid'):
nonlocal child_invalid
metadata = s.setdefault('metadata', {})
invalid = metadata.get('invalid', None)
if invalid is True:
invalid_schemas.append(s)
return recurse(s, _is_schema_valid)
elif invalid is False:
return s
s = recurse(s, _is_schema_valid)
if invalid is True:
child_invalid = True
metadata['invalid'] = child_invalid
return s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate on the motivation of this change? Maybe needs a comment, it's quite winding logic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The invalid thing already existed. What this is doing is marking a schema as "there are no invalid schemas in this subtree" (which is the most common case) so that we can skip recursing into it after the first time we recurse through it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now that I think of it we should be able to do something similar to https://github.com/pydantic/pydantic/pull/7529/files#diff-2ebfa892ec1d09440eefd4b22f488ec38c1cde365394e5a4cea6edb484827ba5 and do this from within GenerateSchema during our first recursion through the CoreSchema (we build it recursively).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll wait to make that change until #7529 is merged so I can re-use post_process_schema and such


walk_core_schema(schema, _is_schema_valid)
return invalid_schemas
Expand Down