Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix int_validator not catching overflows #3112

Merged
merged 2 commits into from
Aug 5, 2022

Conversation

ojii
Copy link
Contributor

@ojii ojii commented Aug 18, 2021

Change Summary

int(value) can raise an OverflowError, for example when called with very large floats.

Related issue number

#2345

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI and coverage remains at 100%
  • Documentation reflects the changes where applicable
  • changes/<pull request or issue id>-<github username>.md file added describing change
    (see changes/README.md for details)
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Sorry, something went wrong.

@ojii
Copy link
Contributor Author

ojii commented Aug 18, 2021

please review

@PrettyWood
Copy link
Collaborator

I reckon it would be better to catch the OverflowError in a separate except clause with a more explicit error. We could probably do the exact same for float

@ojii
Copy link
Contributor Author

ojii commented Sep 4, 2021

I reckon it would be better to catch the OverflowError in a separate except clause with a more explicit error.

Sorry, first time contributing to pydantic. Does that mean just changing the msg or the type of the error? If the latter, is there a way type codes are chosen?

except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
except (TypeError, ValueError):
raise errors.IntegerError()
except OverflowError:
raise errors.IntegerOverflowError()

I think this is meant.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then, like 144

-        return float(v)
+        rv = float(v)
+        if rv in (float("inf"), float("-inf")):
+            raise errors.FloatOverflowError()
+        if rv != rv:
+            raise errors.FloatNanError()
+        return rv

P.S. the weird line of code from https://stackoverflow.com/a/44154660/705086

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

floats are a separate case. I would say that float('inf') is a valid float, arguably so is float('nan').

Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, just a few things to fix and more test cases to add.

except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

floats are a separate case. I would say that float('inf') is a valid float, arguably so is float('nan').

Model(a=2.2250738585072011e308)
assert exc_info.value.errors() == [
{'loc': ('a',), 'msg': 'value is not a valid integer', 'type': 'type_error.integer'}
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the case of passing float('inf') etc.

@samuelcolvin
Copy link
Member

please update.

@github-actions github-actions bot added the awaiting author revision awaiting changes from the PR author label Aug 4, 2022
@github-actions github-actions bot assigned ojii and unassigned samuelcolvin and PrettyWood Aug 4, 2022
@ojii
Copy link
Contributor Author

ojii commented Aug 5, 2022

This is looking good, just a few things to fix and more test cases to add.

@samuelcolvin i've added more test cases, do you think the error code/msg should be handled different? If so, how are new messages/codes decided?

@samuelcolvin
Copy link
Member

samuelcolvin commented Aug 5, 2022

At this stage, best if you can roughly follow the code from pydantic-core to minimize the incompatibility with v2.

@ojii
Copy link
Contributor Author

ojii commented Aug 5, 2022

Since the structure of the error dict is different between v1 and v2 anyway, does it make sense trying to make it v2-like? I feel making it fit in with the rest of v1 makes more sense (eg in v1, error codes seem to always be prefixed with type_error. or value_error. but in v2 not?)

@samuelcolvin
Copy link
Member

Yes, that's what I meant by roughly follow, e.g. use value_error.int_nan?

@ojii
Copy link
Contributor Author

ojii commented Aug 5, 2022

Yes, that's what I meant by roughly follow, e.g. use value_error.int_nan?

When I change it to that, interestingly enough float('nan') gets caught in the already existing exceptions (ValueError). This proposed change (catching OverflowError) only catches float('inf') or other huge floats, so int_nan is probably not the right thing here.

Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let's just stick with this then.

@samuelcolvin samuelcolvin enabled auto-merge (squash) August 5, 2022 11:08
@samuelcolvin
Copy link
Member

Good catch, thanks so much for the PR.

@samuelcolvin samuelcolvin merged commit 88ade4b into pydantic:master Aug 5, 2022
@ojii ojii deleted the int-overflow branch August 6, 2022 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting author revision awaiting changes from the PR author
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants