Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more type annotations #7142

Merged
merged 29 commits into from
Jun 5, 2020
Merged

Add more type annotations #7142

merged 29 commits into from
Jun 5, 2020

Conversation

bluetech
Copy link
Member

@bluetech bluetech commented May 1, 2020

Currently based on: #6999, #7019. Not based on #7121 but would be nice to rebase on it once it's merged. #7253.

This PR adds type annotations to many parts of pytest, enough that we can enable check_untyped_defs = True for all of pytest (including tests). I worked on this intermittently since I started working on pytest, and I think it would be good to have for the planned 6.0 release.

Types act as a "super linter", and makes the code easier to refactor and understand. The types themselves can sometimes be complex, but hopefully not too much, and I usually take it as a hint that the code should be simplified.

Most of the changes are mundane. I kept the code changes to a minimum -- mostly asserts and trivial variable renamings to make the annotations work.

I understand this is hard to review in detail. I hope however to get an ack, based mostly on trust. If this causes regressions or other issues I promise to handle that.

This PR doesn't yet "publish" the types (py.typed file). I'd like to do this too for 6.0, but I think it should be discussed separately, as it has implications for API stability (can type change, should types be exposed for purposes of annotation in user code, etc.) and rollout plans.

Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bluetech and congratulations on reaching a milestone on fully typing pytest. 😁

I'm approving with the caveat that I'm no typing expert, but the types at a glance seem correct, so I will defer to your judgment. 👍

Thanks again.

@RonnyPfannschmidt
Copy link
Member

i started to read trough, and i'll continue tomorrow, so far i like what i see, great work, that must have been a total pain to get so far

@bluetech
Copy link
Member Author

bluetech commented Jun 3, 2020

@nicoddemus @RonnyPfannschmidt Thanks a lot for reviewing this.

I have a rebase ready, will push it after @RonnyPfannschmidt is finished reviewing so it doesn't interfere midway.

@RonnyPfannschmidt
Copy link
Member

Please push the rebase

@RonnyPfannschmidt
Copy link
Member

Since I didn't have a chance to look at that many files, I'd rather review the current state and catch rebase issues rather than continue on the old stuff

@bluetech
Copy link
Member Author

bluetech commented Jun 3, 2020

@RonnyPfannschmidt sure, pushed!

Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i finished my overall readup, great work

i noted some minor details and some followups

thanks for tackling this

@@ -465,27 +465,27 @@ def test_report_extra_parameters(reporttype: "Type[reports.BaseReport]") -> None


def test_callinfo() -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

followup note: the item below could probably be actually 3 tests

@@ -877,7 +886,7 @@ def getini(self, name):
xmlpath = str(tmpdir.join("junix.xml"))
register = gotten.append

fake_config = FakeConfig()
fake_config = cast(Config, FakeConfig())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wasn't aware that one works, its such a helpful tool to go that way with fake objects

nodeid = "test_node_id"

log.pytest_sessionstart()
log.add_global_property("foo", 1)
log.add_global_property("bar", 2)
log.add_global_property("foo", "1")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe we need a followup to warn on non-string properties

@dataclass
class SimpleDataObjectOne:
field_a: int = field()
field_b: int = field()
field_b: str = field()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lovely find

assert w[0].lineno == inspect.currentframe().f_lineno - 1
assert w[0].filename == __file__
nodes.Node(name="test", config=ms, session=ms, nodeid="None") # type: ignore
assert w[0].lineno == inspect.currentframe().f_lineno - 1 # type: ignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was the line drop here intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite remember but seems like it was not intentional, I'll add it back.

@@ -367,13 +402,13 @@ def write(self, content: str, *, flush: bool = False, **markup: bool) -> None:
def flush(self) -> None:
self._tw.flush()

def write_line(self, line, **markup):
def write_line(self, line: Union[str, bytes], **markup: bool) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have any way to ensure this will turn str only?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pytest itself doesn't need the bytes support (linting & tests pass without it), but it's a public method so would be a breaking change technically. I can send a PR to drop it for 6.x if you think we should.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should note it for 7.x, 6.x is a bit close and won't give external users time to get the change, but a deprecation might be nice to add soon

# Will return a dummy fixture if the setuponly option is provided.
if request.config.option.setupplan:
my_cache_key = fixturedef.cache_key(request)
fixturedef.cached_result = (None, my_cache_key, None)
return fixturedef.cached_result
return None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are those explicit returns annotations for mypy?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, mypy has a warning when if one code path has a return <value> then all code paths should.

@bluetech
Copy link
Member Author

bluetech commented Jun 5, 2020

Besides @RonnyPfannschmidt comment, since master updated to mypy 0.780 since last rebase, I also get a few new warnings which I'll fix:

src/_pytest/assertion/rewrite.py:93: error: Invalid self argument "AssertionRewritingHook" to attribute function "_find_spec" with type "Callable[[str, Optional[Sequence[Union[bytes, str]]], Optional[Module]], Optional[ModuleSpec]]"  [misc]
src/_pytest/assertion/rewrite.py:93: error: Argument 2 has incompatible type "Optional[Sequence[Union[str, bytes]]]"; expected "Optional[Module]"  [arg-type]
src/_pytest/capture.py:387: error: Argument 1 to "EncodedFile" has incompatible type "IO[bytes]"; expected "BinaryIO"  [arg-type]
testing/test_debugging.py:898: error: Non-overlapping identity check (left operand type: "Literal[True]", right operand type: "Literal[False]")  [comparison-overlap]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants