Make slice generic #11637
Draft
Make slice generic #11637
+9
−6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Want to see the primer hits.
This comment has been minimized.
This comment has been minimized.
|
cc @cdce8p I think the primer hits suggest a mypy bug where it looks up "_StartT" in the wrong scope. |
Recursive TypeVar defaults was one of the things that didn't make it into If you're interested, maybe that's also something for @AlexWaygood, a few weeks back I created a custom repo to upload dev releases of mypy to PyPI under the mypy-dev project name. They always track a mypy commit upstream and are great to test / use the lasted version (even in production if you feel like it). https://github.com/cdce8p/mypy-dev |
|
I just assumed my attempt at implementation was wrong and gave up. Should this be reopened, then? |
|
Thanks @cdce8p, I forgot that PR didn't make it into 1.9. Hopefully we'll have another release soon. Your mypy-dev project is interesting; maybe that's even something we could do on mypy itself. For example, we could have a weekly cron job uploading alpha releases. @Sachaa-Thanasius I think we may be able to merge this PR after mypy 1.10 is released (when that will happen, I don't know). You can leave it open in the meantime and mark it as deferred. With some hackery it may be possible to point typeshed's CI at the mypy master branch so we can test earlier whether everything works as expected. |
|
Understood. Should slice be changed in the runtime cpython as well to support |
|
Also, sidenote: The pyright errors specifically for |
This comment has been minimized.
This comment has been minimized.
@Sachaa-Thanasius You could try to replace the mypy version listed in typeshed/requirements-tests.txt Line 8 in 5c75292 -mypy==1.9.0
+mypy-dev==1.10.0a2That release tracks python/mypy@b013cc0 which was commited fairly recently to mypy master (5 days ago). Update: Seems like the mypy primer doesn't work for it, unfortunately. |
srittau
added
the
status: deferred
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Well, that's far more promising than before. Still not sure if this needs to be reflected in cpython by adding a |
srittau
removed
the
status: deferred
|
I've removed the Deferred label. I've just looked at spark primer output. The problem is on their side, since they reuse the |
|
The error that pyright is hitting is that the usage of # Truncated setup.
from collections.abc import Iterator, Sequence
from itertools import combinations, repeat, starmap
from operator import getitem
from typing import TypeVar, cast
_T = TypeVar("_T")
# Actual function
def subslices(seq: Sequence[_T]) -> Iterator[Sequence[_T]]:
slices = starmap(slice, combinations(range(len(seq) + 1), 2))
reveal_type(slices) # Type of "slices" is "starmap[slice[_StartT@slice, _StopT@slice, _StepT@slice]]"
# Adding this cast removes the error on the final line:
# slices = cast(starmap[slice[int, int, int | None]], slices)
# And using a generator instead of starmap removes the error on the final line as well:
# slices = (slice(*args) for args in combinations(range(len(seq) + 1), 2))
return map(getitem, repeat(seq), slices) # Long error.Full playground example here to recreate the conditions of the error. Might be more of a deficiency in the typing of |
|
I might be missing something obvious but it is not clear to me why the type of $ cat t.py
from typing_extensions import reveal_type
s = slice(10)
reveal_type(s)
reveal_type(s.start)
reveal_type(s.stop)
reveal_type(s.step)
$ python t.py
Runtime type is 'slice'
Runtime type is 'NoneType'
Runtime type is 'int'
Runtime type is 'NoneType'Similarly, the type of So I would expect the definitions of the type variables to be: _StartT = TypeVar("_StartT", default=None)
_StopT = TypeVar("_StopT", default=int)
_StepT = TypeVar("_StepT", default=None) |
|
I won’t lie; I just copy-pasted the example implementation from PEP 696 without much thought, since this PR wasn’t originally opened with the intention of being merged. Haven’t double-checked since. Definitely will take another look though. |
|
Diff from mypy_primer, showing the effect of this PR on open source code: manticore (https://github.com/trailofbits/manticore)
- tests/wasm/json2mc.py:103: note: def __getitem__(self, slice, /) -> list[Any]
+ tests/wasm/json2mc.py:103: note: def __getitem__(self, slice[int, int, int | None], /) -> list[Any]
pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/array_algos/transforms.py:41: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/array_algos/transforms.py:41: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1368: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/algorithms.py:1368: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1368: error: Argument 1 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1368: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/algorithms.py:1379: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/algorithms.py:1379: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1383: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/algorithms.py:1383: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1383: error: Argument 1 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ pandas/core/algorithms.py:1383: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/arrays/arrow/array.py:620: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/io/pytables.py:4951: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/io/pytables.py:4951: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/indexing.py:1427: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/indexing.py:2598: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/indexing.py:2598: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/window/rolling.py:1148: error: Argument 1 to "tuple" has incompatible type "list[slice[int, None, int | None]]"; expected "Iterable[ndarray[Any, dtype[integer[Any]]] | ndarray[Any, dtype[bool_]]]" [arg-type]
+ pandas/core/internals/managers.py:1118: error: Argument 1 to "iget" of "Block" has incompatible type "tuple[slice[int, None, int | None], int]"; expected "int | tuple[int, int] | tuple[slice[int, int, int | None], int]" [arg-type]
+ pandas/core/internals/managers.py:1929: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/internals/blocks.py:2050: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/indexes/datetimelike.py:331: error: Argument 1 to "slice" has incompatible type "ndarray[Any, dtype[signedinteger[Any]]]"; expected "int" [arg-type]
+ pandas/core/indexes/datetimelike.py:331: error: Argument 2 to "slice" has incompatible type "ndarray[Any, dtype[signedinteger[Any]]]"; expected "int" [arg-type]
+ pandas/core/groupby/groupby.py:5271: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/core/groupby/groupby.py:5312: error: Argument 2 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/io/formats/style_render.py:1205: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/io/formats/style.py:1751: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ pandas/io/formats/style.py:3532: error: Argument 1 to "non_reducing_slice" has incompatible type "slice[int, None, int | None] | slice[int, int, int | None] | Sequence[Any] | Index"; expected "slice[int, int, int | None] | Sequence[Any] | Index" [arg-type]
kornia (https://github.com/kornia/kornia)
+ kornia/contrib/models/sam/architecture/mask_decoder.py:92: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web_request.py:612:22: error: Argument 1 to "slice" has incompatible type "Any | None"; expected "int" [arg-type]
+ aiohttp/web_request.py:612:29: error: Argument 2 to "slice" has incompatible type "Any | None"; expected "int" [arg-type]
+ aiohttp/web_fileresponse.py:255:36: error: If condition is always true [redundant-expr]
+ aiohttp/web_fileresponse.py:255:36: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-redundant-expr for more info
werkzeug (https://github.com/pallets/werkzeug)
- tests/test_wrappers.py:522: note: def __setitem__(self, slice, Iterable[tuple[str, str]], /) -> None
+ tests/test_wrappers.py:522: note: def __setitem__(self, slice[int, int, int | None], Iterable[tuple[str, str]], /) -> None
- tests/test_wrappers.py:563: note: def __setitem__(self, slice, Iterable[tuple[str, str]], /) -> None
+ tests/test_wrappers.py:563: note: def __setitem__(self, slice[int, int, int | None], Iterable[tuple[str, str]], /) -> None
spark (https://github.com/apache/spark)
+ python/pyspark/pandas/indexing.py:1034: error: Incompatible types in assignment (expression has type "list[Any]", variable has type "int") [assignment]
+ python/pyspark/pandas/indexing.py:1035: error: Value of type "int" is not indexable [index]
+ python/pyspark/pandas/indexing.py:1035: error: Argument 1 to "len" has incompatible type "int"; expected "Sized" [arg-type]
+ python/pyspark/pandas/indexing.py:1037: error: Incompatible types in assignment (expression has type "list[Any]", variable has type "int") [assignment]
+ python/pyspark/pandas/indexing.py:1038: error: Value of type "int" is not indexable [index]
+ python/pyspark/pandas/indexing.py:1038: error: Argument 1 to "len" has incompatible type "int"; expected "Sized" [arg-type]
+ python/pyspark/pandas/indexing.py:1092: error: Incompatible types in assignment (expression has type "tuple[int]", variable has type "int") [assignment]
+ python/pyspark/pandas/indexing.py:1093: error: Argument 1 to "len" has incompatible type "int"; expected "Sized" [arg-type]
+ python/pyspark/pandas/indexing.py:1098: error: Incompatible types in assignment (expression has type "tuple[int]", variable has type "int") [assignment]
+ python/pyspark/pandas/indexing.py:1099: error: Argument 1 to "len" has incompatible type "int"; expected "Sized" [arg-type]
+ python/pyspark/pandas/indexing.py:1103: error: Argument 1 to "len" has incompatible type "int"; expected "Sized" [arg-type]
operator (https://github.com/canonical/operator)
- ops/framework.py:1280: note: def __getitem__(self, slice, /) -> MutableSequence[Any]
+ ops/framework.py:1280: note: def __getitem__(self, slice[int, int, int | None], /) -> MutableSequence[Any]
- ops/framework.py:1280: note: def __getitem__(self, slice, /) -> Sequence[Any]
+ ops/framework.py:1280: note: def __getitem__(self, slice[int, int, int | None], /) -> Sequence[Any]
- ops/framework.py:1283: note: def __setitem__(self, slice, Iterable[Any], /) -> None
+ ops/framework.py:1283: note: def __setitem__(self, slice[int, int, int | None], Iterable[Any], /) -> None
- ops/framework.py:1287: note: def __delitem__(self, slice, /) -> None
+ ops/framework.py:1287: note: def __delitem__(self, slice[int, int, int | None], /) -> None
pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/test_frame.py:2039: error: Expression is of type "tuple[Index[int], slice[Any, Any, Any]]", not "tuple[Index[int], slice[int, int, int | None]]" [assert-type]
ibis (https://github.com/ibis-project/ibis)
+ ibis/util.py:587: error: Unsupported operand types for + ("IntegerScalar" and "int") [operator]
+ ibis/util.py:591: error: Unsupported operand types for + ("IntegerScalar" and "int") [operator]
- ibis/util.py:602: error: Incompatible types in assignment (expression has type "Value", variable has type "int | None") [assignment]
+ ibis/util.py:602: error: Incompatible types in assignment (expression has type "Value", variable has type "int") [assignment]
- ibis/util.py:603: error: Incompatible return value type (got "tuple[int | None, Any | int]", expected "tuple[int | IntegerScalar, int | IntegerScalar]") [return-value]
+ ibis/util.py:602: error: Unsupported operand types for - ("int" and "IntegerScalar") [operator]
- ibis/selectors.py:646: error: Item "slice" of "slice | Iterable[int | str]" has no attribute "__iter__" (not iterable) [union-attr]
+ ibis/selectors.py:646: error: Item "slice[int, int, int | None]" of "slice[int, int, int | None] | Iterable[int | str]" has no attribute "__iter__" (not iterable) [union-attr]
- ibis/selectors.py:649: error: Item "Iterable[int | str]" of "slice | Iterable[int | str]" has no attribute "start" [union-attr]
+ ibis/selectors.py:649: error: Item "Iterable[int | str]" of "slice[int, int, int | None] | Iterable[int | str]" has no attribute "start" [union-attr]
- ibis/selectors.py:650: error: Item "Iterable[int | str]" of "slice | Iterable[int | str]" has no attribute "stop" [union-attr]
+ ibis/selectors.py:650: error: Item "Iterable[int | str]" of "slice[int, int, int | None] | Iterable[int | str]" has no attribute "stop" [union-attr]
- ibis/selectors.py:651: error: Item "Iterable[int | str]" of "slice | Iterable[int | str]" has no attribute "step" [union-attr]
+ ibis/selectors.py:651: error: Item "Iterable[int | str]" of "slice[int, int, int | None] | Iterable[int | str]" has no attribute "step" [union-attr]
+ ibis/expr/types/strings.py:109: error: Argument 2 to "StringSlice" has incompatible type "int"; expected "Value[Integer, Any] | None" [arg-type]
+ ibis/expr/types/strings.py:109: error: Argument 3 to "StringSlice" has incompatible type "int"; expected "Value[Integer, Any] | None" [arg-type]
+ ibis/expr/types/arrays.py:145: error: Argument 2 to "ArraySlice" has incompatible type "int"; expected "Value[Integer, Any]" [arg-type]
+ ibis/expr/types/arrays.py:145: error: Argument 3 to "ArraySlice" has incompatible type "int"; expected "Value[Integer, Any] | None" [arg-type]
jax (https://github.com/google/jax)
+ jax/_src/op_shardings.py:90: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/sharding_specs.py:77: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/sharding_impls.py:452: error: Dict entry 0 has incompatible type Device?: "tuple[slice[int, None, int | None], ...]"; expected Device?: "tuple[slice[int, int, int | None], ...]" [dict-item]
+ jax/_src/core.py:1964: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/state/indexing.py:83: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/array.py:684: error: Argument 1 has incompatible type "tuple[slice[int, None, int | None], ...]"; expected "tuple[slice[int, int, int | None], ...] | None" [arg-type]
+ jax/_src/numpy/fft.py:92: error: Argument 1 to "map" has incompatible type "type[slice[Any, Any, Any]]"; expected "Callable[[int], slice[Never, int, Never]]" [arg-type]
+ jax/_src/numpy/lax_numpy.py:379: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/numpy/lax_numpy.py:814: error: Argument 1 to "slice" has incompatible type "None"; expected "int" [arg-type]
+ jax/_src/numpy/lax_numpy.py:814: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ jax/experimental/sparse/bcoo.py:1980: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ jax/experimental/sparse/bcoo.py:1981: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
+ jax/experimental/sparse/bcoo.py:1985: error: Argument 2 to "slice" has incompatible type "int"; expected "None" [arg-type]
speedrun.com_global_scoreboard_webapp (https://github.com/Avasam/speedrun.com_global_scoreboard_webapp)
- backend/api/tournament_scheduler_api.py:327: note: def __getitem__(self, slice, /) -> list[Any]
+ backend/api/tournament_scheduler_api.py:327: note: def __getitem__(self, slice[int, int, int | None], /) -> list[Any]
- backend/api/tournament_scheduler_api.py:331: note: def __getitem__(self, slice, /) -> list[Any]
+ backend/api/tournament_scheduler_api.py:331: note: def __getitem__(self, slice[int, int, int | None], /) -> list[Any]
- backend/api/tournament_scheduler_api.py:335: note: def __getitem__(self, slice, /) -> list[Any]
+ backend/api/tournament_scheduler_api.py:335: note: def __getitem__(self, slice[int, int, int | None], /) -> list[Any]
jinja (https://github.com/pallets/jinja)
+ src/jinja2/nodes.py:895: error: Argument 1 to "slice" has incompatible type "Optional[Any]"; expected "int" [arg-type]
+ src/jinja2/nodes.py:895: error: Argument 2 to "slice" has incompatible type "Optional[Any]"; expected "int" [arg-type]
discord.py (https://github.com/Rapptz/discord.py)
- discord/http.py:228: note: def __setitem__(self, slice, Iterable[Embed], /) -> None
+ discord/http.py:228: note: def __setitem__(self, slice[int, int, int | None], Iterable[Embed], /) -> None
streamlit (https://github.com/streamlit/streamlit)
- lib/tests/streamlit/runtime/state/query_params_test.py:242:37: note: def __getitem__(self, slice, /) -> Tuple[Any, ...]
+ lib/tests/streamlit/runtime/state/query_params_test.py:242:37: note: def __getitem__(self, slice[int, int, Optional[int]], /) -> Tuple[Any, ...]
|
srittau
reviewed
May 10, 2024
Comment on lines
+94
to
+96
| _StartT = TypeVar("_StartT", default=int) | ||
| _StopT = TypeVar("_StopT", default=_StartT) | ||
| _StepT = TypeVar("_StepT", default=int | None) |
There was a problem hiding this comment.
Maybe as a very first step, try using Any as default. We could change this later, but this should allow explicit annotations without regressions.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mainly doing this out of curiosity to see the primer hits.
EDIT: Would close #8647.