Spec: Define "type expression", "annotation expression", "type qualifier" #1693

JelleZijlstra · 2024-04-06T15:23:17Z

As discussed in https://discuss.python.org/t/basic-terminology-for-types-and-type-forms/46741

This is not meant to prescribe any concrete change in specified behavior, but it makes the spec more precise and explicit. I added several crosslinks where appropriate and added glossary entries for the new terms.

…ier" As discussed in https://discuss.python.org/t/basic-terminology-for-types-and-type-forms/46741

erictraut

This is a great addition to the typing spec. It provides significant clarification on a bunch of issues that have caused confusion for a long time, and it gives us a much stronger foundation to build on in the future. Thanks for doing this, Jelle!

Overall, it looks good to me. I added a couple of minor suggestions.

docs/spec/annotations.rst

docs/spec/class-compat.rst

docs/spec/historical.rst

docs/spec/annotations.rst

gvanrossum

Found a small number of nits, and one suggestion (breaking lines) that you can take or leave.

gvanrossum · 2024-04-11T00:37:51Z

docs/spec/annotations.rst

+The terms *type expression* and *annotation expression* denote specific
+subsets of Python expressions that are used in the type system.


Would it make sense to clarify that all type expressions are annotation expressions, but some annotation expressions aren't type expressions? (E.g. ClassDef[...].)

gvanrossum · 2024-04-11T00:48:54Z

docs/spec/annotations.rst

+
+The following grammar describes the allowed elements of type and annotation expressions:
+
+.. productionlist:: expression-grammar


The rendered version of this block requires a lot of horizontal scrolling. Maybe manually break up the longer lines?

Broke up the lines so that it no longer scrolls horizontally on my laptop.

gvanrossum · 2024-04-11T00:51:20Z

docs/spec/annotations.rst

+
+* ``<Name>`` refers to a :term:`special form`. Most special forms must be imported
+  from :py:mod:`typing` or ``typing_extensions``, except for ``None``, ``type``,
+  ``tuple``, and ``InitVar``. The latter two have aliases in :py:mod:`typing`: :py:class:`typing.Type`


I think "the latter two" no longer refers to type and tuple.

Reordered them so this is true again

gvanrossum · 2024-04-11T00:52:29Z

docs/spec/annotations.rst

+  acceptable in any annotation or type expression, including ``Generic``, ``Protocol``,
+  and ``TypedDict``.
+
+* Any leaf denoted as ``name`` may also be a qualified name (i.e., ``module '.' name``).


Or multiply-qualified, e.g. m1.m2.name.

Added an extra example

gvanrossum · 2024-04-11T00:53:17Z

docs/spec/annotations.rst

+                   : | `string_annotation`
+                   :       (must evaluate to a valid `type_expression`)
+    maybe_unpacked: `type_expression` | `unpacked`
+    unpacked: '*' `unpackable``


Stray extra backtick.

…ython#1684) Conformance tests: Mark error for unannotated "field()" as optional, plus pytype changes The spec doesn't explicitly mention this case. Since it produces a runtime error, it is reasonable for type checkers to error on it, but it should not be required. For pytype, mark a few errors as optionally appearing on a different line. Also, change "x ** 2" to "x * x"; pytype infers "x ** 2" as being of type "float | int" for whatever reason.

…rsion_platform (python#1685)

…on#1687)

…on#1689)

…on#1691)

python#1699) Part of python#1692

…1700) Pyright produces an error unrelated to the purpose of the test, and I don't see a good way to write the test to avoid the error. Part of python#1692

…#1702) Pyright produces warnings instead of errors for many issues in this test case. Part of python#1692

I can't find the quoted phrase in the spec and I don't see a reason why this should error. Part of python#1692

It does not produce an error for `x: Literal[4]; assert_type(x, int)`. Part of python#1692

Pyre and pytype produced an expected error on the wrong line. Part of python#1692

It fails to produce errors on a few lines expected by the test. Also fix an unrelated pytype error and ignore an unrelated pyre error. Part of python#1692

python#1701) Part of python#1692

@rchen152

* Added draft chapter to typing spec for constructors. * Update docs/spec/constructors.rst Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Update docs/spec/constructors.rst Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Update docs/spec/constructors.rst Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Added section on signature consistency between `__new__` and `__init__`. * Incorporated PR feedback. * Added clarification based on question in forum. * Incorporated feedback about callable conversion. Clarified behaviors when a class doesn't inherit a `__new__` or `__init__` from a class other than `object`. * Tweaked the spec based on Jelle's feedback about a `__new__` method that returns `Any`. Also clarified what happens when a call to `__call__` or `__new__` evaluates to a union type. * Updated handling of `Any` return types for `__call__` and `__new__` methods to reflect suggestion from @rchen152 in [this post](https://discuss.python.org/t/draft-typing-spec-chapter-for-constructors/49744/22). * Incorporated feedback from @gvanrossum. --------- Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

python#1708) Fixed some formatting issues in the new "constructor" chapter that caused problems in the table of contents.

erictraut · 2024-04-15T17:40:05Z

@JelleZijlstra, we should think about whether the conformance tests need to be updated for this section. I think that this test already covers most of it, but it would be good to do a review of the test and see if there's anything missing.

Do you want to do this? If not, I'm happy to add it to my to-do list.

JelleZijlstra · 2024-04-15T17:42:26Z

I think more tests would be helpful. I probably won't have time soon to work on them.

erictraut · 2024-04-16T02:43:28Z

I reviewed the new section with test cases in mind. I think our coverage is really high already. I didn't see anything obvious that was missing. Some of the coverage is spread out among various tests. The basic type forms are tested in annotations_typeexpr.py, Annotated is covered in qualifiers_annotated.py, ClassVar is covered in classes_classvar.py, Required and NotRequired are covered in typeddicts_required.py, etc.

Spec: Define "type expression", "annotation expression", "type qualif…

a905e16

…ier" As discussed in https://discuss.python.org/t/basic-terminology-for-types-and-type-forms/46741

JelleZijlstra added the Typing Council decision Needs to be approved by the Typing Council. Do not merge until approved. label Apr 6, 2024

erictraut reviewed Apr 6, 2024

View reviewed changes

JelleZijlstra added 3 commits April 7, 2024 06:20

Fix and improve wording

4a17ca2

Remove covariance reference from ClassVar

ba1f544

"shortcut" is questionable

61bb52a

JelleZijlstra mentioned this pull request Apr 8, 2024

Incorrect error when quoted type annotation is an f-string microsoft/pyright#7640

Closed

JelleZijlstra commented Apr 8, 2024

View reviewed changes

docs/spec/annotations.rst Outdated Show resolved Hide resolved

Update docs/spec/annotations.rst

5207bbc

JelleZijlstra mentioned this pull request Apr 8, 2024

Typing spec update for defining "type expression" and other terms python/typing-council#26

Closed

5 tasks

hauntsaninja approved these changes Apr 10, 2024

View reviewed changes

gvanrossum reviewed Apr 11, 2024

View reviewed changes

JelleZijlstra and others added 18 commits April 11, 2024 05:55

Update PEPs list (python#1695)

c97386e

Spec: Replace various direct references to PEPs (python#1694)

83908d9

Fix heading levels and a link (python#1697)

1805874

Conformance test suite: Fix automation discrepancies in directives_ve…

21055d9

…rsion_platform (python#1685)

Conformance tests: Fix automated checking on overloads_basic.py (pyth…

46ad319

…on#1687)

Conformance tests: Update Never test (python#1688)

3bf8c28

Conformance tests: Allow mypy error on typeddicts_alt_syntax.py (pyth…

6d1a3c9

…on#1689)

Conformance tests: Allow TD.get(known_key) to be T | None (python#1690)

07240ea

Conformance tests: Fix automated scoring in classes_override.py (pyth…

d98f259

…on#1691)

Replace PEP list with link (python#1696)

5ef71e2

Conformance tests: Fix automated scoring for pyright dataclasses tests (

ad9fe6e

python#1699) Part of python#1692

Conformance tests: Fix generics_defaults scoring for pyright (python#…

af96dff

…1700) Pyright produces an error unrelated to the purpose of the test, and I don't see a good way to write the test to avoid the error. Part of python#1692

Conformance tests: Fix pyright scoring for protocols_variance (python…

ddb8697

…#1702) Pyright produces warnings instead of errors for many issues in this test case. Part of python#1692

Conformance tests: Fix tuples_unpacked.py (python#1703)

3398751

I can't find the quoted phrase in the spec and I don't see a reason why this should error. Part of python#1692

Conformance tests: mark pytype as failing assert_type test (python#1706)

8537aac

It does not produce an error for `x: Literal[4]; assert_type(x, int)`. Part of python#1692

conformance tests: fix scoring for typeddicts_inheritance (python#1705)

e253bf6

Pyre and pytype produced an expected error on the wrong line. Part of python#1692

Conformance tests: mark pytype as failing generics_scope (python#1704)

9e9d253

It fails to produce errors on a few lines expected by the test. Also fix an unrelated pytype error and ignore an unrelated pyre error. Part of python#1692

JelleZijlstra and others added 7 commits April 11, 2024 05:55

Update results for pyright 1.1.358 (python#1707)

4c591c3

Conformance tests: Ignore overlapping overload errors in LiteralString (

48e7891

python#1701) Part of python#1692

Fixed some formatting issues in the new "constructor" chapter that ca… (

2a02927

python#1708) Fixed some formatting issues in the new "constructor" chapter that caused problems in the table of contents.

spec: clarify interaction of Final and dataclass (python#1669)

4110874

Merge remote-tracking branch 'upstream/main' into type-expr2

29061b9

Guido feedback

2e9d0cf

gvanrossum approved these changes Apr 11, 2024

View reviewed changes

Add missing | in grammar

ddb75f7

Bigdog000 approved these changes Apr 12, 2024

View reviewed changes

JelleZijlstra merged commit b3b902d into python:main Apr 15, 2024
5 checks passed

JelleZijlstra deleted the type-expr2 branch April 15, 2024 16:49

JelleZijlstra mentioned this pull request May 23, 2024

Spec: Mandate support for typing-extensions #1532

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spec: Define "type expression", "annotation expression", "type qualifier" #1693

Spec: Define "type expression", "annotation expression", "type qualifier" #1693

JelleZijlstra commented Apr 6, 2024

erictraut left a comment

gvanrossum left a comment

gvanrossum Apr 11, 2024

JelleZijlstra Apr 11, 2024

gvanrossum Apr 11, 2024

JelleZijlstra Apr 11, 2024

gvanrossum Apr 11, 2024

JelleZijlstra Apr 11, 2024

gvanrossum Apr 11, 2024

JelleZijlstra Apr 11, 2024

gvanrossum Apr 11, 2024

JelleZijlstra Apr 11, 2024

erictraut commented Apr 15, 2024

JelleZijlstra commented Apr 15, 2024

erictraut commented Apr 16, 2024

		The terms type expression and annotation expression denote specific
		subsets of Python expressions that are used in the type system.


		The following grammar describes the allowed elements of type and annotation expressions:

		.. productionlist:: expression-grammar

Spec: Define "type expression", "annotation expression", "type qualifier" #1693

Spec: Define "type expression", "annotation expression", "type qualifier" #1693

Conversation

JelleZijlstra commented Apr 6, 2024

erictraut left a comment

Choose a reason for hiding this comment

gvanrossum left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erictraut commented Apr 15, 2024

JelleZijlstra commented Apr 15, 2024

erictraut commented Apr 16, 2024