From 34a7b6505a2c924ef12d674795d9ff0b6be7e77f Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 07:46:55 -0400 Subject: [PATCH 01/11] Implicit Annotation Expression Values: Delete section --- peps/pep-0747.rst | 53 ----------------------------------------------- 1 file changed, 53 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 2bf954c1afe..7954242dadd 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -466,59 +466,6 @@ New kinds of type expressions that are introduced should define how they will be recognized in a value expression context. -Implicit Annotation Expression Values -''''''''''''''''''''''''''''''''''''' - -Although this PEP is mostly concerned with *type expressions* rather than -*annotation expressions*, it is straightforward to extend the rules for -:ref:`recognizing type expressions ` -to similar rules for recognizing annotation expressions, -so this PEP takes the opportunity to define those rules as well: - -The following **unparameterized annotation expressions** can be recognized unambiguously: - -- As a value expression, ``X`` has type ``object``, - for each of the following values of X: - - - ```` - -The following **parameterized annotation expressions** can be recognized unambiguously: - -- As a value expression, ``X`` has type ``object``, - for each of the following values of X: - - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - - `` '[' ... ']'`` - -**Annotated**: The annotation expression ``Annotated[...]`` is ambiguous with -the type expression ``Annotated[...]``, -so must be :ref:`disambiguated based on its argument type `. - -The following **syntactic annotation expressions** -cannot be recognized in a value expression context at all: - -- ``'*' unpackable`` -- ``name '.' 'args'`` (where ``name`` must be an in-scope ParamSpec) -- ``name '.' 'kwargs'`` (where ``name`` must be an in-scope ParamSpec) - -The **stringified annotation expression** ``"T"`` is ambiguous with both -the stringified type expression ``"T"`` -and the string literal ``"T"``, and -cannot be recognized in a value expression context at all: - -- As a value expression, ``"T"`` continues to have type ``Literal["T"]``. - -No other kinds of annotation expressions currently exist. - -New kinds of annotation expressions that are introduced should define how they -will (or will not) be recognized in a value expression context. - - Literal[] TypeExprs ''''''''''''''''''' From 1cb02d0eece3f1ad794339f060f458e78308f9ea Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 07:49:26 -0400 Subject: [PATCH 02/11] Subtyping: Replace 'plain type' -> 'unparameterized type' --- peps/pep-0747.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 7954242dadd..c41702ab278 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -523,8 +523,8 @@ determined by the following rules: - ``type[C1]`` is a subtype of ``TypeExpr[C2]`` iff ``C1`` is a subtype of ``C2``. -A plain ``type`` can be assigned to a plain ``TypeExpr`` but not the -other way around: +An unparameterized ``type`` can be assigned to an unparameterized ``TypeExpr`` +but not the other way around: - ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the other way around.) From 46842ba88ed0b723079ce827960dc70f0bf23aef Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 07:54:38 -0400 Subject: [PATCH 03/11] Literal[] TypeExprs: Rephrase. Add new justification. Add clarifying example. --- peps/pep-0747.rst | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index c41702ab278..d5b2fc7765f 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -469,15 +469,21 @@ will be recognized in a value expression context. Literal[] TypeExprs ''''''''''''''''''' -To simplify static type checking, a ``Literal[...]`` value is *not* -considered assignable to a ``TypeExpr`` variable even if all of its members -spell valid types: +A value of ``Literal[...]`` type is *not* considered assignable to +a ``TypeExpr`` variable even if all of its members spell valid types because +dynamic values are not allowed in type expressions: :: STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str' STRS_TYPE: TypeExpr = STRS_TYPE_NAME # ERROR: Literal[] value is not a TypeExpr +However ``Literal[...]`` itself is still a ``TypeExpr``: + +:: + + DIRECTION_TYPE: TypeExpr = Literal['left', 'right'] # OK + Static vs. Runtime Representations of TypeExprs ''''''''''''''''''''''''''''''''''''''''''''''' From 7b7e45eb5d553975a451c285a462728f82fb32ac Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 08:10:15 -0400 Subject: [PATCH 04/11] Alter unparameterized TypeExpr to mean TypeExpr[Any] --- peps/pep-0747.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index d5b2fc7765f..fe65b88b4b5 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -254,9 +254,8 @@ A ``TypeExpr`` value represents a :ref:`type expression such as ``str | None``, ``dict[str, int]``, or ``MyTypedDict``. A ``TypeExpr`` type is written as ``TypeExpr[T]`` where ``T`` is a type or a type variable. It can also be -written without brackets as just ``TypeExpr``, in which case a type -checker should apply its usual type inference mechanisms to determine -the type of its argument, possibly ``Any``. +written without brackets as just ``TypeExpr``, which is treated the same as +to ``TypeExpr[Any]``. Using TypeExprs @@ -278,7 +277,6 @@ or a variable type: :: STR_TYPE: TypeExpr = str # variable type - assert_type(STR_TYPE, TypeExpr[str]) Note however that an *unannotated* variable assigned a type expression literal will not be inferred to be of ``TypeExpr`` type by type checkers because PEP From 515ecb135069dd7d040c2f85e582195ecb393521 Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 08:10:31 -0400 Subject: [PATCH 05/11] Optional[T] -> T | None --- peps/pep-0747.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index fe65b88b4b5..6fa444033f5 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -350,7 +350,7 @@ not spell a type are not ``TypeExpr`` values. :: OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None] # OK - assert isassignable(Optional[int], OPTIONAL_INT_TYPE) + assert isassignable(int | None, OPTIONAL_INT_TYPE) .. _non_universal_typeexpr: From 26204a70971d725b75869d43d541c1b7b68cd052 Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 08:13:35 -0400 Subject: [PATCH 06/11] Reference Implemention: typing_extensions implemention now exists --- peps/pep-0747.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 6fa444033f5..07ebabce8c7 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -842,8 +842,9 @@ The following will be true when `mypy#9773 `__ is implemented: The mypy type checker supports ``TypeExpr`` types. - A reference implementation of the runtime component is provided in the - ``typing_extensions`` module. + +A reference implementation of the runtime component is provided in the +``typing_extensions`` module. Rejected Ideas From 71b5c1ad7589797426ea8ba65f86cba8695890e9 Mon Sep 17 00:00:00 2001 From: David Foster Date: Tue, 2 Jul 2024 08:27:24 -0400 Subject: [PATCH 07/11] How to Teach This: Contrast TypeExpr with TypeAlias --- peps/pep-0747.rst | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 07ebabce8c7..9a15a7fd2e9 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -668,6 +668,32 @@ spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``. including those with brackets (like ``list[int]``) or pipes (like ``int | None``), and including special types like ``Any``, ``LiteralString``, or ``Never``. +A ``TypeExpr`` variable looks similar to a ``TypeAlias`` definition, but +can only be used where a dynamic value is expected. +``TypeAlias`` (and the ``type`` statement) by contrast define a name that can +be used where a fixed type is expected: + +- Okay, but discouraged in Python 3.12+: + + :: + + MaybeFloat: TypeAlias = float | None + def sqrt(n: float) -> MaybeFloat: ... + +- Yes: + + :: + + type MaybeFloat = float | None + def sqrt(n: float) -> MaybeFloat: ... + +- No: + + :: + + maybe_float: TypeExpr = float | None + def sqrt(n: float) -> maybe_float: ... # ERROR: Can't use TypeExpr value in a type annotation + It is uncommon for a programmer to define their *own* function which accepts a ``TypeExpr`` parameter or returns a ``TypeExpr`` value. Instead it is more common for a programmer to pass a literal type expression to an *existing* function From c20c9b1591a903202a9639b8df0461c837a62a49 Mon Sep 17 00:00:00 2001 From: David Foster Date: Thu, 4 Jul 2024 08:20:00 -0400 Subject: [PATCH 08/11] Fix rules related to the value expression (T1 | T2) and UnionType --- peps/pep-0747.rst | 96 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 86 insertions(+), 10 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 9a15a7fd2e9..2fe122d36a5 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -440,14 +440,29 @@ so must be disambiguated based on its argument type: - As a value expression, ``Annotated[x, ...]`` has type ``object`` if ``x`` has a type that is not ``type[C]`` or ``TypeExpr[T]``. -**Union**: The type expression ``T1 | T2`` is ambiguous with the value ``int1 | int2``, -so must be disambiguated based on its argument type: +**Union**: The type expression ``T1 | T2`` is ambiguous with +the value ``int1 | int2``, ``set1 | set2``, ``dict1 | dict2``, and more, +so must be disambiguated based on its argument types: + +- As a value expression, ``x | y`` has type equal to the return type of ``type(x).__or__`` + if ``type(x)`` overrides the ``__or__`` method. + + - When ``x`` has type ``builtins.type``, ``types.GenericAlias``, or the + internal type of a typing special form, ``type(x).__or__`` has a return type + in the format ``TypeExpr[T1 | T2]``. + +- As a value expression, ``x | y`` has type equal to the return type of ``type(y).__ror__`` + if ``type(y)`` overrides the ``__ror__`` method. -- As a value expression, ``x | y`` has type ``TypeExpr[x | y]`` - if ``x`` has type ``TypeExpr[t1]`` (or ``type[t1]``) - and ``y`` has type ``TypeExpr[t2]`` (or ``type[t2]``). -- As a value expression, ``x | y`` has type ``int`` - if ``x`` has type ``int`` and ``y`` has type ``int`` + - When ``y`` has type ``builtins.type``, ``types.GenericAlias``, or the + internal type of a typing special form, ``type(y).__ror__`` has a return type + in the format ``TypeExpr[T1 | T2]``. + +- As a value expression, ``x | y`` has type ``UnionType`` + in all other situations. + + - This rule is intended to be consistent with the preexisting fallback rule + used by static type checkers. The **stringified type expression** ``"T"`` is ambiguous with both the stringified annotation expression ``"T"`` @@ -520,6 +535,9 @@ Subtyping Whether a ``TypeExpr`` value can be assigned from one variable to another is determined by the following rules: +Relationship with type +'''''''''''''''''''''' + ``TypeExpr[]`` is covariant in its argument type, just like ``type[]``: - ``TypeExpr[T1]`` is a subtype of ``TypeExpr[T2]`` iff ``T1`` is a @@ -533,6 +551,22 @@ but not the other way around: - ``type[Any]`` is assignable to ``TypeExpr[Any]``. (But not the other way around.) +Relationship with UnionType +''''''''''''''''''''''''''' + +``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is a non-empty union type: + +- ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``. +- ``TypeExpr[Union[X, Y, ...]]`` is a subtype of ``UnionType``. +- ``TypeExpr[Optional[X]]`` is a subtype of ``UnionType``. +- ``TypeExpr[Never]`` is *not* a subtype of ``UnionType``. +- ``TypeExpr[NoReturn]`` is *not* a subtype of ``UnionType``. + +``UnionType`` is assignable to ``TypeExpr[Any]``. + +Relationship with object +'''''''''''''''''''''''' + ``TypeExpr[]`` is a kind of ``object``, just like ``type[]``: - ``TypeExpr[T]`` for any ``T`` is a subtype of ``object``. @@ -574,11 +608,33 @@ Changed signatures '''''''''''''''''' The following signatures related to type expressions introduce -``TypeExpr`` where previously ``object`` existed: +``TypeExpr`` where previously ``object`` or ``Any`` existed: - ``typing.cast`` - ``typing.assert_type`` +The following signatures transforming union type expressions introduce +``TypeExpr`` where previously ``UnionType`` existed so that a more-precise +``TypeExpr`` type can be inferred: + +- ``builtins.type[T].__or__`` + + - Old: ``def __or__(self, value: Any, /) -> types.UnionType: ...`` + - New: ``def __or__[T2](self, value: TypeExpr[T2], /) -> TypeExpr[T | T2]: ...`` + +- ``builtins.type[T].__ror__`` + + - Old: ``def __ror__(self, value: Any, /) -> types.UnionType: ...`` + - New: ``def __ror__[T1](self, value: TypeExpr[T1], /) -> TypeExpr[T1 | T]: ...`` + +- ``types.GenericAlias.{__or__,__ror__}`` +- «the internal type of a typing special form»``.{__or__,__ror__}`` + +However the implementations of those methods continue to return ``UnionType`` +instances at runtime so that runtime ``isinstance`` checks like +``isinstance('42', int | str)`` and ``isinstance(int | str, UnionType)`` +continue to work. + Unchanged signatures '''''''''''''''''''' @@ -613,12 +669,32 @@ not propose those changes now: - Returns annotation expressions +The following signatures accepting union type expressions continue +to use ``UnionType``: + +- ``builtins.isinstance`` +- ``builtins.issubclass`` +- ``typing.get_origin`` (used in an ``@overload``) + +The following signatures transforming union type expressions continue +to use ``UnionType`` because it is not possible to infer a more-precise +``TypeExpr`` type: + +- ``types.UnionType.{__or__,__ror__}`` + Backwards Compatibility ======================= -Previously the rules for recognizing type expression objects -in a value expression context were not defined, so static type checkers +As a value expression, ``X | Y`` previously had type ``UnionType`` (via :pep:`604`) +but this PEP gives it the more-precise static type ``TypeExpr[X | Y]`` +(a subtype of ``UnionType``) while continuing to return a ``UnionType`` instance at runtime. +Preserving compability with ``UnionType`` is important because ``UnionType`` +supports ``isinstance`` checks, unlike ``TypeExpr``, and existing code relies +on being able to perform those checks. + +The rules for recognizing other kinds of type expression objects +in a value expression context were not previously defined, so static type checkers `varied in what types were assigned `_ to such objects. Existing programs manipulating type expression objects were already limited in manipulating them as plain ``object`` values, From 463a1336eaf1ebbffb4fd0f41e496fb9afbc4c26 Mon Sep 17 00:00:00 2001 From: David Foster Date: Fri, 5 Jul 2024 10:06:13 -0400 Subject: [PATCH 09/11] Be explicit about what kind of TypeExpr a Literal[...] is Co-authored-by: Jelle Zijlstra --- peps/pep-0747.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 2fe122d36a5..423c96a1ece 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -495,7 +495,7 @@ However ``Literal[...]`` itself is still a ``TypeExpr``: :: - DIRECTION_TYPE: TypeExpr = Literal['left', 'right'] # OK + DIRECTION_TYPE: TypeExpr[Literal['left', 'right']] = Literal['left', 'right'] # OK Static vs. Runtime Representations of TypeExprs From 7863e2fba2c48dd8b2bbc441e1f66e6f3d440b67 Mon Sep 17 00:00:00 2001 From: David Foster Date: Mon, 8 Jul 2024 07:50:07 -0400 Subject: [PATCH 10/11] Relationship with UnionType: Retain only the rule required for backward compatibility with: X | Y | ... --- peps/pep-0747.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index 423c96a1ece..c7b504a30ad 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -554,13 +554,10 @@ but not the other way around: Relationship with UnionType ''''''''''''''''''''''''''' -``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is a non-empty union type: +``TypeExpr[U]`` is a subtype of ``UnionType`` iff ``U`` is +the type expression ``X | Y | ...``: - ``TypeExpr[X | Y | ...]`` is a subtype of ``UnionType``. -- ``TypeExpr[Union[X, Y, ...]]`` is a subtype of ``UnionType``. -- ``TypeExpr[Optional[X]]`` is a subtype of ``UnionType``. -- ``TypeExpr[Never]`` is *not* a subtype of ``UnionType``. -- ``TypeExpr[NoReturn]`` is *not* a subtype of ``UnionType``. ``UnionType`` is assignable to ``TypeExpr[Any]``. From af8ea3de3c935c3472b4c88da83f9832b9206f2f Mon Sep 17 00:00:00 2001 From: David Foster Date: Mon, 8 Jul 2024 21:59:17 -0400 Subject: [PATCH 11/11] Grammar fix --- peps/pep-0747.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0747.rst b/peps/pep-0747.rst index c7b504a30ad..98bc6f080e5 100644 --- a/peps/pep-0747.rst +++ b/peps/pep-0747.rst @@ -735,7 +735,7 @@ assigned to variables and manipulated like any other data in a program: ``TypeExpr[]`` is how you spell the type of a variable containing a type annotation object describing a type. -``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only used to +``TypeExpr[]`` is similar to ``type[]``, but ``type[]`` can only spell simple **class objects** like ``int``, ``str``, ``list``, or ``MyClass``. ``TypeExpr[]`` by contrast can additionally spell more complex types, including those with brackets (like ``list[int]``) or pipes (like ``int | None``),