Friday 19 July 2024

Mypy 1.11 Released

Mypy 1.11 Released

We’ve just uploaded mypy 1.11 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Support Python 3.12 Syntax for Generics (PEP 695)

Mypy now supports the new type parameter syntax introduced in Python 3.12 (PEP 695).

This feature is still experimental and must be enabled with the --enable-incomplete-feature=NewGenericSyntax flag, or with enable_incomplete_feature = NewGenericSyntax in the mypy configuration file.

We plan to enable this by default in the next mypy feature release.

This example demonstrates the new syntax:

    # Generic function
    def fT -> T: ...
    
    reveal_type(f(1))  # Revealed type is 'int'
    
    # Generic class
    class C[T]:
        def __init__(self, x: T) -> None:
           self.x = x
    
    c = C('a')
    reveal_type(c.x)  # Revealed type is 'str'
    
    # Type alias
    type A[T] = C[list[T]]

This feature was contributed by Jukka Lehtosalo.

Support for functools.partial

Mypy now type checks uses of functools.partial. Previously mypy would accept arbitrary arguments.

This example will now produce an error:

    from functools import partial
    
    def f(a: int, b: str) -> None: ...
    
    g = partial(f, 1)
    
    # Argument has incompatible type "int"; expected "str"
    g(11)

This feature was contributed by Shantanu (PR 16939).

Stricter Checks for Untyped Overrides

Past mypy versions didn't check if untyped methods were compatible with overridden methods. This would result in false negatives. Now mypy performs these checks when using --check-untyped-defs.

For example, this now generates an error if using --check-untyped-defs:

    class Base:
        def f(self, x: int = 0) -> None: ...
    
    class Derived(Base):
        # Signature incompatible with "Base"
        def f(self): ...

This feature was contributed by Steven Troxler (PR 17276).

Type Inference Improvements

The new polymorphic inference algorithm introduced in mypy 1.5 is now used in more situations. This improves type inference involving generic higher-order functions, in particular.

This feature was contributed by Ivan Levkivskyi (PR 17348).

Mypy now uses unions of tuple item types in certain contexts to enable more precise inferred types. Example:

    for x in (1, 'x'):
        # Previously inferred as 'object'
        reveal_type(x)  # Revealed type is 'int | str'

This was also contributed by Ivan Levkivskyi (PR 17408).

Improvements to Detection of Overlapping Overloads

The details of how mypy checks if two @overload signatures are unsafely overlapping were overhauled. This both fixes some false positives, and allows mypy to detect additional unsafe signatures.

This feature was contributed by Ivan Levkivskyi (PR 17392).

Better Support for Type Hints in Expressions

Mypy now allows more expressions that evaluate to valid type annotations in all expression contexts. The inferred types of these expressions are also sometimes more precise. Previously they were often object.

This example uses a union type that includes a callable type as an expression, and it no longer generates an error:

    from typing import Callable
    
    print(Callable[[], int] | None)  # No error

This feature was contributed by Jukka Lehtosalo (PR 17404).

Mypyc Improvements

Mypyc now supports the new syntax for generics introduced in Python 3.12 (see above). Another notable improvement is signficantly faster basic operations on int values.

  • Support Python 3.12 syntax for generic functions and classes (Jukka Lehtosalo, PR 17357)
  • Support Python 3.12 type alias syntax (Jukka Lehtosalo, PR 17384)
  • Fix ParamSpec (Shantanu, PR 17309)
  • Inline fast paths of integer unboxing operations (Jukka Lehtosalo, PR 17266)
  • Inline tagged integer arithmetic and bitwise operations (Jukka Lehtosalo, PR 17265)
  • Allow specifying primitives as pure (Jukka Lehtosalo, PR 17263)

Changes to Stubtest

  • Ignore _ios_support (Alex Waygood, PR 17270)
  • Improve support for Python 3.13 (Shantanu, PR 17261)

Changes to Stubgen

  • Gracefully handle invalid Optional and recognize aliases to PEP 604 unions (Ali Hamdan, PR 17386)
  • Fix for Python 3.13 (Jelle Zijlstra, PR 17290)
  • Preserve enum value initialisers (Shantanu, PR 17125)

Miscellaneous New Features

  • Add error format support and JSON output option via --output json (Tushar Sadhwani, PR 11396)
  • Support enum.member in Python 3.11+ (Nikita Sobolev, PR 17382)
  • Support enum.nonmember in Python 3.11+ (Nikita Sobolev, PR 17376)
  • Support namedtuple.__replace__ in Python 3.13 (Shantanu, PR 17259)
  • Support rename=True in collections.namedtuple (Jelle Zijlstra, PR 17247)
  • Add support for __spec__ (Shantanu, PR 14739)

Changes to Error Reporting

  • Mention --enable-incomplete-feature=NewGenericSyntax in messages (Shantanu, PR 17462)
  • Do not report plugin-generated methods with explicit-override (sobolevn, PR 17433)
  • Use and display namespaces for function type variables (Ivan Levkivskyi, PR 17311)
  • Fix false positive for Final local scope variable in Protocol (GiorgosPapoutsakis, PR 17308)
  • Use Never in more messages, use ambiguous in join (Shantanu, PR 17304)
  • Log full path to config file in verbose output (dexterkennedy, PR 17180)
  • Added [prop-decorator] code for unsupported property decorators (Christopher Barber, PR 14461">16571)
  • Suppress second error message with := and [truthy-bool] (Nikita Sobolev, PR 15941)
  • Generate error for assignment of functional Enum to variable of different name (Shantanu, PR 16805)
  • Fix error reporting on cached run after uninstallation of third party library (Shantanu, PR 17420)

Fixes for Crashes

  • Fix daemon crash on invalid type in TypedDict (Ivan Levkivskyi, PR 17495)
  • Fix crash and bugs related to partial() (Ivan Levkivskyi, PR 17423)
  • Fix crash when overriding with unpacked TypedDict (Ivan Levkivskyi, PR 17359)
  • Fix crash on TypedDict unpacking for ParamSpec (Ivan Levkivskyi, PR 17358)
  • Fix crash involving recursive union of tuples (Ivan Levkivskyi, PR 17353)
  • Fix crash on invalid callable property override (Ivan Levkivskyi, PR 17352)
  • Fix crash on unpacking self in NamedTuple (Ivan Levkivskyi, PR 17351)
  • Fix crash on recursive alias with an optional type (Ivan Levkivskyi, PR 17350)
  • Fix crash on type comment inside generic definitions (Bénédikt Tran, PR 16849)

Changes to Documentation

  • Use inline config in documentation for optional error codes (Shantanu, PR 17374)
  • Use lower-case generics in documentation (Seo Sanghyeon, PR 17176)
  • Add documentation for show-error-code-links (GiorgosPapoutsakis, PR 17144)
  • Update CONTRIBUTING.md to include commands for Windows (GiorgosPapoutsakis, PR 17142)

Other Notable Improvements and Fixes

  • Fix ParamSpec inference against TypeVarTuple (Ivan Levkivskyi, PR 17431)
  • Fix explicit type for partial (Ivan Levkivskyi, PR 17424)
  • Always allow lambda calls (Ivan Levkivskyi, PR 17430)
  • Fix isinstance checks with PEP 604 unions containing None (Shantanu, PR 17415)
  • Fix self-referential upper bound in new-style type variables (Ivan Levkivskyi, PR 17407)
  • Consider overlap between instances and callables (Ivan Levkivskyi, PR 17389)
  • Allow new-style self-types in classmethods (Ivan Levkivskyi, PR 17381)
  • Fix isinstance with type aliases to PEP 604 unions (Shantanu, PR 17371)
  • Properly handle unpacks in overlap checks (Ivan Levkivskyi, PR 17356)
  • Fix type application for classes with generic constructors (Ivan Levkivskyi, PR 17354)
  • Update typing_extensions to >=4.6.0 to fix Python 3.12 error (Ben Brown, PR 17312)
  • Avoid "does not return" error in lambda (Shantanu, PR 17294)
  • Fix bug with descriptors in non-strict-optional mode (Max Murin, PR 17293)
  • Don’t leak unreachability from lambda body to surrounding scope (Anders Kaseorg, PR 17287)
  • Fix issues with non-ASCII characters on Windows (Alexander Leopold Shon, PR 17275)
  • Fix for type narrowing of negative integer literals (gilesgc, PR 17256)
  • Fix confusion between .py and .pyi files in mypy daemon (Valentin Stanciu, PR 17245)
  • Fix type of tuple[X, Y] expression (urnest, PR 17235)
  • Don't forget that a TypedDict was wrapped in Unpack after a name-defined error occurred (Christoph Tyralla, PR 17226)
  • Mark annotated argument as having an explicit, not inferred type (bzoracler, PR 17217)
  • Don't consider Enum private attributes as enum members (Ali Hamdan, PR 17182)
  • Fix Literal strings containing pipe characters (Jelle Zijlstra, PR 17148)

Typeshed Updates

Please see git log for full list of standard library typeshed stub changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Alexander Leopold Shon
  • Ali Hamdan
  • Anders Kaseorg
  • Ben Brown
  • Bénédikt Tran
  • bzoracler
  • Christoph Tyralla
  • Christopher Barber
  • dexterkennedy
  • gilesgc
  • GiorgosPapoutsakis
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • Jukka Lehtosalo
  • Marc Mueller
  • Matthieu Devlin
  • Michael R. Crusoe
  • Nikita Sobolev
  • Seo Sanghyeon
  • Shantanu
  • sobolevn
  • Steven Troxler
  • Tadeu Manoel
  • Tamir Duberstein
  • Tushar Sadhwani
  • urnest
  • Valentin Stanciu

I’d also like to thank my employer, Dropbox, for supporting mypy development.

Wednesday 24 April 2024

Mypy 1.10 Released

Mypy 1.10 Released

We’ve just uploaded mypy 1.10 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Support TypeIs (PEP 742)

Mypy now supports TypeIs (PEP 742), which allows functions to narrow the type of a value, similar to isinstance(). Unlike TypeGuard, TypeIs can narrow in both the if and else branches of an if statement:

    from typing_extensions import TypeIs
    
    def is_str(s: object) -> TypeIs[str]:
        return isinstance(s, str)
    
    def f(o: str | int) -> None:
        if is_str(o):
            # Type of o is 'str'
            ...
        else:
            # Type of o is 'int'
            ...
TypeIs will be added to the typing module in Python 3.13, but it can be used on earlier Python versions by importing it from typing_extensions.

This feature was contributed by Jelle Zijlstra (PR 16898).

Support TypeVar Defaults (PEP 696)

PEP 696 adds support for type parameter defaults.

Example:

    from typing import Generic
    from typing_extensions import TypeVar
    
    T = TypeVar("T", default=int)
    
    class C(Generic[T]):
       ...
    
    x: C = ...
    y: C[str] = ...
    reveal_type(x)  # C[int], because of the default
    reveal_type(y)  # C[str]

TypeVar defaults will be added to the typing module in Python 3.13, but they can be used with earlier Python releases by importing TypeVar from typing_extensions.

This feature was contributed by Marc Mueller (PR 16878 and PR 16925).

Support TypeAliasType (PEP 695)

As part of the initial steps towards implementing PEP 695, mypy now supports TypeAliasType. TypeAliasType provides a backport of the new type statement in Python 3.12.

    type ListOrSet[T] = list[T] | set[T]
is equivalent to:
    T = TypeVar("T")
    ListOrSet = TypeAliasType("ListOrSet", list[T] | set[T], type_params=(T,))

Example of use in mypy:

    from typing_extensions import TypeAliasType, TypeVar
    
    NewUnionType = TypeAliasType("NewUnionType", int | str)
    x: NewUnionType = 42
    y: NewUnionType = 'a'
    z: NewUnionType = object()
    # error: Incompatible types in assignment (expression has type "object",
    # variable has type "int | str")  [assignment]
    
    T = TypeVar("T")
    ListOrSet = TypeAliasType("ListOrSet", list[T] | set[T], type_params=(T,))
    a: ListOrSet[int] = [1, 2]
    b: ListOrSet[str] = {'a', 'b'}
    c: ListOrSet[str] = 'test'
    # error: Incompatible types in assignment (expression has type "str",
    # variable has type "list[str] | set[str]")  [assignment]
TypeAliasType was added to the typing module in Python 3.12, but it can be used with earlier Python releases by importing from typing_extensions.

This feature was contributed by Ali Hamdan (PR 16926, PR 17038 and PR 17053)

Detect Additional Unsafe Uses of super()

Mypy will reject unsafe uses of super() more consistently, when the target has a trivial (empty) body. Example:

    class Proto(Protocol):
        def method(self) -> int: ...
    
    class Sub(Proto):
        def method(self) -> int:
            return super().meth()  # Error (unsafe)

This feature was contributed by Shantanu (PR 16756).

Stubgen Improvements

  • Preserve empty tuple annotation (Ali Hamdan, PR 16907)
  • Add support for PEP 570 positional-only parameters (Ali Hamdan, PR 16904)
  • Replace obsolete typing aliases with builtin containers (Ali Hamdan, PR 16780)
  • Fix generated dataclass __init__ signature (Ali Hamdan, PR 16906)

Mypyc Improvements

  • Provide an easier way to define IR-to-IR transforms (Jukka Lehtosalo, PR 16998)
  • Implement lowering pass and add primitives for int (in)equality (Jukka Lehtosalo, PR 17027)
  • Implement lowering for remaining tagged integer comparisons (Jukka Lehtosalo, PR 17040)
  • Optimize away some bool/bit registers (Jukka Lehtosalo, PR 17022)
  • Remangle redefined names produced by async with (Richard Si, PR 16408)
  • Optimize TYPE_CHECKING to False at Runtime (Srinivas Lade, PR 16263)
  • Fix compilation of unreachable comprehensions (Richard Si, PR 15721)
  • Don't crash on non-inlinable final local reads (Richard Si, PR 15719)

Documentation Improvements

  • Import TypedDict from typing instead of typing_extensions (Riccardo Di Maio, PR 16958)
  • Add missing mutable-override to section title (James Braza, PR 16886)

Error Reporting Improvements

  • Use lower-case generics more consistently in error messages (Jukka Lehtosalo, PR 17035)

Other Notable Changes and Fixes

  • Fix incorrect inferred type when accessing descriptor on union type (Matthieu Devlin, PR 16604)
  • Fix crash when expanding invalid Unpack in a Callable alias (Ali Hamdan, PR 17028)
  • Fix false positive when string formatting with string enum (roberfi, PR 16555)
  • Narrow individual items when matching a tuple to a sequence pattern (Loïc Simon, PR 16905)
  • Fix false positive from type variable within TypeGuard or TypeIs (Evgeniy Slobodkin, PR 17071)
  • Improve yield from inference for unions of generators (Shantanu, PR 16717)
  • Fix emulating hash method logic in attrs classes (Hashem, PR 17016)
  • Add reverted typeshed commit that uses ParamSpec for functools.wraps (Tamir Duberstein, PR 16942)
  • Fix type narrowing for types.EllipsisType (Shantanu, PR 17003)
  • Fix single item enum match type exhaustion (Oskari Lehto, PR 16966)
  • Improve type inference with empty collections (Marc Mueller, PR 16994)
  • Fix override checking for decorated property (Shantanu, PR 16856)
  • Fix narrowing on match with function subject (Edward Paget, PR 16503)
  • Allow +N within Literal[...] (Spencer Brown, PR 16910)
  • Experimental: Support TypedDict within type[...] (Marc Mueller, PR 16963)
  • Experimtental: Fix issue with TypedDict with optional keys in type[...] (Marc Mueller, PR 17068)

Typeshed Updates

Please see git log for full list of standard library typeshed stub changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Ali Hamdan
  • Edward Paget
  • Evgeniy Slobodkin
  • Hashem
  • hesam
  • Hugo van Kemenade
  • Ihor
  • James Braza
  • Jelle Zijlstra
  • jhance
  • Jukka Lehtosalo
  • Loïc Simon
  • Marc Mueller
  • Matthieu Devlin
  • Michael R. Crusoe
  • Nikita Sobolev
  • Oskari Lehto
  • Riccardo Di Maio
  • Richard Si
  • roberfi
  • Roman Solomatin
  • Sam Xifaras
  • Shantanu
  • Spencer Brown
  • Srinivas Lade
  • Tamir Duberstein
  • youkaichao

I’d also like to thank my employer, Dropbox, for supporting mypy development.

Friday 8 March 2024

Mypy 1.9 Released

 

Mypy 1.9 Released

We’ve just uploaded mypy 1.9 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Breaking Changes

Because the version of typeshed we use in mypy 1.9 doesn't support 3.7, neither does mypy 1.9. (Jared Hance, PR 16883)

We are planning to enable local partial types (enabled via the --local-partial-types flag) later this year by default. This change was announced years ago, but now it's finally happening. This is a major backward-incompatible change, so we'll probably include it as part of the upcoming mypy 2.0 release. This makes daemon and non-daemon mypy runs have the same behavior by default.

Local partial types can also be enabled in the mypy config file:

    local_partial_types = True

We are looking at providing a tool to make it easier to migrate projects to use --local-partial-types, but it's not yet clear whether this is practical. The migration usually involves adding some explicit type annotations to module-level and class-level variables.

Basic Support for Type Parameter Defaults (PEP 696)

This release contains new experimental support for type parameter defaults (PEP 696). Please try it out! This feature was contributed by Marc Mueller.

Since this feature will be officially introduced in the next Python feature release (3.13), you will need to import TypeVar, ParamSpec or TypeVarTuple from typing_extensions to use defaults for now.

This example adapted from the PEP defines a default for BotT:

    from typing import Generic
    from typing_extensions import TypeVar
    
    class Bot: ...
    
    <p>BotT = TypeVar("BotT", bound=Bot, default=Bot)
    
    class Context(Generic[BotT]):
        bot: BotT
    
    class MyBot(Bot): ...
    
    # type is Bot (the default)
    reveal_type(Context().bot)
    # type is MyBot
    reveal_type(ContextMyBot.bot)

Type-checking Improvements

  • Fix missing type store for overloads (Marc Mueller, PR 16803)
  • Fix 'WriteToConn' object has no attribute 'flush' (Charlie Denton, PR 16801)
  • Improve TypeAlias error messages (Marc Mueller, PR 16831)
  • Support narrowing unions that include type[None] (Christoph Tyralla, PR 16315)
  • Support TypedDict functional syntax as class base type (anniel-stripe, PR 16703)
  • Accept multiline quoted annotations (Shantanu, PR 16765)
  • Allow unary + in Literal (Jelle Zijlstra, PR 16729)
  • Substitute type variables in return type of static methods (Kouroche Bouchiat, PR 16670)
  • Consider TypeVarTuple to be invariant (Marc Mueller, PR 16759)
  • Add alias support to field() in attrs plugin (Nikita Sobolev, PR 16610)
  • Improve attrs hashability detection (Tin Tvrtković, PR 16556)

Performance Improvements

  • Speed up finding function type variables (Jukka Lehtosalo, PR 16562)

Documentation Updates

  • Document supported values for --enable-incomplete-feature in "mypy --help" (Froger David, PR 16661)
  • Update new type system discussion links (thomaswhaley, PR 16841)
  • Add missing class instantiation to cheat sheet (Aleksi Tarvainen, PR 16817)
  • Document how evil --no-strict-optional is (Shantanu, PR 16731)
  • Improve mypy daemon documentation note about local partial types (Makonnen Makonnen, PR 16782)
  • Fix numbering error (Stefanie Molin, PR 16838)
  • Various documentation improvements (Shantanu, PR 16836)

Stubtest Improvements

  • Ignore private function/method parameters when they are missing from the stub (private parameter names start with a single underscore and have a default, PR 16507)
  • Ignore a new protocol dunder (Alex Waygood, PR 16895)
  • Private parameters can be omitted (Sebastian Rittau, PR 16507)
  • Add support for setting enum members to "..." (Jelle Zijlstra, PR 16807)
  • Adjust symbol table logic (Shantanu, PR 16823)
  • Fix posisitional-only handling in overload resolution (Shantanu, PR 16750)

Stubgen Improvements

  • Fix crash on star unpack of TypeVarTuple (Ali Hamdan, PR 16869)
  • Use PEP 604 unions everywhere (Ali Hamdan, PR 16519)
  • Do not ignore property deleter (Ali Hamdan, PR 16781)
  • Support type stub generation for staticmethod (WeilerMarcel, PR 14934)

Acknowledgements

​Thanks to all mypy contributors who contributed to this release:

  • Aleksi Tarvainen
  • Alex Waygood
  • Ali Hamdan
  • anniel-stripe
  • Charlie Denton
  • Christoph Tyralla
  • Dheeraj
  • Fabian Keller
  • Fabian Lewis
  • Froger David
  • Ihor
  • Jared Hance
  • Jelle Zijlstra
  • Jukka Lehtosalo
  • Kouroche Bouchiat
  • Lukas Geiger
  • Maarten Huijsmans
  • Makonnen Makonnen
  • Marc Mueller
  • Nikita Sobolev
  • Sebastian Rittau
  • Shantanu
  • Stefanie Molin
  • Stephen Morton
  • thomaswhaley
  • Tin Tvrtković
  • WeilerMarcel
  • Wesley Collin Wright
  • zipperer

I’d also like to thank my employer, Dropbox, for supporting mypy development.