Wednesday, 10 May 2023

Mypy 1.3 Released

We’ve just uploaded mypy 1.3 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:
    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Performance Improvements

  • Improve performance of union subtyping (Shantanu, PR 15104)
  • Add negative subtype caches (Ivan Levkivskyi, PR 14884)

Stub Tooling Improvements

  • Stubtest: Check that the stub is abstract if the runtime is, even when the stub is an overloaded method (Alex Waygood, PR 14955)
  • Stubtest: Verify stub methods or properties are decorated with @final if they are decorated with @final at runtime (Alex Waygood, PR 14951)
  • Stubtest: Fix stubtest false positives with TypedDicts at runtime (Alex Waygood, PR 14984)
  • Stubgen: Support @functools.cached_property (Nikita Sobolev, PR 14981)
  • Improvements to stubgenc (Chad Dombrova, PR 14564)

Improvements to attrs

  • Add support for converters with TypeVars on generic attrs classes (Chad Dombrova, PR 14908)
  • Fix attrs.evolve on bound TypeVar (Ilya Konstantinov, PR 15022)

Documentation Updates

  • Improve async documentation (Shantanu, PR 14973)
  • Improvements to cheat sheet (Shantanu, PR 14972)
  • Add documentation for bytes formatting error code (Shantanu, PR 14971)
  • Convert insecure links to use HTTPS (Marti Raudsepp, PR 14974)
  • Also mention overloads in async iterator documentation (Shantanu, PR 14998)
  • stubtest: Improve allowlist documentation (Shantanu, PR 15008)
  • Clarify "Using types... but not at runtime" (Jon Shea, PR 15029)
  • Fix alignment of cheat sheet example (Ondřej Cvacho, PR 15039)
  • Fix error for callback protocol matching against callable type object (Shantanu, PR 15042)

Error Reporting Improvements

  • Improve bytes formatting error (Shantanu, PR 14959)

Mypyc Improvements

  • Fix unions of bools and ints (Tomer Chachamu, PR 15066)

Other Fixes and Improvements

  • Fix narrowing union types that include Self with isinstance (Christoph Tyralla, PR 14923)
  • Allow objects matching SupportsKeysAndGetItem to be unpacked (Bryan Forbes, PR 14990)
  • Check type guard validity for staticmethods (EXPLOSION, PR 14953)
  • Fix sys.platform when cross-compiling with emscripten (Ethan Smith, PR 14888)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Amin Alaee
  • Bryan Forbes
  • Chad Dombrova
  • Charlie Denton
  • Christoph Tyralla
  • dosisod
  • Ethan Smith
  • EXPLOSION
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jon Shea
  • Jukka Lehtosalo
  • KotlinIsland
  • Marti Raudsepp
  • Nikita Sobolev
  • Ondřej Cvacho
  • Shantanu
  • sobolevn
  • Tomer Chachamu
  • Yaroslav Halchenko

Thursday, 6 April 2023

Mypy 1.2 Released

We’ve just uploaded mypy 1.2 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Improvements to Dataclass Transforms

  • Support implicit default for "init" parameter in field specifiers (Wesley Collin Wright and Jukka Lehtosalo, PR 15010)
  • Support descriptors in dataclass transform (Jukka Lehtosalo, PR 15006)
  • Fix frozen_default in incremental mode (Wesley Collin Wright)
  • Fix frozen behavior for base classes with direct metaclasses (Wesley Collin Wright, PR 14878)

Mypyc: Native Floats

Mypyc now uses a native, unboxed representation for values of type float. Previously these were heap-allocated Python objects. Native floats are faster and use less memory. Code that uses floating-point operations heavily can be several times faster when using native floats.

Various float operations and math functions also now have optimized implementations. Refer to the documentation for a full list.

This can change the behavior of existing code that uses subclasses of float. When assigning an instance of a subclass of float to a variable with the float type, it gets implicitly converted to a float instance when compiled:

    from lib import MyFloat  # MyFloat ia a subclass of "float"
    def example() -> None:
        x = MyFloat(1.5)
        y: float = x  # Implicit conversion from MyFloat to float
        print(type(y))  # float, not MyFloat

Previously, implicit conversions were applied to int subclasses but not float subclasses.

Also, int values can no longer be assigned to a variable with type float in compiled code, since these types now have incompatible representations. An explicit conversion is required:

    def example(n: int) -> None:
        a: float = 1  # Error: cannot assign "int" to "float"
        b: float = 1.0  # OK
        c: float = n  # Error
        d: float = float(n)  # OK

This restriction only applies to assignments, since they could otherwise narrow down the type of a variable from float to int. int values can still be implicitly converted to float when passed as arguments to functions that expect float values.

Note that mypyc still doesn’t support arrays of unboxed float values. Using list[float] involves heap-allocated float objects, since list can only store boxed values. Support for efficient floating point arrays is one of the next major planned mypyc features.

Related changes:

  • Use a native unboxed representation for floats (Jukka Lehtosalo, PR 14880)
  • Document native floats and integers (Jukka Lehtosalo, PR 14927)
  • Fixes to float to int conversion (Jukka Lehtosalo, PR 14936)

Mypyc: Native Integers

Mypyc now supports signed 32-bit and 64-bit integer types in addition to the arbitrary-precision int type. You can use the types mypy_extensions.i32 and mypy_extensions.i64 to speed up code that uses integer operations heavily.

Simple example:

    from mypy_extensions import i64
    def inc(x: i64) -> i64:
        return x + 1

Refer to the documentation for more information. This feature was contributed by Jukka Lehtosalo.

Other Mypyc Fixes and Improvements

  • Support iterating over a TypedDict (Richard Si, PR 14747)
  • Faster coercions between different tuple types (Jukka Lehtosalo, PR 14899)
  • Faster calls via type aliases (Jukka Lehtosalo, PR 14784)
  • Faster classmethod calls via cls (Jukka Lehtosalo, PR 14789)

Fixes to Crashes

  • Fix crash on class-level import in protocol definition (Ivan Levkivskyi, PR 14926)
  • Fix crash on single item union of alias (Ivan Levkivskyi, PR 14876)
  • Fix crash on ParamSpec in incremental mode (Ivan Levkivskyi, PR 14885)

Documentation Updates

  • Update adopting --strict documentation for 1.0 (Shantanu, PR 14865)
  • Some minor documentation tweaks (Jukka Lehtosalo, PR 14847)
  • Improve documentation of top level mypy: disable-error-code comment (Nikita Sobolev, PR 14810)

Error Reporting Improvements

  • Add error code to typing_extensions suggestion (Shantanu, PR 14881)
  • Add a separate error code for top-level await (Nikita Sobolev, PR 14801)
  • Don’t suggest two obsolete stub packages (Jelle Zijlstra, PR 14842)
  • Add suggestions for pandas-stubs and lxml-stubs (Shantanu, PR 14737)

Other Fixes and Improvements

  • Multiple inheritance considers callable objects as subtypes of functions (Christoph Tyralla, PR 14855)
  • stubtest: Respect @final runtime decorator and enforce it in stubs (Nikita Sobolev, PR 14922)
  • Fix false positives related to type[<type-var>] (sterliakov, PR 14756)
  • Fix duplication of ParamSpec prefixes and properly substitute ParamSpecs (EXPLOSION, PR 14677)
  • Fix line number if __iter__ is incorrectly reported as missing (Jukka Lehtosalo, PR 14893)
  • Fix incompatible overrides of overloaded generics with self types (Shantanu, PR 14882)
  • Allow SupportsIndex in slice expressions (Shantanu, PR 14738)
  • Support if statements in bodies of dataclasses and classes that use dataclass_transform (Jacek Chałupka, PR 14854)
  • Allow iterable class objects to be unpacked (including enums) (Alex Waygood, PR 14827)
  • Fix narrowing for walrus expressions used in match statements (Shantanu, PR 14844)
  • Add signature for attr.evolve (Ilya Konstantinov, PR 14526)
  • Fix Any inference when unpacking iterators that don't directly inherit from typing.Iterator (Alex Waygood, PR 14821)
  • Fix unpack with overloaded __iter__ method (Nikita Sobolev, PR 14817)
  • Reduce size of JSON data in mypy cache (dosisod, PR 14808)
  • Improve “used before definition” checks when a local definition has the same name as a global definition (Stas Ilinskiy, PR 14517)
  • Honor NoReturn as __setitem__ return type to mark unreachable code (sterliakov, PR 12572)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Avasam
  • Christoph Tyralla
  • dosisod
  • EXPLOSION
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jacek Chałupka
  • Jelle Zijlstra
  • Jukka Lehtosalo
  • Marc Mueller
  • Max Murin
  • Nikita Sobolev
  • Richard Si
  • Shantanu
  • Stas Ilinskiy
  • sterliakov
  • Wesley Collin Wright

Monday, 6 March 2023

Mypy 1.1.1 Released

We’ve just uploaded mypy 1.1.1 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:
    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Support for dataclass_transform

This release adds full support for the dataclass_transform decorator defined in PEP 681. This allows decorators, base classes, and metaclasses that generate a __init__ method or other methods based on the properties of that class (similar to dataclasses) to have those methods recognized by mypy.

This was contributed by Wesley Collin Wright.

Dedicated Error Code for Method Assignments

Mypy can’t safely check all assignments to methods (a form of monkey patching), so mypy generates an error by default. To make it easier to ignore this error, mypy now uses the new error code method-assign for this. By disabling this error code in a file or globally, mypy will no longer complain about assignments to methods if the signatures are compatible.

Mypy also supports the old error code assignment for these assignments to prevent a backward compatibility break. More generally, we can use this mechanism in the future if we wish to split or rename another existing error code without causing backward compatibility issues.

This was contributed by Ivan Levkivskyi (PR 14570).

Fixes to Crashes

  • Fix a crash on walrus in comprehension at class scope (Ivan Levkivskyi, PR 14556)
  • Fix crash related to value-constrained TypeVar (Shantanu, PR 14642)

Fixes to Cache Corruption

  • Fix generic TypedDict/NamedTuple caching (Ivan Levkivskyi, PR 14675)

Mypyc Fixes and Improvements

  • Raise "non-trait base must be first..." error less frequently (Richard Si, PR 14468)
  • Generate faster code for bool comparisons and arithmetic (Jukka Lehtosalo, PR 14489)
  • Optimize __(a)enter__/__(a)exit__ for native classes (Jared Hance, PR 14530)
  • Detect if attribute definition conflicts with base class/trait (Jukka Lehtosalo, PR 14535)
  • Support __(r)divmod__ dunders (Richard Si, PR 14613)
  • Support __pow__, __rpow__, and __ipow__ dunders (Richard Si, PR 14616)
  • Fix crash on star unpacking to underscore (Ivan Levkivskyi, PR 14624)
  • Fix iterating over a union of dicts (Richard Si, PR 14713)

Fixes to Detecting Undefined Names (used-before-def)

  • Correctly handle walrus operator (Stas Ilinskiy, PR 14646)
  • Handle walrus declaration in match subject correctly (Stas Ilinskiy, PR 14665)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Allow aliases below the top level (Chad Dombrova, PR 14388)
  • Fix crash with PEP 604 union in type variable bound (Shantanu, PR 14557)
  • Preserve PEP 604 unions in generated .pyi files (hamdanal, PR 14601)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Update message format so that it’s easier to go to error location (Avasam, PR 14437)
  • Handle name-mangling edge cases better (Alex Waygood, PR 14596)

Changes to Error Reporting and Messages

  • Add new TypedDict error code typeddict-unknown-key (JoaquimEsteves, PR 14225)
  • Give arguments a more reasonable location in error messages (Max Murin, PR 14562)
  • In error messages, quote just the module's name (Ilya Konstantinov, PR 14567)
  • Improve misleading message about Enum() (Rodrigo Silva, PR 14590)
  • Suggest importing from typing_extensions if definition is not in typing (Shantanu, PR 14591)
  • Consistently use type-abstract error code (Ivan Levkivskyi, PR 14619)
  • Consistently use literal-required error code for TypedDicts (Ivan Levkivskyi, PR 14621)
  • Adjust inconsistent dataclasses plugin error messages (Wesley Collin Wright, PR 14637)
  • Consolidate literal bool argument error messages (Wesley Collin Wright, PR 14693)

Other Fixes and Improvements

  • Check that type guards accept a positional argument (EXPLOSION, PR 14238)
  • Fix bug with in operator used with a union of Container and Iterable (Max Murin, PR 14384)
  • Support protocol inference for type[T] via metaclass (Ivan Levkivskyi, PR 14554)
  • Allow overlapping comparisons between bytes-like types (Shantanu, PR 14658)
  • Fix mypy daemon documentation link in README (Ivan Levkivskyi, PR 14644)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Avasam
  • Chad Dombrova
  • dosisod
  • EXPLOSION
  • hamdanal
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jared Hance
  • JoaquimEsteves
  • Jukka Lehtosalo
  • Marc Mueller
  • Max Murin
  • Michael Lee
  • Michael R. Crusoe
  • Richard Si
  • Rodrigo Silva
  • Shantanu
  • Stas Ilinskiy
  • Wesley Collin Wright
  • Yilei "Dolee" Yang
  • Yurii Karabas

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Monday, 6 February 2023

Mypy 1.0 Released

We’ve just uploaded mypy 1.0 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

New Release Versioning Scheme

Now that mypy reached 1.0, we’ll switch to a new versioning scheme. Mypy version numbers will be of form x.y.z.

Rules:

  • The major release number (x) is incremented if a feature release includes a significant backward incompatible change that affects a significant fraction of users.
  • The minor release number (y) is incremented on each feature release. Minor releases include updated stdlib stubs from typeshed.
  • The point release number (z) is incremented when there are fixes only.

Mypy doesn't use SemVer, since most minor releases have at least minor backward incompatible changes in typeshed, at the very least. Also, many type checking features find new legitimate issues in code. These are not considered backward incompatible changes, unless the number of new errors is very high.

Any significant backward incompatible change must be announced in the blog post for the previous feature release, before making the change. The previous release must also provide a flag to explicitly enable or disable the new behavior (whenever practical), so that users will be able to prepare for the changes and report issues. We should keep the feature flag for at least a few releases after we've switched the default.

See ”Release Process” in the mypy wiki for more details and for the most up-to-date version of the versioning scheme.

Performance Improvements

Mypy 1.0 is up to 40% faster than mypy 0.991 when type checking the Dropbox internal codebase. We also set up a daily job to measure the performance of the most recent development version of mypy to make it easier to track changes in performance.

Many optimizations contributed to this improvement:

  • Improve performance for errors on class with many attributes (Shantanu, PR 14379)
  • Speed up make_simplified_union (Jukka Lehtosalo, PR 14370)
  • Micro-optimize get_proper_type(s) (Jukka Lehtosalo, PR 14369)
  • Micro-optimize flatten_nested_unions (Jukka Lehtosalo, PR 14368)
  • Some semantic analyzer micro-optimizations (Jukka Lehtosalo, PR 14367)
  • A few miscellaneous micro-optimizations (Jukka Lehtosalo, PR 14366)
  • Optimization: Avoid a few uses of contextmanagers in semantic analyzer (Jukka Lehtosalo, PR 14360)
  • Optimization: Enable always defined attributes in Type subclasses (Jukka Lehtosalo, PR 14356)
  • Optimization: Remove expensive context manager in type analyzer (Jukka Lehtosalo, PR 14357)
  • subtypes: fast path for Union/Union subtype check (Hugues, PR 14277)
  • Micro-optimization: avoid Bogus[int] types that cause needless boxing (Jukka Lehtosalo, PR 14354)
  • Avoid slow error message logic if errors not shown to user (Jukka Lehtosalo, PR 14336)
  • Speed up the implementation of hasattr() checks (Jukka Lehtosalo, PR 14333)
  • Avoid the use of a context manager in hot code path (Jukka Lehtosalo, PR 14331)
  • Change various type queries into faster bool type queries (Jukka Lehtosalo, PR 14330)
  • Speed up recursive type check (Jukka Lehtosalo, PR 14326)
  • Optimize subtype checking by avoiding a nested function (Jukka Lehtosalo, PR 14325)
  • Optimize type parameter checks in subtype checking (Jukka Lehtosalo, PR 14324)
  • Speed up freshening type variables (Jukka Lehtosalo, PR 14323)
  • Optimize implementation of TypedDict types for **kwds (Jukka Lehtosalo, PR 14316)

Warn About Variables Used Before Definition

Mypy will now generate an error if you use a variable before it’s defined. This feature is enabled by default. By default mypy reports an error when it infers that a variable is always undefined.

    y = x  # E: Name "x" is used before definition [used-before-def]
    x = 0

This feature was contributed by Stas Ilinskiy.

Detect Possibly Undefined Variables (Experimental)

A new experimental possibly-undefined error code is now available that will detect variables that may be undefined:

    if b:
        x = 0
    print(x)  # Error: Name "x" may be undefined [possibly-undefined]

The error code is disabled be default, since it can generate false positives.

This feature was contributed by Stas Ilinskiy.

Support the “Self” Type

There is now a simpler syntax for declaring generic self types introduced in PEP 673: the Self type. You no longer have to define a type variable to use “self types”, and you can use them with attributes. Example from mypy documentation:

    from typing import Self
    
    class Friend:
        other: Self | None = None
    
        @classmethod
        def make_pair(cls) -> tuple[Self, Self]:
            a, b = cls(), cls()
            a.other = b
            b.other = a
            return a, b
    
    class SuperFriend(Friend):
        pass
    
    # a and b have the inferred type "SuperFriend", not "Friend"
    a, b = SuperFriend.make_pair()

The feature was introduced in Python 3.11. In earlier Python versions a backport of Self is available in typing_extensions.

This was contributed by Ivan Levkivskyi (PR 14041).

Support ParamSpec in Type Aliases

ParamSpec and Concatenate can now be used in type aliases. Example:
    from typing import ParamSpec, Callable
    
    P = ParamSpec("P")
    A = Callable[P, None]
    
    def f(c: A[int, str]) -> None:
        c(1, "x")

This feature was contributed by Ivan Levkivskyi (PR 14159).

ParamSpec and Generic Self Types No Longer Experimental

Support for ParamSpec (PEP 612) and generic self types are no longer considered experimental.

Miscellaneous New Features

  • Minimal, partial implementation of dataclass_transform (PEP 681) (Wesley Collin Wright, PR 14523)
  • Add basic support for typing_extensions.TypeVar (Marc Mueller, PR 14313)
  • Add --debug-serialize option (Marc Mueller, PR 14155)
  • Constant fold initializers of final variables (Jukka Lehtosalo, PR 14283)
  • Enable Final instance attributes for attrs (Tin Tvrtković, PR 14232)
  • Allow function arguments as base classes (Ivan Levkivskyi, PR 14135)
  • Allow super() with mixin protocols (Ivan Levkivskyi, PR 14082)
  • Add type inference for dict.keys membership (Matthew Hughes, PR 13372)
  • Generate error for class attribute access if attribute is defined with __slots__ (Harrison McCarty, PR 14125)
  • Support additional attributes in callback protocols (Ivan Levkivskyi, PR 14084)

Fixes to Crashes

  • Fix crash on prefixed ParamSpec with forward reference (Ivan Levkivskyi, PR 14569)
  • Fix internal crash when resolving the same partial type twice (Shantanu, PR 14552)
  • Fix crash in daemon mode on new import cycle (Ivan Levkivskyi, PR 14508)
  • Fix crash in mypy daemon (Ivan Levkivskyi, PR 14497)
  • Fix crash on Any metaclass in incremental mode (Ivan Levkivskyi, PR 14495)
  • Fix crash in await inside comprehension outside function (Ivan Levkivskyi, PR 14486)
  • Fix crash in Self type on forward reference in upper bound (Ivan Levkivskyi, PR 14206)
  • Fix a crash when incorrect super() is used outside a method (Ivan Levkivskyi, PR 14208)
  • Fix crash on overriding with frozen attrs (Ivan Levkivskyi, PR 14186)
  • Fix incremental mode crash on generic function appearing in nested position (Ivan Levkivskyi, PR 14148)
  • Fix daemon crash on malformed NamedTuple (Ivan Levkivskyi, PR 14119)
  • Fix crash during ParamSpec inference (Ivan Levkivskyi, PR 14118)
  • Fix crash on nested generic callable (Ivan Levkivskyi, PR 14093)
  • Fix crashes with unpacking SyntaxError (Shantanu, PR 11499)
  • Fix crash on partial type inference within a lambda (Ivan Levkivskyi, PR 14087)
  • Fix crash with enums (Michael Lee, PR 14021)
  • Fix crash with malformed TypedDicts and disllow-any-expr (Michael Lee, PR 13963)

Error Reporting Improvements

  • More helpful error for missing self (Shantanu, PR 14386)
  • Add error-code truthy-iterable (Marc Mueller, PR 13762)
  • Fix pluralization in error messages (KotlinIsland, PR 14411)

Mypyc: Support Match Statement

Mypyc can now compile Python 3.10 match statements.

This was contributed by dosisod (PR 13953).

Other Mypyc Fixes and Improvements

  • Optimize int(x)/float(x)/complex(x) on instances of native classes (Richard Si, PR 14450)
  • Always emit warnings (Richard Si, PR 14451)
  • Faster bool and integer conversions (Jukka Lehtosalo, PR 14422)
  • Support attributes that override properties (Jukka Lehtosalo, PR 14377)
  • Precompute set literals for "in" operations and iteration (Richard Si, PR 14409)
  • Don't load targets with forward references while setting up non-extension class __annotations__ (Richard Si, PR 14401)
  • Compile away NewType type calls (Richard Si, PR 14398)
  • Improve error message for multiple inheritance (Joshua Bronson, PR 14344)
  • Simplify union types (Jukka Lehtosalo, PR 14363)
  • Fixes to union simplification (Jukka Lehtosalo, PR 14364)
  • Fix for typeshed changes to Collection (Shantanu, PR 13994)
  • Allow use of enum.Enum (Shantanu, PR 13995)
  • Fix compiling on Arch Linux (dosisod, PR 13978)

Documentation Improvements

  • Various documentation and error message tweaks (Jukka Lehtosalo, PR 14574)
  • Improve Generics documentation (Shantanu, PR 14587)
  • Improve protocols documentation (Shantanu, PR 14577)
  • Improve dynamic typing documentation (Shantanu, PR 14576)
  • Improve the Common Issues page (Shantanu, PR 14581)
  • Add a top-level TypedDict page (Shantanu, PR 14584)
  • More improvements to getting started documentation (Shantanu, PR 14572)
  • Move truthy-function documentation from “optional checks” to “enabled by default” (Anders Kaseorg, PR 14380)
  • Avoid use of implicit optional in decorator factory documentation (Tom Schraitle, PR 14156)
  • Clarify documentation surrounding install-types (Shantanu, PR 14003)
  • Improve searchability for module level type ignore errors (Shantanu, PR 14342)
  • Advertise mypy daemon in README (Ivan Levkivskyi, PR 14248)
  • Add link to error codes in README (Ivan Levkivskyi, PR 14249)
  • Document that report generation disables cache (Ilya Konstantinov, PR 14402)
  • Stop saying mypy is beta software (Ivan Levkivskyi, PR 14251)
  • Flycheck-mypy is deprecated, since its functionality was merged to Flycheck (Ivan Levkivskyi, PR 14247)
  • Update code example in "Declaring decorators" (ChristianWitzler, PR 14131)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Improve error message for __all__-related errors (Alex Waygood, PR 14362)
  • Improve heuristics for determining whether global-namespace names are imported (Alex Waygood, PR 14270)
  • Catch BaseException on module imports (Shantanu, PR 14284)
  • Associate exported symbol error with __all__ object_path (Nikita Sobolev, PR 14217)
  • Add __warningregistry__ to the list of ignored module dunders (Nikita Sobolev, PR 14218)
  • If a default is present in the stub, check that it is correct (Jelle Zijlstra, PR 14085)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Treat dlls as C modules (Shantanu, PR 14503)

Other Notable Fixes and Improvements

  • Update stub suggestions based on recent typeshed changes (Alex Waygood, PR 14265)
  • Fix attrs protocol check with cache (Marc Mueller, PR 14558)
  • Fix strict equality check if operand item type has custom __eq__ (Jukka Lehtosalo, PR 14513)
  • Don't consider object always truthy (Jukka Lehtosalo, PR 14510)
  • Properly support union of TypedDicts as dict literal context (Ivan Levkivskyi, PR 14505)
  • Properly expand type in generic class with Self and TypeVar with values (Ivan Levkivskyi, PR 14491)
  • Fix recursive TypedDicts/NamedTuples defined with call syntax (Ivan Levkivskyi, PR 14488)
  • Fix type inference issue when a class inherits from Any (Shantanu, PR 14404)
  • Fix false positive on generic base class with six (Ivan Levkivskyi, PR 14478)
  • Don't read scripts without extensions as modules in namespace mode (Tim Geypens, PR 14335)
  • Fix inference for constrained type variables within unions (Christoph Tyralla, PR 14396)
  • Fix Unpack imported from typing (Marc Mueller, PR 14378)
  • Allow trailing commas in ini configuration of multiline values (Nikita Sobolev, PR 14240)
  • Fix false negatives involving Unions and generators or coroutines (Shantanu, PR 14224)
  • Fix ParamSpec constraint for types as callable (Vincent Vanlaer, PR 14153)
  • Fix type aliases with fixed-length tuples (Jukka Lehtosalo, PR 14184)
  • Fix issues with type aliases and new style unions (Jukka Lehtosalo, PR 14181)
  • Simplify unions less aggressively (Ivan Levkivskyi, PR 14178)
  • Simplify callable overlap logic (Ivan Levkivskyi, PR 14174)
  • Try empty context when assigning to union typed variables (Ivan Levkivskyi, PR 14151)
  • Improvements to recursive types (Ivan Levkivskyi, PR 14147)
  • Make non-numeric non-empty FORCE_COLOR truthy (Shantanu, PR 14140)
  • Fix to recursive type aliases (Ivan Levkivskyi, PR 14136)
  • Correctly handle Enum name on Python 3.11 (Ivan Levkivskyi, PR 14133)
  • Fix class objects falling back to metaclass for callback protocol (Ivan Levkivskyi, PR 14121)
  • Correctly support self types in callable ClassVar (Ivan Levkivskyi, PR 14115)
  • Fix type variable clash in nested positions and in attributes (Ivan Levkivskyi, PR 14095)
  • Allow class variable as implementation for read only attribute (Ivan Levkivskyi, PR 14081)
  • Prevent warnings from causing dmypy to fail (Andrzej Bartosiński, PR 14102)
  • Correctly process nested definitions in mypy daemon (Ivan Levkivskyi, PR 14104)
  • Don't consider a branch unreachable if there is a possible promotion (Ivan Levkivskyi, PR 14077)
  • Fix incompatible overrides of overloaded methods in concrete subclasses (Shantanu, PR 14017)
  • Fix new style union syntax in type aliases (Jukka Lehtosalo, PR 14008)
  • Fix and optimise overload compatibility checking (Shantanu, PR 14018)
  • Improve handling of redefinitions through imports (Shantanu, PR 13969)
  • Preserve (some) implicitly exported types (Shantanu, PR 13967)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alessio Izzo
  • Alex Waygood
  • Anders Kaseorg
  • Andrzej Bartosiński
  • Avasam
  • ChristianWitzler
  • Christoph Tyralla
  • dosisod
  • Harrison McCarty
  • Hugo van Kemenade
  • Hugues
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • jhance
  • johnthagen
  • Jonathan Daniel
  • Joshua Bronson
  • Jukka Lehtosalo
  • KotlinIsland
  • Lakshay Bisht
  • Lefteris Karapetsas
  • Marc Mueller
  • Matthew Hughes
  • Michael Lee
  • Nick Drozd
  • Nikita Sobolev
  • Richard Si
  • Shantanu
  • Stas Ilinskiy
  • Tim Geypens
  • Tin Tvrtković
  • Tom Schraitle
  • Valentin Stanciu
  • Vincent Vanlaer

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Tuesday, 8 November 2022

Mypy 0.991 Released

We’ve just uploaded mypy 0.991 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Basic Python 3.11 Support

Mypy and mypyc now have partial official support for Python 3.11. We provide compiled wheels for 3.11, and many 3.11 stdlib features are supported.

Most new language and typing features are still unsupported or are experimental. Adding support for the remaining 3.11 features is a priority for the mypy team.

Breaking Change: No Implicit Optional Types for Arguments

Mypy now disables implicit optional types such as these by default to fix a long-standing incompatibility with PEP 484, and they will have to be explicitly enabled:

    def foo(s: str = None) -> int: ...  # Error!

Mypy will now only accept the example with an explicit optional type (e.g. str | None) for the s argument, when using the default options:

    # s: Optional[str] also works
    def foo(s: str | None = None) -> int: ...  # OK

You can still explicitly enable implicit optional types via --implicit-optional or no_implicit_optional = False in your mypy config file. We don’t recommend relying on implicit optional types, since they are a non-standard feature, but we have no plans to stop supporting them.

You can also use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase to explicit optional types.

This was contributed by Shantanu:

  • Use --no-implicit-optional by default (Shantanu, PR 13401)
  • Suggest codemod for --no-implicit-optional (Shantanu, PR 13747)

Breaking Change: Namespace Packages Enabled by Default

Mypy now enables --namespace-packages by default (PEP 420), so that namespace packages (packages with a missing __init__.py file) are supported without any extra configuration.

Refer to the documentation of --namespace-packages and to Mapping file paths to modules in mypy documentation for more information.

In some cases you may need to use --explicit-package-bases and to explicitly declare additional package roots that mypy can’t figure out automatically by using the MYPYPATH environment variable or using mypy_path = <dirs> in your config file. Otherwise you may get errors about missing imports.

You can explicitly disable the option by using --no-namespace-packages or namespace_packages = False in your config file to return to the old behavior. We recommend first trying --explicit-package-bases discussed above if you encounter any issues.

This was contributed by Shantanu (PR 9636).

Recursive Types Enabled By Default

Mypy support for general recursive types is now official. These were available in mypy 0.981 through the --enable-recursive-aliases flag. Now you no longer need to explicitly enable them. Example:

    # This is now allowed by default
    NaiveJSON = str | list["NaiveJSON"] | dict[str, "NaiveJSON"]
    test_data: NaiveJSON = {"foo": {"bar": "baz"}}  # OK

Note that you will need to use string escapes on the right hand side (e.g. "NaiveJSON") if you define recursive types outside of stub files, since Python doesn’t support forward references at runtime. Also, mypy generally doesn’t infer recursive types by default. Similar to TypedDict types, you will often need to explicitly annotate a variable with a recursive type or mypy will try to infer a non-recursive type instead.

You can still disable recursive types using --disable-recursive-aliases if they cause any problems. Please open an issue on the mypy issue tracker if this is the case.

See “Experimental Support for General Recursive Types” in the mypy 0.981 release blog post for more details.

This change was contributed by Ivan Levkivskyi in (PR 13516), (PR 13852) and (PR 13754).

Error Codes Shown by Default

Mypy now shows error codes at the end of each error message by default, within square brackets. Example:

    error: 
    Unsupported operand types for + ("int" and "str")  [operator]

This makes it easier to silence only specific error codes on a line by using # type: ignore[<error-code>]. Example:

    print(1 + "x")  # type: ignore[operator]

You can find helpful information about specific errors by looking up the error code in the documentation.

You can hide error codes by using --hide-error-codes or by adding show_error_codes = False to your configuration file.

This was contributed by Marc Mueller (PR 13542).

Safe Handling of Empty Function Bodies

For historical reasons, mypy used to always allow trivial function/method bodies (e.g. a body with only a single pass statement and/or a docstring), even if this is not type safe. Example:

    # Error: Missing return statement (no error before 0.990)
    def some_func(arg: str) -> int:
        pass

This could hide bugs and was a generally confusing behavior. Mypy is now handling such empty bodies safely. In the example above you will now get an error (unless you are using --no-strict-optional). Mypy only allows an empty/trivial body in a method with a non-optional return type if the method is abstract:

    class Base(ABC):
        @abstractmethod
        def must_implement_in_subclass(self, arg: str) -> int:
            pass  # Still OK

Additionally, mypy now flags unsafe super() calls to a method with an empty body that’s decorated with @abstractmethod. Mypy will now issue an error in this case.

To disable this functionality, use the --allow-empty-bodies flag.

This change was contributed by Ivan Levkivskyi in (PR 13729).

Enabling Experimental Features Individually

Experimental mypy features can now be enabled individually. With this change, using --enable-incomplete-features to enable all of them becomes deprecated and the recommended way forward is to enable experimental features individually with --enable-incomplete-features=FEATURE.

This change was contributed by Ivan Levkivskyi and Nikita Sobolev in (PR 13790).

Configuring Packages/Modules in the Config File

It is now possible to tell mypy to implicitly type check specific packages by using the options modules and packages in the configuration file, without having to explicitly include these on the command line. These options take a comma separated list of packages which will be checked by mypy if none are given on the command line.

Mypy will not recursively type check any submodules of the provided modules by modules and will recursively type check any submodules of the provided modules by packages.

These options may only be set in the global section. See also the documentation.

This feature was contributed by sameer-here (PR 12639).

Warn about Variable Annotations in Unchecked Functions

Mypy will no generate a note if you use a type annotation in a function without a type annotation (and you are not using --check-untyped-defs), since these annotations have no effect. Example:

    def foo():
        x: int = bar()
        print(x)

This is treated as a warning, so it doesn’t cause a mypy run to fail:

    $ mypy prog.py
    prog.py:2: note: By default the bodies of untyped functions 
        are not checked, consider using --check-untyped-defs  
        [annotation-unchecked]
    Success: no issues found in 1 source file

This was contributed by Ivan Levkivskyi (PR 13851).

Error Code for Using an Abstract Class as type[T]

Mypy disallows using an abstract type object (ABC) if type type[T] is expected. This would be unsafe, since mypy always allows calling type[T] values to instantiate objects, and ABCs can’t be instantiated.

Now this error has the dedicated error code type-abstract, and you can use --disable-error-code type-abstract to disable this check, since it can generate undesirable errors in some use cases. Example:

    import abc
    
    class A(abc.ABC):
        @abc.abstractmethod
        def m(self) -> None: pass
    
    class C(A):
        def m(self) -> None: print('m')
    
    def check(o: object t: type[A]) -> bool:
        return isinstance(o, t)
    
    # Error: Only concrete class can be given here [type-abstract]
    check(C(), A)
    check(C(), C)  # OK

This was contributed by Ivan Levkivskyi (PR 13785).

Performance Improvements

Mypy includes a potential fix for a performance regression edge case introduced in mypy 0.982:

  • Revert literal type change in builtins.sum (Shantanu, PR 13961)

Changes to Error Reporting and Messages

  • Return exit status 0 if there are only notes and no errors (Ivan Levkivskyi, PR 13879)
  • Generate an error if a function object is used in a boolean context, with the truthy-function error code, even if the truthy-bool error code is disabled (Marc Mueller, PR 13686)
  • Silence errors from third-party packages in mypy daemon (Jukka Lehtosalo, PR 13768)
  • Preserve file order of messages during successive daemon runs (Jukka Lehtosalo, PR 13780)
  • Suggest additional types-* packages from typeshed (Jukka Lehtosalo, PR 13698)
  • Better diagnostic for conditional function mismatch (Shantanu, PR 13604)
  • Use consistent capitalization for TypeVar (Marti Raudsepp, PR 13687)
  • Suggest using upper bound for unbound type variable (Shantanu, PR 13730)
  • Show error codes for some notes (Ivan Levkivskyi, PR 13880)
  • Suggest using a protocol if trying to use a module as a type (Nikita Sobolev, PR 13861)
  • Improve error message for --strict-concatenate (Shantanu, PR 13777)
  • Improve error message for implicitly abstract functions (Shantanu, PR 13776)
  • Mention implicit export on missing attribute access (Shantanu, PR 13917)
  • Always mention explicit export when relevant (Shantanu, PR 13925)
  • Fix error code when reporting an invalid Literal type (Shantanu, PR 13763)
  • Replace invalid __set__ with __get__ in messages (Ilya Konstantinov, PR 13913)

Mypyc Fixes and Improvements

  • Add support for building mypyc code on WASM (Ethan Smith and Shantanu, PR 13446)
  • Fix C errors about shifting negative integers (Jukka Lehtosalo, PR 13876)

Documentation Updates

Several parts of the documentation received major updates.

  • Major update to "Using mypy with existing codebase" (Shantanu, PR 13683)
  • Update cheat sheet (PR 13873, PR 13679) (Shantanu)
  • Update “Runtime troubles” (Shantanu, PR 13680)
  • Mention files config file option in "Specifying code to check" (Shantanu, PR 13682)
  • Make language more consistent (Shantanu, PR 13677)
  • Update “Getting started” (Shantanu, PR 13734)
  • Update “Type inference and annotations” (Shantanu, PR 13736)
  • Discuss user defined protocols before built-in protocols (Shantanu, PR 13737)
  • Reorder sections in “Running mypy” (Shantanu, PR 13738)
  • Improve "Precise typing of alternative constructors" example (Jelle Zijlstra)
  • Improve “Getting started” docs (Shantanu, PR 13875)
  • Update “Extending mypy” docs (Nikita Sobolev, PR 13924)
  • Add clear warning about stubtest code execution (Nikita Sobolev, PR 13927)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Introduce an object-oriented system for extracting function signatures (Chad Dombrova, PR 13473)
  • Add known return types to magic methods (Nikita Sobolev, PR 13909)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Detect abstract properties mismatches (Nikita Sobolev, PR 13647)
  • Catch SyntaxError from inspect.getsourcelines (Shantanu, PR 13848)

Other Notable Fixes and Improvements

  • Detect invalid ParamSpec annotations used with *args and **kwargs (Nikita Sobolev, PR 13892)
  • Fix crash with generic class definition in function (Michael Lee, PR 13678)
  • Fix joining a function against metaclass-using object constructors (Michael Lee, PR 13648)
  • Fixes to attrs magic attribute handling (Tin Tvrtković and Nikita Sobolev, PR 13522)
  • Support __attrs_init__ method in attrs classes (Spencer Brown, PR 13865)
  • Fix crash on match statement with value restricted TypeVar (Shantanu, PR 13728)
  • Fix crash in match statement with unmatchable class pattern (Shantanu)
  • Fix unsound variance (Shantanu, PR 13714)
  • Fix crash with report generation on namespace packages (Shantanu, PR 13733)
  • Remove use of LiteralString in builtins (Shantanu, PR 13743)
  • Treat __dataclass_fields__ as a ClassVar (Shantanu, PR 13721)
  • Respect per-module follow_import for empty folders (Shantanu, PR 13758)
  • Fix a crash related to dataclasses and type aliases (Ivan Levkivskyi, PR 13759)
  • Add install-types extra that depends on pip (David Runge, PR 13739)
  • Restore Type vs Callable special-casing that was broken in refactoring (Ivan Levkivskyi, PR 13784)
  • Fix module and protocol subtyping and hasattr with a module (Shantanu, PR 13778)
  • Make join of recursive types more robust (Ivan Levkivskyi, PR 13808)
  • Fix minor issues with **kwargs that use TypedDict unpacking (Ivan Levkivskyi, PR 13854)
  • Fix crash on missing indirect dependencies (Ivan Levkivskyi, PR 13847)
  • Ignore promotions when simplifying unions (Ivan Levkivskyi, PR 13781)
  • Fall back to FORCE_COLOR environment variable if MYPY_FORCE_COLOR is not present (Alex Waygood, PR 13814)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Bas van Beek
  • Ben Raz
  • Chad Dombrova
  • David Runge
  • Ethan Smith
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • iyanging
  • Jared Hance
  • Jelle Zijlstra
  • Jeroen Van Goey
  • Jinze Wu
  • Jukka Lehtosalo
  • Kevin Kirsche
  • KotlinIsland
  • Marc Mueller
  • Marti Raudsepp
  • Matt Wozniski
  • Michael Lee
  • Nikita Sobolev
  • pranavrajpal
  • Richard Si
  • Ryan Soklaski
  • sameer-here
  • Shantanu
  • Spencer Brown
  • Stas Ilinskiy
  • Tim D. Smith
  • Tin Tvrtković
  • Valentin Stanciu

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Monday, 26 September 2022

Mypy 0.981 Released

 

We’ve just uploaded mypy 0.981 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:
    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Support for Python 3.6 and 2 Dropped

Support for Python 2 has been completely removed in this release.

Support for Python 3.6 has also been dropped, since Python 3.6 has reached its end of life. It is still possible to target Python 3.6 using --python-version in future mypy versions, but no bugs will be fixed that affect only Python 3.6 (unless they are regressions). Also note that typeshed just recently dropped Python 3.6 support, so standard library features only available in Python 3.6 will not be supported.

The implementation was jointly contributed by Shantanu, Nikita Sobolev and Ivan Levkivskyi.

Generate Error on Unbound TypeVar Return Type

Code like this will now generate an error, since the value of T cannot be inferred in calls:

    from typing import TypeVar
    
    T = TypeVar("T")
    
    # Error: A function returning TypeVar should receive at least one 
    # argument containing the same Typevar
    def f() -> T: ... 

Having an unbound type variable in a nested position (e.g. list[T]) is still accepted in a return type, since it has valid use cases. This was contributed by Aurélien Jaquier (PR 13166).

Methods with Empty Bodies in Protocols Are Abstract

Any method with an empty body in a protocol is now correctly treated as implicitly abstract and must be implemented even if a class explicitly inherits the protocol:

    from typing import Protocol
    
    class P(Protocol):
        # "..." is the literal ellipsis and indicates an empty body
        def meth(self) -> int: ...
    
    class Cls(P):
        pass
    
    # Error: Cannot instantiate abstract class "Cls" with abstract 
    # attribute "method"
    Cls()

This was contributed by Thomas MK (PR 12118).

Implicit Optional Types Will Be Disabled by Default

A future mypy feature release (possibly the next one after 0.98x) will disable implicit optional types such as these by default, and they will have to be explicitly enabled:

    def foo(s: str = None) -> int: ...  No error currently
    
    foo(None)  # Ok

In the future mypy will only accept the example with an explicit optional type (e.g. str | None) for the s argument, when using the default options:

    # s: Optional[str] also works
    def foo(s: str | None = None) -> int: ...
    
    foo(None)  # No change to callers

To prepare for the change, you can set the relevant configuration option explicitly. Either disable implicit optional types by using --no-implicit-optional, or no_implicit_optional = True in your mypy configuration file, or enable them via --implicit-optional or no_implicit_optional = False. We don’t recommend relying on implicit optional types, since they are a non-standard feature, but we have no plans to stop supporting them.

We hope to provide a migration tool that will automatically switch implicit optional types to explicit ones in function signatures.

Precise Types for **kwds Using TypedDict

Mypy now supports precise type annotations for kwds parameters. You can use this feature as following:

    from typing import TypedDict
    from typing_extensions import Unpack
    
    class Style(TypedDict, total=False):
        margin: int
        sticky: bool
    
    def add_button(label: str, **kwds: Unpack[Style]) -> None:
        ...
    def add_check_box(active: bool, **kwds: Unpack[Style]) -> None:
        ...
    
    add_button("Cancel", margin=0, sticky=False)  # This works
    tight: Style = {"margin": 0}
    add_button("OK", **tight)  # This works as well

You can alternatively use Required[...] and NotRequired[...] TypedDict annotations to control whether a keyword argument is required or not. Note that arbitrary extra keywords are not allowed for such functions (this is consistent with how mypy currently handles TypedDicts). Although this feature is complete, Unpack[...] is still considered experimental, so you will need to use --enable-incomplete-features to enable it.

This feature was contributed by Ivan Levkivskyi (PR 13471).

Experimental Support for General Recursive Types

Until recently mypy only supported recursively defined classes (e.g. trees), but not arbitrary recursive types. In this version we add (experimental) support for arbitrary recursively defined types, including type aliases, TypedDicts, NamedTuples, and so on. This example uses a recursively defined type alias:

    from typing import TypeVar, Sequence
    
    T = TypeVar("T")
    
    # String literal escaping needed to avoid NameError at runtime
    Nested = Sequence[T | "Nested[T]"]
    
    def flatten(seq: Nested[T]) -> list[T]:
        result = []
        for item in seq:
            if isinstance(item, list):
                result.extend(flatten(item))
            else:
                result.append(item)
        return result
    
    flatten([1, [2, 3]])  # This works, inferred type is list[int]

This feature can be enabled using --enable-recursive-aliases. If mypy doesn’t work as expected with some recursive types, you can try adding more explicit annotations, and/or using covariant collections (e.g. Sequence instead of list). Please file an issue on the issue tracker if you think you have found a bug.

List of relevant PRs:

  • Enable recursive type aliases behind a flag (Ivan Levkivskyi, PR 13297)
  • Simplify types in some error messages (Ivan Levkivskyi, PR 13326)
  • Support recursive TypedDicts (Ivan Levkivskyi, PR 13373)
  • Support recursive named tuples (Ivan Levkivskyi, PR 13371)
  • Fail gracefully on diverging recursive type aliases (Ivan Levkivskyi, PR 13352)
  • Fail gracefully on invalid and/or unsupported recursive type aliases (Ivan Levkivskyi, PR 13336)
  • Handle interactions between recursive aliases and recursive instances (Ivan Levkivskyi, PR 13328)
  • Fix crash on deferred upper bound with recursive aliases enabled (Ivan Levkivskyi, PR 13410)
  • Remove Optional[...] special-casing to avoid infinite recursion (Ivan Levkivskyi, PR 13357)

Generic NamedTuples and TypedDicts

Mypy now supports defining generic TypedDicts, NamedTuples, and user defined tuple types. You can use either the class or call syntax for this:

    from typing import Generic, TypeVar
    from typing_extensions import TypedDict  # From "typing" on Python 3.11
    
    T = TypeVar("T")
    
    class Param(TypedDict, Generic[T]):
        name: str
        value: T
    
    Opt = TypedDict("Opt", {"value": T, "is_set": bool})
    
    def apply(param: Param[T]) -> T: ...
    
    x = apply({"name": "test", "value": 42})  # Inferred type is "int"
Note: While you can freely use this feature in stub files, if you want to use it in source files, you will need to either use Python 3.11 (which is not officially supported by mypy yet), import TypedDict/NamedTuple from typing_extensions, or use an if TYPE_CHECKING: ... block.

List of relevant PRs:

  • Enable generic TypedDicts (Ivan Levkivskyi, PR 13389)
  • Enable generic NamedTuples (Ivan Levkivskyi, PR 13396)

Better Support for Callable Attributes

Mypy now has better support for instance attributes with Callable[...] types. Previously, mypy couldn’t reliably distinguish between callable instance attributes and method aliases. Now there are consistent rules. If a variable is defined using and assignment and/or annotation in a class body and has a callable type, it is an instance attribute if both these conditions are true:

  • There is an explicit annotation, and it is not a ClassVar[...].
  • The variable name is not a special dunder method name, such as __add__.

Conversely, such an assignment/annotation defines a method in these cases:

  • There is no explicit annotation, or there is a ClassVar[...] annotation.
  • The variable name is a special dunder method name.

Example:

    from typing import ClassVar, Callable
    
    class Test:
        runner: Callable[[], None]  # An instance variable
    
        def hook(self) -> None: ...
            self.runner = lambda: print("pass")
    
        legacy_hook = hook  # A method
        other_hook: ClassVar[Callable[[Test], None]]  # Also a method
    
    Test.other_hook = lambda self: None
    t = Test()
    t.legacy_hook()
    t.runner()
    t.other_hook()

This feature was contribute by Ivan Levkivskyi (PR 13400).

Per-Module Error Code Configuration

You can now disable and/or enable individual error codes per module and per directory using config files and/or inline # mypy: comments. Example:

    # mypy.ini
    [mypy]
    strict = True
    
    [mypy-tests.*]
    allow_untyped_defs = True
    allow_untyped_calls = True
    disable_error_code = var-annotated
    # source.py
    x = []  # Error: Need type annotation for "x"
    # tests/foo.py
    x = []  # OK
    # tests/bar.py
    # mypy: enable-error-code="var-annotated"
    x = []  # Error: Need type annotation for "x"

Note that these per-directory settings act as incremental overrides. The precedence is the following:

  • Command line options and/or the main section of a config file set global codes.
  • Config sections adjust them per glob/module.
  • Inline # mypy: comments adjust them again.

You can, for example, enable an error code globally, disable it for all tests in the config file, and then re-enable it in a particular file by using an inline comment.

This feature was contributed by Ivan Levkivskyi (PR 13502).

Experimental Support for Interactive Inspection of Expressions

Mypy now has an experimental support for various static inspections using the mypy daemon dmypy inspect command. These inspections are currently included:

  • Print the type of an expression at given location.
  • Print the attributes defined for a given expression.
  • Go to definition of a reference expression (name or attribute).

Note that you can run this only after the file has been type-checked by the daemon, for example by using dmypy check or dmypy run. Basic usage is like this:

    dmypy inspect [--show type|attrs|definition] \
        path/to/file.py:line:column[:end_line:end_column]

Please read the docs for more details. This feature was contributed by Ivan Levkivskyi (PR 13209).

Mypyc Improvements

Mypyc compiles Python modules to C extensions and is bundled with mypy. It uses standard Python type hints to generate fast code (documentation).

  • Support async for as a statement and in comprehensions (Michael J. Sullivan, PR 13444)
  • Support async with (Michael J. Sullivan, PR 13442)
  • Fix clang warning on different signs integer (Paul m. p. Peny, PR 13239)
  • Fix AttributeError message (Jukka Lehtosalo, PR 13382)
  • Fix __call__ subclasses (Ken Jin, PR 13152)
  • Fix setup conflict with attributes named up (davfsa, PR 13012)
  • Fix bad C generated for multiple assignment (Jukka Lehtosalo, PR 13147)
  • Update, simplify check version test (Shantanu, PR 13125)

Documentation Updates

  • Update theme to furo (Michael Wentz, PR 13345)
  • Use ParamSpec in "Declaring decorators" (Ilya Konstantinov, PR 13237)
  • Fix protocols documentation (Zsolt Cserna, PR 13137)
  • Prefer f-strings over format and underscores in numeric literals in docs (Kevin Kirsche, PR 13317)
  • Use GitHub's zipball as alternative to 'git+https' in README.md (Peter Badida, PR 13208)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Allow passing arguments to stubgen.main (Matthew Woehlke, PR 13095)
  • Increase import timeout (Matthew Woehlke, PR 13109)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Fallback to getattr_static on AttributeError (Alex Waygood, PR 13285)
  • Reduce false positives on runtime type aliases (Alex Waygood, PR 13116)
  • Catch-all for errors from the runtime (Shantanu, PR 13160)
  • Allow stubtest to ignore Python 2 only magic methods (Nikita Sobolev, PR 13290)
  • Improve docs (Shantanu, PR 13293)
  • Ignore __pyx_vtable__ (Shantanu, PR 13302)
  • Improve signature checking (Shantanu, PR 13307)
  • Show path to stub file (Shantanu, PR 13342)
  • Allow stubtest to raise errors on abstract state mismatch (Nikita Sobolev, PR 13323)
  • Fix broken error message for async/sync mismatch (Alex Waygood, PR 13414)
  • Don't error for a missing submodule if the submodule name is private (Alex Waygood, PR 13417)
  • Ignore __vectorcalloffset__ (Alex Waygood, PR 13416)
  • Use single line error message (Shantanu, PR 13301)
  • Verify the contents of __all__ in a stub (Alex Waygood, PR 12214)
  • Fix error for Protocol.__init__ and __annotations__ (Alex Waygood, PR 13179)
  • Fix custom_typeshed_dir regression (PR 13658, PR 13656) (Shantanu)

Miscellaneous New Features

  • Enable support for decorated properties where possible (Ivan Levkivskyi, PR 13409)
  • Make any callable compatible with (*args: Any, kwargs: Any) (Shantanu, PR 11203)
  • Allow generic decorators on abstract classes (Ivan Levkivskyi, PR 13398)
  • Print and show error end locations (Ivan Levkivskyi, PR 13148)
  • Basic support for typing_extensions.NamedTuple (Alex Waygood, PR 13178)
  • Union types: Support narrowing to Ellipsis (...) cases of Unions (László Vaskó, PR 13157)

Other Notable Fixes and Improvements

  • Allow unpacking from TypeVars with iterable bounds (PR 13644, PR 13425) (Shantanu)
  • Fix error codes option serialization (Ivan Levkivskyi, PR 13523)
  • Fix daemon crashes related to ParamSpec and TypeVarTuple (Jukka Lehtosalo, PR 13381)
  • Allow stubs to use newer syntax than 3.7 (Shantanu, PR 13500)
  • Fix --custom-typeshed-dir handling (Ivan Levkivskyi, PR 13629)
  • Remove blocking special cased error for bool subclass (wookie184, PR 13420)
  • Remove useless parameter script_path of mypy.main() and make arguments keyword-only (Jingchen Ye, PR 13399)
  • Fix type inference for tuples in iterable context (Ivan Levkivskyi, PR 13406)
  • Fix type narrowing of TypedDict value with key name in final variable (Jingchen Ye, PR 11813)
  • Fix crash from invalid location in aliased types (Michael Krasnyk, PR 12745)
  • Advertise typing status via Trove classifier (Mike Fiedler, PR 13350)
  • Use a better colour for gray in mypy output (Ivan Levkivskyi, PR 13338)
  • Support type aliases in metaclasses (Nikita Sobolev, PR 13335)
  • Handle files ending with __init__ better (Shantanu, PR 13314)
  • Respect tuple.__getitem__ from typeshed (Shantanu, PR 13313)
  • Make subtype checks more consistent in edge cases (Ivan Levkivskyi, PR 13303)
  • Fix --custom-typeshed-dir crash (Alex Waygood, PR 13296)
  • Ensure builtin modules are from typeshed sooner (Konstantin, PR 13155)
  • Disallow bytes in TypeVar, NewType, and TypedDict names (Jelle Zijlstra, PR 13273)
  • Prohibit bytes literals in named tuples (Ivan Levkivskyi, PR 13271)
  • Suggest using a newer Python version if possibly needed (Shantanu, PR 13197)
  • Check implicit None return is valid when using --no-warn-no-return (Shantanu, PR 13219)
  • Fix site package on MYPYPATH check (Shantanu, PR 13223)
  • Make None compatible with SupportsStr protocol (Nikita Sobolev, PR 13184)
  • Fix caching of PEP 561 namespace packages (Shantanu, PR 13124)
  • Improve error message for direct __init__ calls (Shantanu, PR 13183)
  • Allow nonlocal in function scoped classes (Shantanu, PR 13158)
  • Support for Python 3.11's "safe_path" in pyinfo (Shantanu, PR 13164)
  • Generate error when using NoneType instead of None as a type (Jakub Strnad, PR 13153)
  • Add musllinux wheel support, Use 'official' llvm build for compilation (Ben Raz, PR 13228)
  • Skip musllinux wheels, update for lxml changes (Shantanu, PR 13129)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • anilbey
  • Aurélien Jaquier
  • Ben Raz
  • davfsa
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jakub Strnad
  • Jared Hance
  • Jelle Zijlstra
  • Jingchen Ye
  • Jukka Lehtosalo
  • Ken Jin
  • Kevin Kirsche
  • Konstantin
  • KotlinIsland
  • László Vaskó
  • Marc Mueller
  • Matthew Woehlke
  • Michael Krasnyk
  • Michael Sullivan
  • Michael Wentz
  • Mike Fiedler
  • Nikita Sobolev
  • Patryk Gałczyński
  • Paul m. p. Peny
  • Peter Badida
  • Richard Si
  • Shantanu
  • Thomas MK
  • wookie184
  • Zsolt Cserna
  • Zsolt Dollenstein

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Tuesday, 19 July 2022

Mypy 0.971 Released

We’ve just uploaded mypy 0.971 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Last Release Officially Supporting Python 2 and 3.6

Support for Python 2 will be completely removed in the next mypy feature release after this one, mypy 0.980.

Support for Python 3.6 will also be dropped in mypy 0.980, since Python 3.6 has reached its end of life. It will be still possible to target Python 3.6 using --python-version in future mypy versions, but no bugs will be fixed that affect only Python 3.6 (unless they are regressions). Also note that typeshed just recently dropped Python 3.6 support, so standard library features only available in Python 3.6 will not be supported.

Mypyc Improvements and Fixes

Mypyc compiles Python modules to C extensions and is bundled with mypy. It uses standard Python type hints to generate fast code (documentation).

  • Speed up accessing always defined native attributes (Jukka Lehtosalo, PR 12600)
  • Reduce the overhead of reference counting (Jukka Lehtosalo, PR 12805, PR 12810, PR 12817)
  • Fix Python 3.11 C API errors (97littleleaf11, PR 12850)
  • Generate smaller code for casts and some implicit type coercions (Jukka Lehtosalo, PR 12839)
  • Optimize calling Python objects with zero or one arguments (97littleleaf11, PR 12862)
  • Replace integer floor division by a power of two with a shift (Jukka Lehtosalo, PR 12870)
  • Add primitives for faster access of float and tuple type objects (Richard Si, PR 13078)
  • Fix compile error related to operator assignment in a generator function (Zsolt Dollenstein, PR 13144)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Allow ellipsis as default argument (Shantanu, PR 12838)
  • Support --version (Shantanu, PR 12852)
  • Check type variables and ParamSpecs (Shantanu, PR 12851)
  • Add error summary and other output tweaks (KotlinIsland, PR 12855)

Other Notable Fixes and Improvements

  • Disallow assignments to awaited coroutines that do not return (Shantanu, PR 12853)
  • Search sys.path for PEP 561 compliant packages (Ashley Whetter, PR 11143)
  • Treat generators with await as async (Jared Hance, PR 12925)
  • Fix bug in constraints solver regarding ParamSpec upper bounds (Alex Waygood, PR 12938)
  • Fix crash on redefined class variable annotated with Final[<type>] (Alex Waygood, PR 12951)
  • Improve handling of overloads with ParamSpec (Alex Waygood, PR 12953)
  • Don’t suggest installing types packages for some third-party packages that now include types or are obsolete (Shantanu, PR 12959)
  • Add a short note when an error may be fixed by adding an await (Ivan Levkivskyi, PR 12958)
  • Support unannotated converters for attr.ib (t4lz, PR 12815)
  • Disallow undesirable implicit reexport with a from import (Shantanu, PR 12704)
  • Fix crash when subclass method has the same name as a type alias (Wesley Collin Wright, PR 13015)
  • Include end column offset in the mypy AST (bruno messias, PR 12972)
  • Fix "attribute 'arguments' of 'FuncDef' undefined" incremental crash (Frédéric Perrin, PR 12324)
  • Fix false positive error on multiple enum base classes (Alex Waygood, PR 12963)
  • Don't add __match_args__ for dataclasses and named tuples on Python versions lower than 3.10 (Stanislav K, PR 12503)
  • Fix crash when overriding partial-type attribute with method (Jake Lishman, PR 12943)
  • Fix editable installs to current working directory (Shantanu, PR 13161)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • 97littleleaf11
  • Alex Waygood
  • Ashley Whetter
  • Bruno Messias
  • denballakh
  • Erik Kemperman
  • Ethan Smith
  • Fabian Keller
  • Frédéric Perrin
  • Hal Blackburn
  • Ivan Levkivskyi
  • Jake Lishman
  • Jared Hance
  • Jingchen Ye
  • KotlinIsland
  • Poruri Sai Rahul
  • Pranav Rajpal
  • Ramazan Elsunakev
  • Richard Si
  • Shantanu
  • Stanislav K
  • t4lz
  • Wesley Collin Wright

I would like to thank Ivan Levkivskyi, in particular, who did most of the preparations for this mypy release. My contribution was mainly to finish and publish this blog post.

We’d also like to thank our employer, Dropbox, for funding the mypy core team.