Monday, 6 February 2023

Mypy 1.0 Released

We’ve just uploaded mypy 1.0 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows: 

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

New Release Versioning Scheme

Now that mypy reached 1.0, we’ll switch to a new versioning scheme. Mypy version numbers will be of form x.y.z.

Rules:

  • The major release number (x) is incremented if a feature release includes a significant backward incompatible change that affects a significant fraction of users.
  • The minor release number (y) is incremented on each feature release. Minor releases include updated stdlib stubs from typeshed.
  • The point release number (z) is incremented when there are fixes only.

Mypy doesn't use SemVer, since most minor releases have at least minor backward incompatible changes in typeshed, at the very least. Also, many type checking features find new legitimate issues in code. These are not considered backward incompatible changes, unless the number of new errors is very high.

Any significant backward incompatible change must be announced in the blog post for the previous feature release, before making the change. The previous release must also provide a flag to explicitly enable or disable the new behavior (whenever practical), so that users will be able to prepare for the changes and report issues. We should keep the feature flag for at least a few releases after we've switched the default.

See ”Release Process” in the mypy wiki for more details and for the most up-to-date version of the versioning scheme.

Performance Improvements

Mypy 1.0 is up to 40% faster than mypy 0.991 when type checking the Dropbox internal codebase. We also set up a daily job to measure the performance of the most recent development version of mypy to make it easier to track changes in performance.

Many optimizations contributed to this improvement:

  • Improve performance for errors on class with many attributes (Shantanu, PR 14379)
  • Speed up make_simplified_union (Jukka Lehtosalo, PR 14370)
  • Micro-optimize get_proper_type(s) (Jukka Lehtosalo, PR 14369)
  • Micro-optimize flatten_nested_unions (Jukka Lehtosalo, PR 14368)
  • Some semantic analyzer micro-optimizations (Jukka Lehtosalo, PR 14367)
  • A few miscellaneous micro-optimizations (Jukka Lehtosalo, PR 14366)
  • Optimization: Avoid a few uses of contextmanagers in semantic analyzer (Jukka Lehtosalo, PR 14360)
  • Optimization: Enable always defined attributes in Type subclasses (Jukka Lehtosalo, PR 14356)
  • Optimization: Remove expensive context manager in type analyzer (Jukka Lehtosalo, PR 14357)
  • subtypes: fast path for Union/Union subtype check (Hugues, PR 14277)
  • Micro-optimization: avoid Bogus[int] types that cause needless boxing (Jukka Lehtosalo, PR 14354)
  • Avoid slow error message logic if errors not shown to user (Jukka Lehtosalo, PR 14336)
  • Speed up the implementation of hasattr() checks (Jukka Lehtosalo, PR 14333)
  • Avoid the use of a context manager in hot code path (Jukka Lehtosalo, PR 14331)
  • Change various type queries into faster bool type queries (Jukka Lehtosalo, PR 14330)
  • Speed up recursive type check (Jukka Lehtosalo, PR 14326)
  • Optimize subtype checking by avoiding a nested function (Jukka Lehtosalo, PR 14325)
  • Optimize type parameter checks in subtype checking (Jukka Lehtosalo, PR 14324)
  • Speed up freshening type variables (Jukka Lehtosalo, PR 14323)
  • Optimize implementation of TypedDict types for **kwds (Jukka Lehtosalo, PR 14316)

Warn About Variables Used Before Definition

Mypy will now generate an error if you use a variable before it’s defined. This feature is enabled by default. By default mypy reports an error when it infers that a variable is always undefined. 

    y = x  # E: Name "x" is used before definition [used-before-def]
    x = 0

This feature was contributed by Stas Ilinskiy.

Detect Possibly Undefined Variables (Experimental)

A new experimental possibly-undefined error code is now available that will detect variables that may be undefined: 

    if b:
        x = 0
    print(x)  # Error: Name "x" may be undefined [possibly-undefined]

The error code is disabled be default, since it can generate false positives.

This feature was contributed by Stas Ilinskiy.

Support the “Self” Type

There is now a simpler syntax for declaring generic self types introduced in PEP 673: the Self type. You no longer have to define a type variable to use “self types”, and you can use them with attributes. Example from mypy documentation: 

    from typing import Self
    
    class Friend:
        other: Self | None = None
    
        @classmethod
        def make_pair(cls) -> tuple[Self, Self]:
            a, b = cls(), cls()
            a.other = b
            b.other = a
            return a, b
    
    class SuperFriend(Friend):
        pass
    
    # a and b have the inferred type "SuperFriend", not "Friend"
    a, b = SuperFriend.make_pair()

The feature was introduced in Python 3.11. In earlier Python versions a backport of Self is available in typing_extensions.

This was contributed by Ivan Levkivskyi (PR 14041).

Support ParamSpec in Type Aliases

ParamSpec and Concatenate can now be used in type aliases. Example: 
    from typing import ParamSpec, Callable
    
    P = ParamSpec("P")
    A = Callable[P, None]
    
    def f(c: A[int, str]) -> None:
        c(1, "x")

This feature was contributed by Ivan Levkivskyi (PR 14159).

ParamSpec and Generic Self Types No Longer Experimental

Support for ParamSpec (PEP 612) and generic self types are no longer considered experimental.

Miscellaneous New Features

  • Minimal, partial implementation of dataclass_transform (PEP 681) (Wesley Collin Wright, PR 14523)
  • Add basic support for typing_extensions.TypeVar (Marc Mueller, PR 14313)
  • Add --debug-serialize option (Marc Mueller, PR 14155)
  • Constant fold initializers of final variables (Jukka Lehtosalo, PR 14283)
  • Enable Final instance attributes for attrs (Tin Tvrtković, PR 14232)
  • Allow function arguments as base classes (Ivan Levkivskyi, PR 14135)
  • Allow super() with mixin protocols (Ivan Levkivskyi, PR 14082)
  • Add type inference for dict.keys membership (Matthew Hughes, PR 13372)
  • Generate error for class attribute access if attribute is defined with __slots__ (Harrison McCarty, PR 14125)
  • Support additional attributes in callback protocols (Ivan Levkivskyi, PR 14084)

Fixes to Crashes

  • Fix crash on prefixed ParamSpec with forward reference (Ivan Levkivskyi, PR 14569)
  • Fix internal crash when resolving the same partial type twice (Shantanu, PR 14552)
  • Fix crash in daemon mode on new import cycle (Ivan Levkivskyi, PR 14508)
  • Fix crash in mypy daemon (Ivan Levkivskyi, PR 14497)
  • Fix crash on Any metaclass in incremental mode (Ivan Levkivskyi, PR 14495)
  • Fix crash in await inside comprehension outside function (Ivan Levkivskyi, PR 14486)
  • Fix crash in Self type on forward reference in upper bound (Ivan Levkivskyi, PR 14206)
  • Fix a crash when incorrect super() is used outside a method (Ivan Levkivskyi, PR 14208)
  • Fix crash on overriding with frozen attrs (Ivan Levkivskyi, PR 14186)
  • Fix incremental mode crash on generic function appearing in nested position (Ivan Levkivskyi, PR 14148)
  • Fix daemon crash on malformed NamedTuple (Ivan Levkivskyi, PR 14119)
  • Fix crash during ParamSpec inference (Ivan Levkivskyi, PR 14118)
  • Fix crash on nested generic callable (Ivan Levkivskyi, PR 14093)
  • Fix crashes with unpacking SyntaxError (Shantanu, PR 11499)
  • Fix crash on partial type inference within a lambda (Ivan Levkivskyi, PR 14087)
  • Fix crash with enums (Michael Lee, PR 14021)
  • Fix crash with malformed TypedDicts and disllow-any-expr (Michael Lee, PR 13963)

Error Reporting Improvements

  • More helpful error for missing self (Shantanu, PR 14386)
  • Add error-code truthy-iterable (Marc Mueller, PR 13762)
  • Fix pluralization in error messages (KotlinIsland, PR 14411)

Mypyc: Support Match Statement

Mypyc can now compile Python 3.10 match statements.

This was contributed by dosisod (PR 13953).

Other Mypyc Fixes and Improvements

  • Optimize int(x)/float(x)/complex(x) on instances of native classes (Richard Si, PR 14450)
  • Always emit warnings (Richard Si, PR 14451)
  • Faster bool and integer conversions (Jukka Lehtosalo, PR 14422)
  • Support attributes that override properties (Jukka Lehtosalo, PR 14377)
  • Precompute set literals for "in" operations and iteration (Richard Si, PR 14409)
  • Don't load targets with forward references while setting up non-extension class __annotations__ (Richard Si, PR 14401)
  • Compile away NewType type calls (Richard Si, PR 14398)
  • Improve error message for multiple inheritance (Joshua Bronson, PR 14344)
  • Simplify union types (Jukka Lehtosalo, PR 14363)
  • Fixes to union simplification (Jukka Lehtosalo, PR 14364)
  • Fix for typeshed changes to Collection (Shantanu, PR 13994)
  • Allow use of enum.Enum (Shantanu, PR 13995)
  • Fix compiling on Arch Linux (dosisod, PR 13978)

Documentation Improvements

  • Various documentation and error message tweaks (Jukka Lehtosalo, PR 14574)
  • Improve Generics documentation (Shantanu, PR 14587)
  • Improve protocols documentation (Shantanu, PR 14577)
  • Improve dynamic typing documentation (Shantanu, PR 14576)
  • Improve the Common Issues page (Shantanu, PR 14581)
  • Add a top-level TypedDict page (Shantanu, PR 14584)
  • More improvements to getting started documentation (Shantanu, PR 14572)
  • Move truthy-function documentation from “optional checks” to “enabled by default” (Anders Kaseorg, PR 14380)
  • Avoid use of implicit optional in decorator factory documentation (Tom Schraitle, PR 14156)
  • Clarify documentation surrounding install-types (Shantanu, PR 14003)
  • Improve searchability for module level type ignore errors (Shantanu, PR 14342)
  • Advertise mypy daemon in README (Ivan Levkivskyi, PR 14248)
  • Add link to error codes in README (Ivan Levkivskyi, PR 14249)
  • Document that report generation disables cache (Ilya Konstantinov, PR 14402)
  • Stop saying mypy is beta software (Ivan Levkivskyi, PR 14251)
  • Flycheck-mypy is deprecated, since its functionality was merged to Flycheck (Ivan Levkivskyi, PR 14247)
  • Update code example in "Declaring decorators" (ChristianWitzler, PR 14131)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Improve error message for __all__-related errors (Alex Waygood, PR 14362)
  • Improve heuristics for determining whether global-namespace names are imported (Alex Waygood, PR 14270)
  • Catch BaseException on module imports (Shantanu, PR 14284)
  • Associate exported symbol error with __all__ object_path (Nikita Sobolev, PR 14217)
  • Add __warningregistry__ to the list of ignored module dunders (Nikita Sobolev, PR 14218)
  • If a default is present in the stub, check that it is correct (Jelle Zijlstra, PR 14085)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Treat dlls as C modules (Shantanu, PR 14503)

Other Notable Fixes and Improvements

  • Update stub suggestions based on recent typeshed changes (Alex Waygood, PR 14265)
  • Fix attrs protocol check with cache (Marc Mueller, PR 14558)
  • Fix strict equality check if operand item type has custom __eq__ (Jukka Lehtosalo, PR 14513)
  • Don't consider object always truthy (Jukka Lehtosalo, PR 14510)
  • Properly support union of TypedDicts as dict literal context (Ivan Levkivskyi, PR 14505)
  • Properly expand type in generic class with Self and TypeVar with values (Ivan Levkivskyi, PR 14491)
  • Fix recursive TypedDicts/NamedTuples defined with call syntax (Ivan Levkivskyi, PR 14488)
  • Fix type inference issue when a class inherits from Any (Shantanu, PR 14404)
  • Fix false positive on generic base class with six (Ivan Levkivskyi, PR 14478)
  • Don't read scripts without extensions as modules in namespace mode (Tim Geypens, PR 14335)
  • Fix inference for constrained type variables within unions (Christoph Tyralla, PR 14396)
  • Fix Unpack imported from typing (Marc Mueller, PR 14378)
  • Allow trailing commas in ini configuration of multiline values (Nikita Sobolev, PR 14240)
  • Fix false negatives involving Unions and generators or coroutines (Shantanu, PR 14224)
  • Fix ParamSpec constraint for types as callable (Vincent Vanlaer, PR 14153)
  • Fix type aliases with fixed-length tuples (Jukka Lehtosalo, PR 14184)
  • Fix issues with type aliases and new style unions (Jukka Lehtosalo, PR 14181)
  • Simplify unions less aggressively (Ivan Levkivskyi, PR 14178)
  • Simplify callable overlap logic (Ivan Levkivskyi, PR 14174)
  • Try empty context when assigning to union typed variables (Ivan Levkivskyi, PR 14151)
  • Improvements to recursive types (Ivan Levkivskyi, PR 14147)
  • Make non-numeric non-empty FORCE_COLOR truthy (Shantanu, PR 14140)
  • Fix to recursive type aliases (Ivan Levkivskyi, PR 14136)
  • Correctly handle Enum name on Python 3.11 (Ivan Levkivskyi, PR 14133)
  • Fix class objects falling back to metaclass for callback protocol (Ivan Levkivskyi, PR 14121)
  • Correctly support self types in callable ClassVar (Ivan Levkivskyi, PR 14115)
  • Fix type variable clash in nested positions and in attributes (Ivan Levkivskyi, PR 14095)
  • Allow class variable as implementation for read only attribute (Ivan Levkivskyi, PR 14081)
  • Prevent warnings from causing dmypy to fail (Andrzej Bartosiński, PR 14102)
  • Correctly process nested definitions in mypy daemon (Ivan Levkivskyi, PR 14104)
  • Don't consider a branch unreachable if there is a possible promotion (Ivan Levkivskyi, PR 14077)
  • Fix incompatible overrides of overloaded methods in concrete subclasses (Shantanu, PR 14017)
  • Fix new style union syntax in type aliases (Jukka Lehtosalo, PR 14008)
  • Fix and optimise overload compatibility checking (Shantanu, PR 14018)
  • Improve handling of redefinitions through imports (Shantanu, PR 13969)
  • Preserve (some) implicitly exported types (Shantanu, PR 13967)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alessio Izzo
  • Alex Waygood
  • Anders Kaseorg
  • Andrzej Bartosiński
  • Avasam
  • ChristianWitzler
  • Christoph Tyralla
  • dosisod
  • Harrison McCarty
  • Hugo van Kemenade
  • Hugues
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jelle Zijlstra
  • jhance
  • johnthagen
  • Jonathan Daniel
  • Joshua Bronson
  • Jukka Lehtosalo
  • KotlinIsland
  • Lakshay Bisht
  • Lefteris Karapetsas
  • Marc Mueller
  • Matthew Hughes
  • Michael Lee
  • Nick Drozd
  • Nikita Sobolev
  • Richard Si
  • Shantanu
  • Stas Ilinskiy
  • Tim Geypens
  • Tin Tvrtković
  • Tom Schraitle
  • Valentin Stanciu
  • Vincent Vanlaer

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Posted by at