Friday 29 November 2019

Mypy 0.750 Released

We’ve just uploaded mypy 0.750 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many features, bug fixes and library stub (typeshed) updates. You can install it as follows: 

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

More Powerful Self-types

This version adds support for more patterns involving explicit annotations on self argument in methods. In particular, the mixin pattern got basic support via a protocol as a self annotation that specifies what the mixin expects from any class that subclasses it: 

    class Lockable(Protocol):
        @property
        def lock(self) -> Lock: ...
    
    class AtomicCloseMixin:
        def atomic_close(self: Lockable) -> int:
            with self.lock:  # This is OK
                ... # perform some actions
    
    class File(AtomicCloseMixin):
        def __init__(self) -> None:
            self.lock = Lock()
    
    class Bad(AtomicCloseMixin):
        pass
    
    f = File()
    b = Bad()
    f.atomic_close()  # OK
    b.atomic_close()  # Error: Invalid self type for "atomic_close"

Another somewhat common pattern that got support via explicit self-types is overloaded generic constructors (in particular in library stubs): 

    from typing import Any, AnyStr, Generic
    from typing_extensions import Literal
    
    class File(Generic[AnyStr]):
        @overload
        def __init__(self: File[bytes], raw: Literal[True]) -> None: ...
        @overload
        def __init__(self: File[str], raw: Literal[False]) -> None: ...
        @overload
        def __init__(self: File[Any], raw: bool = ...) -> None: ...
    
    reveal_type(File())  # type is File[Any]
    reveal_type(File(raw=True))  # type is File[bytes]

Read the docs for more examples and use cases.

Stubgen Revamped

Stubgen, the automatic stub generator that comes with mypy, has many fixes and improvements. In particular:

  • Recover from various errors that broke stub generation previously, such as inconsistent MROs or modules that crash on import.
  • Export imported names that aren't referenced in the module.
  • Export all names imported from the current package by default. Use --export-less to disable this behavior.
  • Fix various issues that resulted in syntactically invalid stubs being generated.
  • Fix signatures of various C extension methods.
  • Make the docstring parser more robust.
  • Add various heuristics to skip internal modules, such as tests and vendored packages.
  • Allow tweaking the level of logging. Add --verbose and --quiet flags.

Generating stub files has never been so easy and robust! Please read more docs here and try it on the libraries you maintain or use that don’t yet have stubs in Typeshed. Please consider contributing any stubs you create to Typeshed so that the community can share and improve them.

Mypy Daemon is No Longer Experimental

The mypy daemon has been the default way of running mypy locally at Dropbox for well over a year and it's quite stable. It also got several additional commands and flags for better customization (and existing flags got better documentation). In particular, mypy daemon now supports basic integration with external file system event watchers for even faster, sub-second type checking results over tens of thousands of files.

Use dmypy recheck --update FILE and dmypy recheck --remove FILE to directly tell the daemon which files got edited/deleted to avoid waiting for the built-in file system watcher. For more details, see the docs.

Static Inference of Annotations (Experimental)

The mypy daemon now supports (as an experimental feature) statically inferring a draft type annotation for a given function or method. Running dmypy suggest FUNCTION will produce a suggested signature in this format: 

    (param_type_1, param_type_2, ...) -> ret_type

This is a low-level command that can be used by editors, IDEs, and other tools, such as the mypy plugin for PyCharm, to propose an annotation to the user, and/or to insert an annotation to a source file.

In this example, the function format_id() has no annotation: 

    def format_id(user):
        return "User: {}".format(user)
    
    root = format_id(0)

Mypy uses call sites and return statements to infer that format_id() accepts an int and returns a str. For more details, see the docs.

Other Notable Improvements and Bug Fixes

  • Show closest candidates for misspelled module attributes (Theodore Liu, PR 7971)
  • Fix C string encoding in the mypyc compiler (Saleem Rashid, PR 7978)
  • Fix type-checking decorated generic methods (beezee, PR 7933)
  • Make revealed type of final variables distinct from non-final ones (Michael Lee, PR 7955)
  • Fix issubclass() to narrow down types of type variables (Xuanda Yang, PR 7930)
  • Refine types of parent expressions when narrowing expression types (Michael Lee, PR 7917)
  • Fix starting compiled mypy daemon on Windows (PR 7929)
  • Show closest candidates for misspelled keyword arguments (Martijn Wuis, PR 7888)
  • Fix type checking of ternary expressions with an empty collection (Xuanda Yang, PR 7892)
  • Support negative integers in literal types (Xuanda Yang, PR 7878)
  • Make sure type inference works for literals against protocols (PR 7882)
  • Don't add coercions when compiling expression statements via mypyc (jag426, PR 7872)
  • Fix crashes in mypy daemon when sorting union items (PR 7842)
  • Enable whole-file # type: ignore comments in Python 2.7 (Brandt Bucher, PR 7789)
  • Record internal Any types as precise in HTML report generator (ahmadF, PR 7708)
  • Add misc/dump-ast.py script to dump parse trees (Michael Wayne Goodman, PR 7820)
  • Use union component as a self-type in class methods (PR 7822)
  • Correctly handle union types in string interpolation checker (Vincent Barbaresi, PR 7809)
  • Fix capturing stderr in mypy.api.run (David Tucker, PR 7804)
  • Fix internal error when checking dataclasses with a duplicate field (Vincent Barbaresi, PR 7808)
  • Add cross references to mypy command line options in the documentation (Oleg Höfling, PR 7784, PR 7801, PR 7802)
  • Improve missing import error message (Lewis Cowles, PR 7698)
  • Add option to print absolute file paths (Maxim Koltsov, PR 7754)
  • Recommend using string escaping for classes that are non-generic at runtime (Rubin Raithel, PR 7707)
  • Fix accessing unannotated implicit class methods (PR 7739)
  • Ensure that non-total TypedDicts can be false (henribru, PR 7745)
  • Fix type checking __init_subclass__() (PR 7723)

Plugin API Changes and Improvements

This release includes three major mypy plugin API changes:

  • There is a new plugin hook for specifying per-module configuration data: report_config_data(). It can be used if the plugin has some sort of per-module configuration that can affect type checking.
  • *(breaking change)* New TypeAliasType type has been added and is now used for type aliases instead of eagerly expanding them. To get the expanded type use mypy.types.get_proper_type(). See this comment for more compatibility instructions.
  • *(breaking change)* Symbol node methods .fullname() and .name() were replaced with properties. See this comment for compatibility instructions.

These improvements are related to plugins:

  • Invoke the get_dynamic_class_hook() plugin hook also on method calls (Seth Yastrov, PR 7990)
  • Fix plugin invocation on class method calls through Type[...] (PR 7969)
  • Allow new plugins to be added by programs (like mypyc) that directly invoke build (PR 7875)

For more information on plugin hooks see the docs. Breaking API changes are announced in this GitHub issue. Please subscribe to it if you are an active mypy plugin developer.

Typeshed Updates

Many small improvements were made to typeshed — too many to list. Browse the typeshed commit log here.

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • ahmadF
  • Alexandre Viau
  • beezee
  • Bob Whitelock
  • Brandt Bucher
  • Chad Dombrova
  • Daniel Hahler
  • David Euresti
  • David Tucker
  • Djedouas
  • Ekin Dursun
  • Emmanuel Nosa Evbuomwan
  • Ethan Smith
  • Gábor Lipták
  • henribru
  • hoefling
  • Ilaï Deutel
  • jag426
  • Lewis Cowles
  • Maxim Koltsov
  • Michael Lee
  • Michael R. Crusoe
  • Michael Wayne Goodman
  • Rubin Raithel
  • Saleem Rashid
  • Sebastian Rittau
  • Seth Yastrov
  • TH3CHARLie
  • Theodore Liu
  • Thomas Hisch
  • Tim Gates
  • Tudor Brindus
  • Tuomas Suutari
  • Vasily Zakharov
  • Vincent Barbaresi
  • Wuisch
  • Xuanda Yang
  • Zac Hatfield-Dodds

Additional thanks to all contributors to typeshed:

  • Alexander Schlarb
  • Angela Ambroz
  • Anthony Sottile
  • Árni Már Jónsson
  • Benjamin Peterson
  • bianca rosa
  • Brett Cannon
  • Christopher Hunt
  • cptpcrd
  • Daniel Hahler
  • David Tucker
  • Denis Laxalde
  • Diego Elio Pettenò
  • Dima Boger
  • Eric N. Vander Weele
  • Florian Bruhin
  • Greg Ward
  • Ilaï Deutel
  • Jakub Stasiak
  • Jelle Zijlstra
  • Jeremy Lainé
  • Jon Dufresne
  • JR Heard
  • Lawrence Chan
  • Maarten ter Huurne
  • Markus Bauer
  • Martijn Pieters
  • Martin DeMello
  • Michael Lee
  • Michael Seifert
  • Michał Słapek
  • Nathaniel Brahms
  • Nikita Sobolev
  • Nikola Forró
  • Ran Benita
  • Rebecca Chen
  • Rune Tynan
  • Ryan Morshead
  • Sebastian Rittau
  • Taneli Hukkinen
  • Trim21
  • Utkarsh Gupta
  • Vasily Zakharov
  • 秋葉
Posted by at