The Mypy Blog: June 2021

We’ve just uploaded mypy 0.900 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many new features and bug fixes. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Update: Released mypy 0.901 that fixes a crash when using --install-types.

Third-party Library Stubs in Stub Packages (Breaking Change)

Mypy now only ships with type stubs for stdlib modules (plus typing_extensions and mypy_extensions). If you use third-party libraries that don’t include stubs or inline type annotations, you can explicitly install stub packages, usually called types-<name>.

For example, if you use requests, you can install stubs like this:

    python3 -m pip install types-requests

If mypy can’t find stubs, it will suggest the name of the stub package to install:

  t.py:1: error: Library stubs not installed for "requests" 
      (or incompatible with Python 3.8)
  t.py:1: note: Hint: "python3 -m pip install types-requests"
  t.py:1: note: (or run "mypy --install-types" to install all 
      missing stub packages)
  t.py:1: note: See https://mypy.readthedocs.io/en/...

You can also now run mypy --install-types to install all missing stub packages in the mypy run (or the previous run, if you don’t pass any files to check):

  $ mypy --install-types
  Installing missing stub packages:
  /Users/jukka/venv/mypy/bin/python3 -m pip install types-requests

  Install? [yN] y

  Collecting types-requests
    Using cached types_requests-0.1.8-py2.py3-none-any.whl (22 kB)
  Installing collected packages: types-requests
  Successfully installed types-requests-0.1.8

Note: --install-types isn’t supported by mypy daemon.

Note: For security reasons, the suggested stubs are limited to only a small subset of manually selected packages that have been verified by the typeshed team. These packages include only stub files and no executable code.

Note: Even if you are using --ignore-missing-imports, mypy will ask to install stubs for packages that had bundled stubs in previous mypy releases. This helps you avoid losing type checking precision from missing stubs when upgrading to mypy 0.900.

For more information, read the blog post about this change and why we did this. There is more detail in the docs.

Python 2 Support is Opt-in (Breaking Change)

If you want to type check Python 2 code, you now have to install mypy[python2] using pip (on Python 3.8 and later):

    python3 -m pip install -U mypy[python2]

This will install the additional typed-ast pip dependency that is required for Python 2 support. Type checking Python 2 code is still fully supported by mypy.

This was contributed by Shantanu (PR 10183).

TypeGuard

Mypy now supports defining user-defined functions that perform type narrowing, similar to isinstance. Read PEP 647 for the details. Here’s an example adapted from the PEP:

    from typing_extensions import TypeGuard

    def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
        """Are all list items strings?"""
        return all(isinstance(x, str) for x in val)

    def func(val: List[object]) -> None:
        if is_str_list(val):
            print(" ".join(val))  # OK

Since TypeGuard will be included in Python 3.10, which hasn’t been released yet, you will have to import it from a recent version of typing_extensions for now.

This was contributed by Guido van Rossum (PR 9865).

Support for pyproject.toml

You can now use a pyproject.toml file instead of mypy.ini for mypy configuration. Here is an example from the documentation:

    # mypy global options:

    [tool.mypy]
    python_version = "2.7"
    warn_return_any = true
    warn_unused_configs = true

    # mypy per-module options:

    [[tool.mypy.overrides]]
    module = "mycode.foo.*"
    disallow_untyped_defs = true

    [[tool.mypy.overrides]]
    module = "mycode.bar"
    warn_return_any = false

    [[tool.mypy.overrides]]
    module = [
        "somelibrary",
        "some_other_library"
    ]
    ignore_missing_imports = true

This was contributed by Adam Weeden (PR 10219).

Binary Wheels for Apple Silicon

We now include binary wheels for Apple Silicon. This can make mypy runs over 4x faster on supported hardware. This requires a native Apple Silicon build of Python 3.9 to work.

This was contributed by Jingchen Ye (97littleleaf11, PR 10145).

Type Narrowing with type(x)

Mypy now recognizes that checks such as type(x) is str narrow the type of x to str. Previously you had to use isinstance(x, str). Example:

    def example(x: Union[int, str]) -> bool:
        if type(x) is str:
            # Type of "v" is "str" here
            ...
        else:
            # Type of "v" is Union[int, str] here
            # "v" could be a instance of a "str" subclass!
            ...

This was contributed by Pranav Rajpal (PR 10284).

Hosted Documentation for Mypyc

We now have hosted user documentation for mypyc, the compiler we use to speed up mypy. Mypyc is still experimental, but we are looking for early adopters.

New Mypyc Features

Support __setitem__, __delitem__, __len__ and __contains__ in native classes (PR 10451)
Basic support for class-based named tuples (PR 10252)
Support class-based TypedDict definitions (PR 10226)

Mypyc Performance Improvements

Pre-allocate space in list comprehensions (97littleleaf11, PR 10295)
Faster dict true test (97littleleaf11, PR 10333)
Implement dict.setdefault primitive (97littleleaf11, PR 10286)
Optimize truth value testing for strings (pranavrajpal, PR 10269)
Faster set creation with generator expression (97littleleaf11, PR 10261)
Add a primitive for loading the bool type (97littleleaf11, PR 10257)
Faster creation of a tuple from list/tuple (97littleleaf11, PR 10217)
Use vectorcalls to speed up calls to Python objects (PR 10153)
Add primitive for str.replace (97littleleaf11, PR 10088)
Add primitives for list.index() and list.remove() (PR 9961)
Intern string literals (PR 9960)
Add fast path for simple calls in wrapper functions (PR 9948)
Use METH_FASTCALL with __call__ (PR 9946)
Use faster METH_FASTCALL wrapper functions on Python 3.7+ (PR 9894)
Add primitives for list.sort() and list.reverse() (PR 9897)
Recognize six.moves.xrange as an alias of range (PR 9896)
Some tagged integer micro-optimizations (PR 9801)
Add primitive for dict.copy (vsakkas, PR 9721)

Other Mypyc Fixes and Improvements

Fix reference counting in dict.setdefault (97littleleaf11, PR 10334)
Don't coerce types checked with isinstance (Sara Sinback, PR 10245)
Simplify argument parsing in legacy wrapper functions (PR 10234)
Simplify generated C by omitting gotos to the next block (PR 10230)
Box initializers of final variables when required (PR 10229)
Fix compilation of unreachable expressions (PR 10228)
Foundational support for tuple literals (and None and bool literals, PR 10148)
Use tables to construct and store literals (PR 10147)
Fix overflow error handling in list.insert (PR 9895)
Always add implicit None return type to __init__ method (Thomas Johnson, PR 9866)
Small documentation updates (Thomas Johnson, PR 10184)
Updates to introduction in the documentation (PR 10179)
Various small doc updates (PR 10177)
Make mypyc docs use the same theme as mypy docs (PR 10176)
Fix mypyc failing to compile on CPython 3.10.0a6 (Adrian Freund, PR 10202)
Avoid declaring variable in C for loop initialization (Thomas Johnson, PR 10091)

Stubgen Improvements

Properly convert overloaded functions (Chad Dombrova, PR 9613)
Fix invoking mro() (Gen Xu, PR 10138)
Render a bit better stubs (Sergei Izmailov, PR 9903)
Fixed Any and Dict missing from stubgen imports list (Ashley Whetter, PR 9900)
Fix type error (Shantanu, PR 10523)
Fixes and typos (Sergei Izmailov, PR 9877)

Stubtest Improvements

Improve build error messages (Shantanu, PR 10365)
Check overloads are functions, don't early return (Shantanu, PR 10258)
Fix signature construction for overload + implicit classmethod (Shantanu, PR 9921)

Documentation Updates

Add security documentation for --install-types (Adam Weeden, PR 10555)
Improve documentation of extending mypy (Nikita Sobolev, PR 10222)
Update documentation links (Shantanu, PR 10159)
Fix incomplete error codes documentation (Iulian Onofrei, PR 9799)
Document explicit_package_bases config file option (Sam Bull, PR 10020)
Clarify that mypy_path is relative to current working directory (Gustav Gränsbo, PR 9959)
Emphasise that users probably want --follow-imports=normal (Shantanu, PR 9955)

Other Notable Fixes and Improvements

Make ignore_missing_imports work for third-party libraries which previously had bundled stubs (PR 10582)
Restrict the number of errors shown when there are missing stubs (PR 10579)
Mention fullname of PlaceholderNode in assertion to simplify debugging (Adam Weeden, PR 10565)
Use double quotes in error messages instead of single quotes (Dixith)
Use double quotes in various error messages (Prasanth Chettri, PR 10053)
Fix crash on TypeGuard plus "and" (Jelle Zijlstra, PR 10496)
Fix spurious name undefined error in class body within import cycle (PR 10498)
Remove assertion from TypeGuard serialization (Jelle Zijlstra, PR 10486)
Do not allow to use Final and ClassVar at same time (Yurii Karabas, PR 10478)
Allow running mypy if Python 2 typeshed stubs aren't installed (PR 10494)
Allow inline comments in VERSIONS (Sebastian Rittau, PR 10472)
Infer unreachability for await no_return() (Shantanu, PR 10458)
Fix unresolved reference + tuples crashing (EXPLOSION, PR 10401)
Fix crash with type alias inside __init__ in incremental mode (PR 10432)
Fix crash with nested NamedTuple in incremental mode (PR 10431)
Fix crash related to module-level __getattr__ in incremental mode (PR 10430)
Add support for functools.cached_property (Marc Mueller, PR 10408)
Don't narrow Type[X] with a metaclass (PR 10424)
Remove note: setup.cfg does need [mypy] section (James Cooke, PR 10364)
Fix inference of IntEnum value attribute type (Python 2, PR 10417)
Don't report error when using total_ordering in Python 2 mode (PR 10416)
Fix narrowing down TypedDict unions with enum literal types (PR 10415)
Fix inference of IntEnum value attribute type (PR 10412)
Use typeddict-item error code for more errors (PR 10410)
Fix a bunch of tests broken by python 3.10 (Adrian Freund, PR 10404)
Support maximum version marker of stdlib typeshed modules (Sebastian Rittau, PR 10402)
First look into @python2 subdirectory of search path items in Python 2 mode (PR 10392)
Make enum type compatible with union of all enum item literals (PR 10388)
Fix an issue with union types containing literals (Adrian Freund, PR 10373)
Fix re-exporting __all__ in a stub file (PR 10382)
Extend the dataclass plugin to deal with callable properties (Aaron Ecay, PR 10292)
Make TypedDict incompatible type message more understandable (Akshay K, PR 10326)
Make --explicit-package-bases invertible (Gustav Gränsbo, PR 9969)
Fix crash for protocol classes and Hashable false negative (Christoph Tyralla, PR 10308)
Fix strict equality false positive when strict optional disabled (Kamil Turek, PR 10174)
Issue an error when overriding typeguard with non-typeguard in subclass (Kamil Turek, PR 10300)
Support functools.total_ordering (Ashley Whetter, PR 7831)
Support reversed operand order when comparing against sys.version_info (Kamil Turek, PR 10288)
Only import distutils if it is needed (Adrian Freund, PR 10203)
Recombine complete union of enum literals into original type (PR 9097, PR 9063) (Ethan Leba)
Support union type arguments when showing protocol member conflicts (97littleleaf11, PR 10154)
Fix recently added enum value type prediction (Paddy, PR 10057)
Support subclassing of NodeVisitor and TraverserVisitor in plugins (Nikita Sobolev, PR 10125)
Speed up processing of new files in daemon by caching ASTs (PR 10128)
Add narrowing and closures to common issues (Shantanu, PR 9956)
Add Python script to build wheels using cibuildwheel (PR 10096)
Don't try to follow imports to encodings.* (PR 10094)
Speed up case sensitive is-file check in file system cache (PR 10093)
Use fast path for finding modules in mypy daemon (PR 10095)
Fix following stub imports in daemon when using --follow-imports=skip (PR 10092)
Use sys.executable instead of python3 for --install-types (Anthony Sottile, PR 10086)
Add build-requirements.txt to MANIFEST.in (PR 10071)
Add separate requirements file for mypy self-compile (PR 10069)
Expand user home for cache_dir (Marco Zatta, PR 10051)
Fix daemon crash when deleting packages (PR 10036)
Keep track of fine-grained dependencies for modules in typeshed (PR 10034)
Fix import following issue in daemon (PR 10032)
Fix mypy daemon crash when following from a new module (PR 10030)
Detect invalid value for --custom-typeshed-dir (PR 9985)
Fix pip install for new typeshed (Ivan Levkivskyi, PR 9976)
Report incompatible assignments for descriptors with overloaded __set__ methods (Christoph Tyralla, PR 9893)
Fix TypedDict.get("missing_key") with string literal (Adrian Freund, PR 9906)

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

Aaron Ecay
Adam Weeden
Adrian Freund
Akshay K
Akuli
Anthony Sottile
Arjun
Ashley Whetter
Chad Dombrova
Christoph Tyralla
Dixith
Ethan Leba
EXPLOSION
Gen Xu
Gustav Gränsbo
hatal175
Iulian Onofrei
James Cooke
Jelle Zijlstra
Jesús Cea
Jingchen Ye (97littleleaf11)
Kamil Turek
Kyle Verhoog
Marc Mueller
Marco Gorelli
Marco Zatta
Nikita Sobolev
Paddy
Philip Jägenstedt
Pranav Rajpal
Prasanth Chettri
Raphael Gaschignard
Reda Bouaida
Saiprasad Kale
Sam Bull
Sara Sinback
Sebastian Rittau
Sergei Izmailov
Shantanu
Thomas Johnson
vsakkas
wwuck
Xuanda Yang
Yurii Karabas

Additional thanks to all contributors to typeshed:

Adrian Freund
Akuli
Alexander Reynolds
Andre Delfino
Andrew Crabtree
Andrew Zhou
Andrey
Ankur Singh
Anthony Sottile
Anton Agestam
Arie Bovenberg
Árni Már Jónsson
Ashwin Ramaswami
baterflyrity
Cássio Botaro
Christopher Dignam
Christopher Gurnee
Christopher Sang
Daniel Rosenwasser
danieleades
Dante Camarena
David Barker
Dean Way
dearoneesama
dod-wdwilcox
Dominic Davis-Foster
Edgar Handal
Eric Traut
Erik Soma
Giel van Schijndel
Gilad Reti
Greg Ward
Guilhem C
Guillaume Ayoub
harens
hatal175
Henry Schreiner
heroesm
Igor Nehoroshev
Ilya Konstantinov
Iwan
jack1142
Jacob Hayes
Jake Bailey
James
James Morris
Jason Huang
Jelle Zijlstra
Jia Chen
John T. Wodder II
Jon Banafato
Jon Dufresne
JonathanMeans
Jörg Herbel
Jürgen Gmach
Kaleb Barrett
Ken Odegard
krouziciorel
Kyle Altendorf
ldruschk
Mahmoud Harmouch
Marat Sharafutdinov
Mark Vismonte
Marks Polakovs
Marti Raudsepp
Mathieu Parent
Maxime Arthaud
Mohit Goyani
MrGreenTea
n0ob404
naglis
Nick Crews
Nick M
Nipunn Koorapati
Philipp Hahn
Rebecca Chen
Ricardo
rijenkii
Rune Tynan
Ryan McCampbell
sachindavra
Sam Bull
Sean B. Palmer
Sebastian Rittau
Shantanu
Sorin Sbarnea
spaghEddieDoyle
Stanislav Levin
Steve B
Tadek Kijkowski
Takumi Kato
Terence Honles
Thomas Cellerier
Tigran Kostandyan
Tushar Sadhwani
Udi Fuchs
Utsav
Vasily Zakharov
Viktor Roytman
Vyom Pathak
Wilfredo Sánchez Vega
Wolf Honore
Yehuda Davis

The Mypy Blog

Tuesday, 22 June 2021

Mypy 0.910 Released

Installing Stub Packages Non-Interactively

Python 3.5 Deprecation

Stubgen Improvements

Other Fixes and Improvements

Typeshed Updates

Acknowledgements

Tuesday, 8 June 2021

Mypy 0.900 Released

Third-party Library Stubs in Stub Packages (Breaking Change)

Python 2 Support is Opt-in (Breaking Change)

TypeGuard

Support for pyproject.toml

Binary Wheels for Apple Silicon

Type Narrowing with type(x)

Hosted Documentation for Mypyc

New Mypyc Features

Mypyc Performance Improvements

Other Mypyc Fixes and Improvements

Stubgen Improvements

Stubtest Improvements

Documentation Updates

Other Notable Fixes and Improvements

Acknowledgements

Contributors