Mypy 0.770 Released

We’ve just uploaded mypy 0.770 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Tagged Unions

Previously, the only way to distinguish between two types in a union was to use an isinstance check. Now, mypy supports distinguishing between two or more types based on the value of some common shared "tag" field.

This feature is most useful when you want to narrow down a union of TypedDicts: since TypedDicts are really just regular dicts at runtime, using isinstance checks won't work. Instead, you can now label or tag each TypedDict with a distinct Literal type and discriminate by checking that tag:

    from typing import Literal, TypedDict, Union
    
    class NewJob(TypedDict):
        event_type: Literal["new-job"]
        job_description: str
    
    class CancelJob(TypedDict):
        event_type: Literal["cancel-job"]
        job_id: int
    
    request: Union[NewJob, CancelJob]
    
    # We know request["event_type"] must be of type
    # Union[Literal["new-job"], Literal["cancel-job"]]
    if request["event_type"] == "new-job":
        # So if we narrow request["event_type"] down to type
        # Literal["new-job"], mypy will conclude 'request' must
        # therefore be of type 'NewJob' in this branch...
        print("Starting new job", request.job_description)
    else:
        # ...and inferred to be type 'CancelJob' in this one
        print("Cancelling job with id", request.job_id)

Note that you can import Literal from typing_extensions on Python 3.7 and earlier.

You can also use this technique to discriminate between unions of objects, tuples, or named tuples, so long as: 1. Every item in your union is tagged with a unique value 2. Mypy understands how to narrow the union of the tag values.

For more details and examples, see the documentation on tagged unions.

This feature was contributed by Michael Lee.

Type Inference Improvements

Type inference is now more powerful. Mypy can infer types in some cases where you previously needed an explicit type annotation.

Mypy can infer a built-in collection type from augmented assignment statements:

    x = []
    x += [1, 2]  # Infer type List[int] for x

Some dictionary construction idioms are better supported:

    d = {}  # No type annotation needed
    for k in things():
        if k not in d:
            d[k] = 0
        else:
            d[k] += 1

Mypy is better at inferring defaultdict types:

    from collections import defaultdict
    
    d = defaultdict(list)
    d['foo'].append(1)  # Infer type DefaultDict[str, List[int]] for d

Multiple Inheritance and isinstance() Checks

Previously, mypy assumed a variable cannot have two unrelated types when analyzing isinstance checks. For example, in the following example B is not a subclass of A, so mypy would incorrectly conclude the isinstance check is always false, and that the if-branch is unreachable — even though var could actually be a subtype of both types.

This meant that mypy skipped type checking anything inside the if statement, since it (by design) skips analyzing unreachable branches.

    class A: pass
    class B: pass
    
    class Ok(A, B): pass
    
    x: A = Ok()
    if isinstance(x, B):
        reveal_type(x)

Now, mypy will instead decide that the narrowed type of x is test.<subclass of "A" and "B">, instead of marking the branch as unreachable.

If it’s impossible for two types to be a subtype of one another, mypy will continue to not attempt to infer this "ad-hoc intersection":

    # Fails with a "TypeError: multiple bases have instance lay-out confict" error at runtime
    class Impossible(int, str): pass
    
    bad: int = Impossible()
    if instance(bad, str):
        # This branch is unreachable and so is not type checked.
        # If you want to be warned about unreachable branches, use
        # the --warn-unreachable flag.
        reveal_type(bad)

This feature was contributed by Michael Lee.

Other Notable Improvements and Bug Fixes

• Fix some crash bugs involving import * and import cycles (PR 8450)
• Fix interaction of descriptor methods with plugins (Jan Verbeek, PR 8365)
• Allow strict in config file (Ville Skyttä, PR 8192)
• Don’t crash when a module shadows builtin libraries (such as typing or types, PR 8405)
• Fix type join between subclasses of unicode and str (PR 8402)
• Fix type join of fixed-length tuples with mismatching lengths (Marti Raudsepp, PR 8333)
• Fix type join of Sequence (e.g. variadic tuple) and fixed-length tuple (Marti Raudsepp, PR 8335)
• Make mypy.api.run_dmypy actually capture the output (PR 8375)
• Support determining whether a literal is truthy (Jan Verbeek, PR 8368)
• Fix covariant overriding of decorated methods (Xuanda Yang, PR 8350)
• Support typing.Annotated in addition to typing_extensions.Annotated (Jakub Stasiak, PR 8371)
• Add add_method_to_class function to plugins.common (useful when writing mypy plugins) (Maksim Kurnikov, PR 8245)
• Fix module alias as instance attribute (Uwe L. Korn, PR 8259)
• Automatically write a file .gitignore file to cache directory, ignoring everything (Ville Skyttä , PR 8193)
• Don't make dunder attributes enum members (Xuanda Yang, PR 8302)
• Allow redefining TypedDict keys (while still generating an error) (Cohen Karnell, PR 8109)
• Fix some crashes in dataclasses (PR 8271)
• Use TypedDict union as type context when unambiguous (PattenR, PR 8212)
• Fix false positive for subclasses of bytes overriding __str__ (Shantanu, PR 8222)
• Fix deeply nested InitVar definitions in dataclasses with init=False (Jacob Beck, PR 8208)
• Narrow types for walrus assignment in if statements in some cases (Shantanu, PR 8258)
• Narrow types for walrus assignment in if statements in most of the rest of cases (PR 8458)
• Fix incorrect error code indexing (Xuanda Yang, PR 8248)
• Fix regression in container check logic (Michael Lee, PR 8232)
• Make reachability code understand chained comparisons (Michael Lee, PR 8148)
• Fix incorrect name lookup for decorated methods (Xuanda Yang, PR 8175)
• Fix simplifying unions with type aliases (Xuanda Yang, PR 8146)
• Fix crash when overriding __init__ in a dataclass subclass (Jacob Beck, PR 8159)
• Fix some daemon crashes involving classes becoming generic (PR 8157)

Documentation and Error Reporting Improvements

• Use fully qualified names in error messages for class names resembling builtins(Mukuntha N S, PR 8425)
• Improve diagnostics involving missing stubs for a library that is installed in site-packages (Michael Lee, PR 8238)
• Add a section to the documentation about incompatible overrides (RAHUL RAJA, PR 8377)
• Add variable-sized tuples to the cheat sheet (Marcio Mazza, PR 8364)
• Improve documentation of decorators (add decorator factories) (Marti Raudsepp, PR 8336)
• Update documentation of variables and aliases (Xuanda Yang, PR 8200)
• Report an error if a final class has abstract attributes (Denys Halenok, PR 8332)
• Update common issues to include __init__ without arguments (Tan Yuanhong, PR 8303)
• Add a new error code for errors about unreachability (Denys Halenok, PR 8312)
• Fix error indicator position in code with tabs (dosisod, PR 8307)
• Document --local-partial-types (Xuanda Yang, PR 8201)
• Update documentation for Literal types (Michael Lee, PR 8152)

Stubtest rewrite

stubtest is a tool that compares stub definitions to what it finds at runtime with introspection and reports back inconsistencies. It got a complete rewrite. Some features of the new stubtest are:

• Find missing, extraneous or mistyped classes, methods, functions and attributes in the stubs
• Check presence, names and kinds of function arguments, accounting for overloads, decorators, *args and kwargs. Checks argument types against their default values, accounting for type vars.
• Checks @property, @classmethod, @staticmethod declarations
• Check types of module level and class level attributes and enums

Some results of this:

• We now run stubtest in typeshed CI (for stdlib)
• We’ve fixed about 1900 issues in typeshed definitions
• We’ve greatly improved Python 3.8 support in typeshed and it’ll be easier to make the changes needed for future Python versions
• We’ve uncovered a handful of issues in Python itself

The stubtest rewrite was contributed by Shanatu.

Typeshed Updates

Many small improvements were made to typeshed — too many to list. Browse the typeshed commit log here.

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

• Anthony Sottile
• Cohen Karnell
• Denys Halenok
• dosisod
• Ethan Smith
• FooTaGe
• hauntsaninja
• HeShan
• Jacob Beck
• Jakub Stasiak
• Jan Verbeek
• Jérome Perrin
• lazytype
• Maksim Kurnikov
• Marcio Mazza
• Marti Raudsepp
• Michael Lee
• Mukuntha N S
• Nikita Sobolev
• PattenR
• RAHUL RAJA
• Shantanu
• Tan Yuanhong
• Uwe L. Korn
• Ville Skyttä
• Xuanda Yang

Additional thanks to all contributors to typeshed:

• abe
• Alan Du
• Alex Grönholm
• Alex Willmer
• Andrew Svetlov
• Anthony Sottile
• Artjoms Iskovs
• Batuhan Taşkaya
• Benjamin Peterson
• Brian Maissy
• Cal Paterson
• Christopher Dignam
• crusaderky
• Cyril Roelandt
• Daniel Däschle
• Daniel Farley
• Daniel Hahler
• Daniël van Eeden
• Dave Halter
• dave-shawley
• Erick
• Faidon Liambotis
• Fionn Fitzmaurice
• François Freitag
• Gisle Aas
• hauntsaninja
• Ian Good
• ijl
• Ilaï Deutel
• Jacob Beck
• Jacob Ilias Komissar
• Jakub Stasiak
• Jan Verbeek
• Jaromir Latal
• Jason
• Jason Gilholme
• Jelle Zijlstra
• Jens Hedegaard Nielsen
• Jeppe Fihl-Pearson
• layday
• lazytype
• Maarten ter Huurne
• Mark
• Michael Heyns
• Mickaël Schoentgen
• Niklas Fiekas
• Oleg Höfling
• Ophir LOJKINE
• Pavel Savchenko
• Rebecca Chen
• Reid Swan
• rikhilraithatha
• Romain
• Rune Tynan
• Sebastian Rittau
• Shantanu
• Squirrel
• Thomas Schaper
• tikki
• Vury Leo
• Wolf Honore