Monday, 26 September 2022

Mypy 0.981 Released

 

We’ve just uploaded mypy 0.981 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows: 
    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Support for Python 3.6 and 2 Dropped

Support for Python 2 has been completely removed in this release.

Support for Python 3.6 has also been dropped, since Python 3.6 has reached its end of life. It is still possible to target Python 3.6 using --python-version in future mypy versions, but no bugs will be fixed that affect only Python 3.6 (unless they are regressions). Also note that typeshed just recently dropped Python 3.6 support, so standard library features only available in Python 3.6 will not be supported.

The implementation was jointly contributed by Shantanu, Nikita Sobolev and Ivan Levkivskyi.

Generate Error on Unbound TypeVar Return Type

Code like this will now generate an error, since the value of T cannot be inferred in calls: 

    from typing import TypeVar
    
    T = TypeVar("T")
    
    # Error: A function returning TypeVar should receive at least one 
    # argument containing the same Typevar
    def f() -> T: ...

Having an unbound type variable in a nested position (e.g. list[T]) is still accepted in a return type, since it has valid use cases. This was contributed by Aurélien Jaquier (PR 13166).

Methods with Empty Bodies in Protocols Are Abstract

Any method with an empty body in a protocol is now correctly treated as implicitly abstract and must be implemented even if a class explicitly inherits the protocol: 

    from typing import Protocol
    
    class P(Protocol):
        # "..." is the literal ellipsis and indicates an empty body
        def meth(self) -> int: ...
    
    class Cls(P):
        pass
    
    # Error: Cannot instantiate abstract class "Cls" with abstract 
    # attribute "method"
    Cls()

This was contributed by Thomas MK (PR 12118).

Implicit Optional Types Will Be Disabled by Default

A future mypy feature release (possibly the next one after 0.98x) will disable implicit optional types such as these by default, and they will have to be explicitly enabled: 

    def foo(s: str = None) -> int: ...  No error currently
    
    foo(None)  # Ok

In the future mypy will only accept the example with an explicit optional type (e.g. str | None) for the s argument, when using the default options: 

    # s: Optional[str] also works
    def foo(s: str | None = None) -> int: ...
    
    foo(None)  # No change to callers

To prepare for the change, you can set the relevant configuration option explicitly. Either disable implicit optional types by using --no-implicit-optional, or no_implicit_optional = True in your mypy configuration file, or enable them via --implicit-optional or no_implicit_optional = False. We don’t recommend relying on implicit optional types, since they are a non-standard feature, but we have no plans to stop supporting them.

We hope to provide a migration tool that will automatically switch implicit optional types to explicit ones in function signatures.

Precise Types for **kwds Using TypedDict

Mypy now supports precise type annotations for kwds parameters. You can use this feature as following: 

    from typing import TypedDict
    from typing_extensions import Unpack
    
    class Style(TypedDict, total=False):
        margin: int
        sticky: bool
    
    def add_button(label: str, **kwds: Unpack[Style]) -> None:
        ...
    def add_check_box(active: bool, **kwds: Unpack[Style]) -> None:
        ...
    
    add_button("Cancel", margin=0, sticky=False)  # This works
    tight: Style = {"margin": 0}
    add_button("OK", **tight)  # This works as well

You can alternatively use Required[...] and NotRequired[...] TypedDict annotations to control whether a keyword argument is required or not. Note that arbitrary extra keywords are not allowed for such functions (this is consistent with how mypy currently handles TypedDicts). Although this feature is complete, Unpack[...] is still considered experimental, so you will need to use --enable-incomplete-features to enable it.

This feature was contributed by Ivan Levkivskyi (PR 13471).

Experimental Support for General Recursive Types

Until recently mypy only supported recursively defined classes (e.g. trees), but not arbitrary recursive types. In this version we add (experimental) support for arbitrary recursively defined types, including type aliases, TypedDicts, NamedTuples, and so on. This example uses a recursively defined type alias: 

    from typing import TypeVar, Sequence
    
    T = TypeVar("T")
    
    # String literal escaping needed to avoid NameError at runtime
    Nested = Sequence[T | "Nested[T]"]
    
    def flatten(seq: Nested[T]) -> list[T]:
        result = []
        for item in seq:
            if isinstance(item, list):
                result.extend(flatten(item))
            else:
                result.append(item)
        return result
    
    flatten([1, [2, 3]])  # This works, inferred type is list[int]

This feature can be enabled using --enable-recursive-aliases. If mypy doesn’t work as expected with some recursive types, you can try adding more explicit annotations, and/or using covariant collections (e.g. Sequence instead of list). Please file an issue on the issue tracker if you think you have found a bug.

List of relevant PRs:

  • Enable recursive type aliases behind a flag (Ivan Levkivskyi, PR 13297)
  • Simplify types in some error messages (Ivan Levkivskyi, PR 13326)
  • Support recursive TypedDicts (Ivan Levkivskyi, PR 13373)
  • Support recursive named tuples (Ivan Levkivskyi, PR 13371)
  • Fail gracefully on diverging recursive type aliases (Ivan Levkivskyi, PR 13352)
  • Fail gracefully on invalid and/or unsupported recursive type aliases (Ivan Levkivskyi, PR 13336)
  • Handle interactions between recursive aliases and recursive instances (Ivan Levkivskyi, PR 13328)
  • Fix crash on deferred upper bound with recursive aliases enabled (Ivan Levkivskyi, PR 13410)
  • Remove Optional[...] special-casing to avoid infinite recursion (Ivan Levkivskyi, PR 13357)

Generic NamedTuples and TypedDicts

Mypy now supports defining generic TypedDicts, NamedTuples, and user defined tuple types. You can use either the class or call syntax for this: 

    from typing import Generic, TypeVar
    from typing_extensions import TypedDict  # From "typing" on Python 3.11
    
    T = TypeVar("T")
    
    class Param(TypedDict, Generic[T]):
        name: str
        value: T
    
    Opt = TypedDict("Opt", {"value": T, "is_set": bool})
    
    def apply(param: Param[T]) -> T: ...
    
    x = apply({"name": "test", "value": 42})  # Inferred type is "int"
Note: While you can freely use this feature in stub files, if you want to use it in source files, you will need to either use Python 3.11 (which is not officially supported by mypy yet), import TypedDict/NamedTuple from typing_extensions, or use an if TYPE_CHECKING: ... block.

List of relevant PRs:

  • Enable generic TypedDicts (Ivan Levkivskyi, PR 13389)
  • Enable generic NamedTuples (Ivan Levkivskyi, PR 13396)

Better Support for Callable Attributes

Mypy now has better support for instance attributes with Callable[...] types. Previously, mypy couldn’t reliably distinguish between callable instance attributes and method aliases. Now there are consistent rules. If a variable is defined using and assignment and/or annotation in a class body and has a callable type, it is an instance attribute if both these conditions are true:

  • There is an explicit annotation, and it is not a ClassVar[...].
  • The variable name is not a special dunder method name, such as __add__.

Conversely, such an assignment/annotation defines a method in these cases:

  • There is no explicit annotation, or there is a ClassVar[...] annotation.
  • The variable name is a special dunder method name.

Example: 

    from typing import ClassVar, Callable
    
    class Test:
        runner: Callable[[], None]  # An instance variable
    
        def hook(self) -> None: ...
            self.runner = lambda: print("pass")
    
        legacy_hook = hook  # A method
        other_hook: ClassVar[Callable[[Test], None]]  # Also a method
    
    Test.other_hook = lambda self: None
    t = Test()
    t.legacy_hook()
    t.runner()
    t.other_hook()

This feature was contribute by Ivan Levkivskyi (PR 13400).

Per-Module Error Code Configuration

You can now disable and/or enable individual error codes per module and per directory using config files and/or inline # mypy: comments. Example: 

    # mypy.ini
    [mypy]
    strict = True
    
    [mypy-tests.*]
    allow_untyped_defs = True
    allow_untyped_calls = True
    disable_error_code = var-annotated

    # source.py
    x = []  # Error: Need type annotation for "x"

    # tests/foo.py
    x = []  # OK

    # tests/bar.py
    # mypy: enable-error-code="var-annotated"
    x = []  # Error: Need type annotation for "x"

Note that these per-directory settings act as incremental overrides. The precedence is the following:

  • Command line options and/or the main section of a config file set global codes.
  • Config sections adjust them per glob/module.
  • Inline # mypy: comments adjust them again.

You can, for example, enable an error code globally, disable it for all tests in the config file, and then re-enable it in a particular file by using an inline comment.

This feature was contributed by Ivan Levkivskyi (PR 13502).

Experimental Support for Interactive Inspection of Expressions

Mypy now has an experimental support for various static inspections using the mypy daemon dmypy inspect command. These inspections are currently included:

  • Print the type of an expression at given location.
  • Print the attributes defined for a given expression.
  • Go to definition of a reference expression (name or attribute).

Note that you can run this only after the file has been type-checked by the daemon, for example by using dmypy check or dmypy run. Basic usage is like this: 

    dmypy inspect [--show type|attrs|definition] \
        path/to/file.py:line:column[:end_line:end_column]

Please read the docs for more details. This feature was contributed by Ivan Levkivskyi (PR 13209).

Mypyc Improvements

Mypyc compiles Python modules to C extensions and is bundled with mypy. It uses standard Python type hints to generate fast code (documentation).

  • Support async for as a statement and in comprehensions (Michael J. Sullivan, PR 13444)
  • Support async with (Michael J. Sullivan, PR 13442)
  • Fix clang warning on different signs integer (Paul m. p. Peny, PR 13239)
  • Fix AttributeError message (Jukka Lehtosalo, PR 13382)
  • Fix __call__ subclasses (Ken Jin, PR 13152)
  • Fix setup conflict with attributes named up (davfsa, PR 13012)
  • Fix bad C generated for multiple assignment (Jukka Lehtosalo, PR 13147)
  • Update, simplify check version test (Shantanu, PR 13125)

Documentation Updates

  • Update theme to furo (Michael Wentz, PR 13345)
  • Use ParamSpec in "Declaring decorators" (Ilya Konstantinov, PR 13237)
  • Fix protocols documentation (Zsolt Cserna, PR 13137)
  • Prefer f-strings over format and underscores in numeric literals in docs (Kevin Kirsche, PR 13317)
  • Use GitHub's zipball as alternative to 'git+https' in README.md (Peter Badida, PR 13208)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Allow passing arguments to stubgen.main (Matthew Woehlke, PR 13095)
  • Increase import timeout (Matthew Woehlke, PR 13109)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Fallback to getattr_static on AttributeError (Alex Waygood, PR 13285)
  • Reduce false positives on runtime type aliases (Alex Waygood, PR 13116)
  • Catch-all for errors from the runtime (Shantanu, PR 13160)
  • Allow stubtest to ignore Python 2 only magic methods (Nikita Sobolev, PR 13290)
  • Improve docs (Shantanu, PR 13293)
  • Ignore __pyx_vtable__ (Shantanu, PR 13302)
  • Improve signature checking (Shantanu, PR 13307)
  • Show path to stub file (Shantanu, PR 13342)
  • Allow stubtest to raise errors on abstract state mismatch (Nikita Sobolev, PR 13323)
  • Fix broken error message for async/sync mismatch (Alex Waygood, PR 13414)
  • Don't error for a missing submodule if the submodule name is private (Alex Waygood, PR 13417)
  • Ignore __vectorcalloffset__ (Alex Waygood, PR 13416)
  • Use single line error message (Shantanu, PR 13301)
  • Verify the contents of __all__ in a stub (Alex Waygood, PR 12214)
  • Fix error for Protocol.__init__ and __annotations__ (Alex Waygood, PR 13179)
  • Fix custom_typeshed_dir regression (PR 13658, PR 13656) (Shantanu)

Miscellaneous New Features

  • Enable support for decorated properties where possible (Ivan Levkivskyi, PR 13409)
  • Make any callable compatible with (*args: Any, kwargs: Any) (Shantanu, PR 11203)
  • Allow generic decorators on abstract classes (Ivan Levkivskyi, PR 13398)
  • Print and show error end locations (Ivan Levkivskyi, PR 13148)
  • Basic support for typing_extensions.NamedTuple (Alex Waygood, PR 13178)
  • Union types: Support narrowing to Ellipsis (...) cases of Unions (László Vaskó, PR 13157)

Other Notable Fixes and Improvements

  • Allow unpacking from TypeVars with iterable bounds (PR 13644, PR 13425) (Shantanu)
  • Fix error codes option serialization (Ivan Levkivskyi, PR 13523)
  • Fix daemon crashes related to ParamSpec and TypeVarTuple (Jukka Lehtosalo, PR 13381)
  • Allow stubs to use newer syntax than 3.7 (Shantanu, PR 13500)
  • Fix --custom-typeshed-dir handling (Ivan Levkivskyi, PR 13629)
  • Remove blocking special cased error for bool subclass (wookie184, PR 13420)
  • Remove useless parameter script_path of mypy.main() and make arguments keyword-only (Jingchen Ye, PR 13399)
  • Fix type inference for tuples in iterable context (Ivan Levkivskyi, PR 13406)
  • Fix type narrowing of TypedDict value with key name in final variable (Jingchen Ye, PR 11813)
  • Fix crash from invalid location in aliased types (Michael Krasnyk, PR 12745)
  • Advertise typing status via Trove classifier (Mike Fiedler, PR 13350)
  • Use a better colour for gray in mypy output (Ivan Levkivskyi, PR 13338)
  • Support type aliases in metaclasses (Nikita Sobolev, PR 13335)
  • Handle files ending with __init__ better (Shantanu, PR 13314)
  • Respect tuple.__getitem__ from typeshed (Shantanu, PR 13313)
  • Make subtype checks more consistent in edge cases (Ivan Levkivskyi, PR 13303)
  • Fix --custom-typeshed-dir crash (Alex Waygood, PR 13296)
  • Ensure builtin modules are from typeshed sooner (Konstantin, PR 13155)
  • Disallow bytes in TypeVar, NewType, and TypedDict names (Jelle Zijlstra, PR 13273)
  • Prohibit bytes literals in named tuples (Ivan Levkivskyi, PR 13271)
  • Suggest using a newer Python version if possibly needed (Shantanu, PR 13197)
  • Check implicit None return is valid when using --no-warn-no-return (Shantanu, PR 13219)
  • Fix site package on MYPYPATH check (Shantanu, PR 13223)
  • Make None compatible with SupportsStr protocol (Nikita Sobolev, PR 13184)
  • Fix caching of PEP 561 namespace packages (Shantanu, PR 13124)
  • Improve error message for direct __init__ calls (Shantanu, PR 13183)
  • Allow nonlocal in function scoped classes (Shantanu, PR 13158)
  • Support for Python 3.11's "safe_path" in pyinfo (Shantanu, PR 13164)
  • Generate error when using NoneType instead of None as a type (Jakub Strnad, PR 13153)
  • Add musllinux wheel support, Use 'official' llvm build for compilation (Ben Raz, PR 13228)
  • Skip musllinux wheels, update for lxml changes (Shantanu, PR 13129)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • anilbey
  • Aurélien Jaquier
  • Ben Raz
  • davfsa
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • Jakub Strnad
  • Jared Hance
  • Jelle Zijlstra
  • Jingchen Ye
  • Jukka Lehtosalo
  • Ken Jin
  • Kevin Kirsche
  • Konstantin
  • KotlinIsland
  • László Vaskó
  • Marc Mueller
  • Matthew Woehlke
  • Michael Krasnyk
  • Michael Sullivan
  • Michael Wentz
  • Mike Fiedler
  • Nikita Sobolev
  • Patryk Gałczyński
  • Paul m. p. Peny
  • Peter Badida
  • Richard Si
  • Shantanu
  • Thomas MK
  • wookie184
  • Zsolt Cserna
  • Zsolt Dollenstein

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Posted by at