We’ve just uploaded mypy 0.510 to PyPI. This release adds new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install --upgrade mypy
Note: Remember that the package name is now mypy (no longer mypy-lang).
Update (10 May): Released mypy 0.511 that fixes two crash bugs but is otherwise identical to 0.510.
Overloads in Source Files
Previously the @overload decorator was only supported in stub files. Now you can use it in regular source files as well. First write some @overload declarations that define the different signatures your function supports. These should have empty bodies and are only used by mypy. Put the actual runtime definition of the function after these declarations, without a decorator. Here’s an example from the mypy documentation:
from typing import overload, Sequence, TypeVar, Union
T = TypeVar('T')
class MyList(Sequence[T]):
# The @overload definitions are just for the type checker,
# and overwritten by the real implementation below.
@overload
def __getitem__(self, index: int) -> T:
pass # Don't put code here
# All overloads and the implementation must be adjacent
# in the source file, and overload order may matter:
# when two overloads may overlap, the more specific one
# should come first.
@overload
def __getitem__(self, index: slice) -> Sequence[T]:
pass # Don't put code here
# The implementation goes last, without @overload.
# It may or may not have type hints; if it does,
# these are checked against the overload definitions
# as well as against the implementation body.
def __getitem__(self, index):
# This is exactly the same as before.
if isinstance(index, int):
... # Return a T here
elif isinstance(index, slice):
... # Return a sequence of Ts here
else:
raise TypeError(...)
This was contributed by Naomi Seyfer.
Extended Callable Types
As an experimental mypy extension, you can specify Callable types that support keyword arguments, optional arguments, and more. Where you specify the arguments of a Callable, you can choose to supply just the type of a nameless positional argument, or an "argument specifier" representing a more complicated form of argument, using helpers such as Arg and DefaultArg that are defined in the mypy_extensions module. This allows one to more closely emulate the full range of possibilities given by Python’s def statement.
As an example, here's a complicated function definition and the corresponding Callable:
from typing import Callable
from mypy_extensions import (Arg, DefaultArg, NamedArg,
DefaultNamedArg, VarArg, KwArg)
def func(__a: int, # This convention is for nameless arguments
b: int,
c: int = 0,
*args: int,
d: int,
e: int = 0,
**kwargs: int) -> int:
...
F = Callable[[int, # Or Arg(int); an argument without a name
Arg(int, 'b'), # Argument with name and type
DefaultArg(int, 'c'), # Argument that may be omitted
VarArg(int), # *args argument
NamedArg(int, 'd'), # Must be passed as a keyword argument
DefaultNamedArg(int, 'e'), # Like above, but may be omitted
KwArg(int)], # **kwargs argument
int] # Return type
f: F = func
To use this, you need to install mypy_extensions:
python3 -m pip install --upgrade mypy_extensions
This was contributed by Naomi Seyfer.
ClassVar
You can mark names intended to be used as class variables with ClassVar. In a pinch you can also use ClassVar in # type comments. This is supported by the typing module that ships with Python 3.6 (it is part of PEP 526). Example:
from typing import ClassVar
class C:
x: int # instance variable
y: ClassVar[int] # class variable
z = None # type: ClassVar[int]
def foo(self) -> None:
self.x = 0 # OK
self.y = 0 # Error: Cannot assign to class variable "y" via instance
C.y = 0 # This is OK
This was contributed by Dominik Miedziński.
Quick Mode
Run mypy with the --quick-and-dirty (or just --quick) option to try a new experimental, unsafe variant of incremental mode. Quick mode is faster than regular incremental mode, because it only re-checks modules that were modified since their cache file was last written; regular incremental mode also re-checks all modules that depend on one or more modules that were re-checked.
Quick mode is unsafe because it may miss problems caused by a change in a dependency, so you should not use it in a continuous integration build, and you should perhaps run mypy without quick mode before landing commits. Quick mode updates the cache, but regular incremental mode ignores cache files written by quick mode.
Functional API for Enum
Mypy now also supports the functional API for defining an Enum. Previously you had to define a class deriving from Enum. Example (other forms are also supported):
Color = Enum('Color', 'red blue green')
Color.red # Ok
Color.black # "Color" has no attribute "black"
Improvements to None Handling
Mypy now treats the None type more consistently between the default checking mode and the strict optional checking mode. Unlike previously, None is largely treated as a regular type, and things like List[None] mean what you’d expect: a list containing None values. Additionally, this idiom no longer generates an error:
def f() -> None: ...
def g() -> None:
return f() # Okay
Generators and Async Comprehensions (PEP 525 and PEP 530)
Python 3.6 allows coroutines defined with async def to be generators — they may contain yield statements and expressions. It also introduces a syntax for asynchronous comprehensions. Mypy now supports these features. Here’s an example from the mypy documentation:
from typing import AsyncIterator
async def gen() -> AsyncIterator[bytes]:
lst = [b async for b in gen()] # Inferred type is "List[bytes]"
yield 'no way' # Error: Incompatible types (got "str", expected "bytes")
This was contributed by Jelle Zijlstra.
Functions Returning Generic Functions
Mypy now understands that a function may return a generic function. A typical use case is a function that returns a decorator:
from typing import TypeVar, Callable, Any, cast
T = TypeVar('T', bound=Callable[..., Any])
def logged(description: str) -> Callable[[T], T]:
def decorator(f: T) -> T:
def wrapper(*args, **kwargs):
print('entering:', description)
value = f(*args, **kwargs)
print('leaving:', description)
return cast(T, wrapper)
return decorator
@logged('system initialization')
def init() -> None:
print('do something')
init(1) # Too many arguments (signature is correctly preserved)
This was contributed by Naomi Seyfer.
Don’t Simplify Unions Containing Any
Unions containing Any, such as Union[int, Any], are now handled consistently. Previously in some cases mypy would simplify these into just Any, which is not equivalent. This could result in mypy failing to generate errors for buggy code. All operations performed on a union-typed value much be valid for every union item. For example, adding a string to a Union[int, Any] value is an error, since you can’t add a string and an integer. If the union was simplified to Any, mypy would not detect this error, since anything can be added to Any.
Prohibit Parametrizing Built-in Types
You no longer can write list[int] (with lower case l) in type annotations, as this doesn’t work at runtime. The also applies to dict and other types that have capitalized aliases in typing. Use List, Dict and so on (defined in typing) in type annotations. Note that this may require many changes to existing type annotations.
This was contributed by Ivan Levkivskyi.
More New Features
- Support arbitrary member access on Type[Any] so that it’s possible to access class methods and attributes. (Daniel F Moisset)
- Allow keyword arguments in __call__. (Ivan Levkivskyi)
- Infer types from issubclass() calls, similar to isinstance(). (Max Moroz)
- Always write the cache for incremental checking (unless cache_dir is /dev/null).
- Use types.ModuleType instead of module, since the latter doesn’t exist at runtime. (Max Moroz)
- Make it an error to use a class-attribute type variable outside a type. (Naomi Seyfer)
- Allow importing NoReturn from typing. (Ivan Levkivskyi)
- Reject TypeVar with only one constraint, since these never make sense. (Michael Lee)
- Allow instantiation of Type[A], if A is abstract. (Ivan Levkivskyi)
- Allow unions and nested tuples in except handlers. (Jelle Zijlstra)
- Treat type as equivalent to Type[Any]. (Ivan Levkivskyi)
- Make Type[T] compatible with Callable[..., T]. (Dominik Miedziński)
- Always use the default Python 3 version to parse stub files, so that Python 3.6 features can be used in stub files.
- Support default values for items in NamedTuple class definitions. (Jelle Zijlstra)
- Reject generic types as the second argument to isinstance or issubclass, since they don’t work at runtime. (Max Moroz)
- Allow variable as a type in isinstance. (Max Moroz)
- Support Type[x] and type with isinstance. (Max Moroz)
- Allow isinstance/issubclass with nested tuples. (Max Moroz)
- Improve type inference of containers containing callables or overloads. (Ivan Levkivskyi)
- Add support for type aliases typing.ChainMap, typing.Counter, typing.DefaultDict and typing.Deque. (Jelle Zijlstra)
- Improvements to TypedDict (this feature is still incomplete and undocumented). (Roy Williams, Max Moroz and Ivan Levkivskyi)
Removed or Deprecated Features
- The old parser that was available through --no-fast-parser is no longer supported.
- Deprecate --strict-boolean and don’t enable it with --strict. (Max Moroz)
- Drop Python 3.2 as a type checking target, since it has reached its end of life.
- The stubs for sqlalchemy were dropped since they were incomplete and caused problems with common idioms. The stubs now live in a separate repository.
Notable Bugs Fixed
- Fixes to various crashes. (Ivan Levkivskyi, Max Moroz and Jelle Zijlstra)
- Fixes to incremental mode.
- Fix bugs related to narrowing down unions using runtime checks.
- Fix unions containing both bool and float.
- Make union simplification be independent of the order of union items.
- Fixes to confusing error messages due to dict displays. (Daniel F Moisset)
- Fix subtyping for type variables whose upper bound is a union. (Max Moroz)
- Fix invalid complaints about f-string expressions. (Ingmar Steen)
- Fix isinstance check with nested unions. (Ivan Levkivskyi)
- Work around random failures on writing cache files on Windows.
- Fixes to % formatting. (Jelle Zijlstra)
- Fixes to nested classes. (Elazar Gershuni)
- Disallow generic Enums as they don’t work. (Jelle Zijlstra)
Other Changes
- Require version 1.0.3 of typed-ast to run mypy.
- Mypy developers on Windows must run certain git commands as Administrator. This only affects the mypy and typeshed git repositories.
- Add stubtest, a tool for comparing a stub file against the implementation. (Jared Garst)
- Speed up incremental mode. (Max Moroz)
- Many updates to the library stubs in typeshed.
A few highlights (there are too many contributors to list them all here; see below for a full contributor list):
- Many new stubs.
- Many stubs that had separate Python 2 and Python 3 variants were merged into shared Python 2+3 stubs to improve consistency and to simplify maintenance.
Acknowledgments
First of all, I’d like to thank my employer Dropbox for funding the mypy core team.
Thanks to all mypy contributors who contributed to this release:
- Ben Kuhn
- Daniel F Moisset
- Dominik Miedziński
- Elazar Gershuni
- Ethan
- Garrett
- Ingmar Steen
- Ivan Levkivskyi
- Jared Garst
- Jelle Zijlstra
- Łukasz Langa
- Max Moroz
- Michael Lee
- Naomi Seyfer
- Ryan Gonzalez
- Tobin Yehle
Additional thanks to all contributors to typeshed. In particular, I’d like to single out Jelle Zijlstra and David Euresti who have done a huge amount work on improving typeshed. Here’s a list of all typeshed contributors who contributed to this release:
- Alvaro Caceres
- Andrey Vlasovskikh
- Antoine Reversat
- Ashwini Chaudhary
- Bertrand Bonnefoy-Claudet
- Carl Meyer
- Cooper Lees
- David Euresti
- David Wetterau
- Dominik Miedziński
- Eddie Antonio Santos
- Ethan
- George King
- Günther Noack
- Hong Minhee
- Ivan Levkivskyi
- James Saryerwinnie
- Jan Hermann
- Jelle Zijlstra
- Jeremy Apthorp
- jkleint
- John Reese
- Josiah Boning
- Luka Sterbic
- Łukasz Langa
- Manuel Krebber
- Marcin Kurczewski
- Martijn Pieters
- Matthias Kramm
- Max Moroz
- Michael Walter
- Michał Masłowski
- Naomi Seyfer
- Nathan Henrie
- Nikhil Marathe
- nimin98
- rchen152
- Richard Hansen
- Roy Williams
- Sam Dunster
- Samuel Colvin
- Sebastian Meßmer
- Semyon Proshev
- Shengpei Zhang
- Stefan Urbanek
- Tadeu Manoel
- Teddy Sudol
- Thomas Ballinger
- Tim Abbott
— Jukka (on behalf of the rest of the mypy team: Guido, David and Greg)
