We’ve just uploaded mypy 0.530 to the Python Package Index (PyPI). This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the documentation for this release on ReadTheDocs.
Major New Features
We present several new features in this release: Protocols, TypedDict, Per-file Control of Strict Optional Checking and Module Aliases. The features are described below.
Protocols (structural subtyping, a.k.a. static duck typing)
Protocols are a new, experimental feature, intended to better support duck typing. Protocols are on a standardization track (PEP 544), but we are still working on the details of how protocols work. They are not yet included in the standard library typing module. You can import them from the new typing_extensions package instead (not to be confused with mypy_extensions).
To use protocols, first install the package typing_extensions:
python3 -m pip install -U typing_extensions
Once this is installed you can start defining protocols by inheriting from typing_extensions.Protocol. Here’s a simple example:
from typing import Iterable
from typing_extensions import Protocol
class SupportsClose(Protocol):
def close(self) -> None: ... # Use a literal '...' here
class Resource: # No SupportsClose base class!
# ... some methods ...
def close(self) -> None:
self.resource.release()
def close_all(items: Iterable[SupportsClose]) -> None:
for item in items:
item.close()
close_all([Resource(), open('some/file')]) # Okay!
In this example, Resource is compatible with the SupportClose protocol since it defines a compatible close() method. Regular file objects returned by open() are similarly compatible with the protocol, as they support close().
Protocols were implemented by Ivan Levkivskyi. Here’s the documentation.
TypedDict
TypedDict has been an undocumented feature for a while; it’s now officially supported. It’s still experimental, but we think that it’s robust enough to invite everyone to kick the tires.
Python programs often use dictionaries with string keys to represent objects. Here is a typical example:
movie = {'name': 'Blade Runner', 'year': 1982}
You can now define a TypedDict to give a precise type for objects like movie, where the type of each dictionary value depends on the key:
from mypy_extensions import TypedDict
Movie = TypedDict('Movie', {'name': str, 'year': int})
favorite = {'name': 'Blade Runner', 'year': 1982} # type: Movie
Movie is a TypedDict type with two items: 'name' (with type str) and 'year' (with type int). Note that we used an explicit type annotation for the favorite variable. This type annotation is important -- without it, mypy will try to infer a regular, uniform Dict type for favorite, which is not what we want here.
By default mypy ensures that a TypedDict object has exactly all the specified keys. You can provide the total=False argument to TypedDict(...) to allow keys to be left out when creating a TypedDict object:
GuiOptions = TypedDict(
'GuiOptions', {'language': str, 'color': str}, total=False)
options = {} # type: GuiOptions # Okay
options['language'] = 'en'
Python 3.6 also supports an alternative, class-based syntax to define a TypedDict:
from mypy_extensions import TypedDict
class Movie(TypedDict):
name: str
year: int
This syntax also supports the total=False flag, as follows:
class GuiOptions(TypeDict, total=False):
...
Several people contributed to TypedDict, including David Foster (original implementation), Ivan Levkivskyi, Roy Williams, Max Moroz and Jukka Lehtosalo. Here’s the documentation.
Per-file Control of Strict Optional Checking
Mypy has supported the option --strict-optional for a while that causes mypy to detect errors due to None values. However, many users still haven’t switched to using this option as it can require a bit of refactoring to adopt, such as adding Optional[…] to annotations. Now mypy makes this easier as you can enable strict optional checking on a per-file basis, allowing you to gradually migrate your codebase to strict optional checking, while ensuring that previously migrated modules remain strict optional clean.
You can control this through a mypy config file. Example:
; Enable strict optional checking in acme.util
[mypy-acme.util]
strict_optional = True
This was implemented by David Fisher (PR 3206). Most of the work was actually completed for the 0.520/0.521 release but we withheld the announcement because we hadn’t had enough opportunity to test it.
Module Aliases
Mypy now can handle aliases for modules. Example:
import subprocess
sp = subprocess
sp.check_call(['whoami']) # Okay
This was contributed by Carl Meyer, with help from Ivan Levkivskyi.
More New Features
- Support module-level __getattr__() in stubs (Jelle Zijlstra, PR 3647; Ethan Smith, PR 3779)
- Generate note if reveal_type() is used in an unchecked function (Svyatoslav Ilinskiy, PR 3743)
- Add flag --disallow-incomplete-defs to disallow partially annotated functions (Svyatoslav Ilinskiy, PR 3744)
- Add flag --disallow-untyped-decorators to output error if a typed function is used with an untyped decorator (Svyatoslav Ilinskiy, PR 3555)
- Tag Any types with the kind of their origin (Svyatoslav Ilinskiy, PR 3786)
- Make disallow_subclassing_any a per-module option (Daniel Watkins, PR 3801)
- Report sources of Any in html-report per-line as hovertext (Svyatoslav Ilinskiy, PR 3798)
- Add --warn-unused-configs flag (PR 3865)
- Support six.add_metaclass() (Elazar Gershuni, PR 3842)
- Type check arguments for super() (PR 3919)
- Allow creating type aliases for None (Ivan Levkivskyi, PR 3754)
Stubgen Improvements
- Add -o <dir> option (Brian Bruggeman, PR 3902)
- Preserve existing annotations (Daniel F Moisset, PR 3169)
Notable Bugs Fixed
- Fix crashes and failures caused by forward references to types (Ivan Levkivskyi, PR 3952)
- Make configuration file section ordering consistent on Python 3.5 and earlier (PR 3678)
- Re-broaden variable type to Any on assignment if it started as Any but was narrowed in an enclosing frame (lincolnq, PR 3361)
- Suppress errors found in unreachable parts of a Boolean expression (Roy Williams, PR 3666)
- Give a useful message if we can't infer a TypeVar-based type for attribute with value restriction (PR 3651)
- Support docstrings in a class-based TypedDict (Dominik Miedziński, PR 3749)
- Do not follow imports from stubs with --follow-imports=skip (Svyatoslav Ilinskiy, PR 3738)
- Support exported names starting with an underscore in __all__ (David Ross, PR 3746)
- Search lib directory suffixed by ABI (32 or 64), which is used in at least Gentoo Linux (Shunsuke Shimizu, PR 3758)
- Fix crash with unanalyzed branches and old-html report (Svyatoslav Ilinskiy, PR 3782)
- Hide imported names in stubs unless as id is used (to align with PEP 484) (Ivan Levkivskyi, PR 3706)
- Use correct configuration file path when searching for plugins (Tin Tvrtković, PR 3770)
- Use types of actual arguments to infer lambda argument types if callee is a lambda (Elazar Gershuni, PR 3116)
- Don't supply a type context to the expression in yield from (PR 3815)
- Erase type variables of generic types when used in Type[C] (Daniel Watkins, PR 3833)
- Fix using stale metadata in incremental mode when errors are no longer ignored (Mark Hahnenberg, PR 3793)
- Infer better return type for some calls to overloaded functions with Any actual arguments (PR 3917)
- Fix problem with overload resolution when there are union types in signatures (PR 3300)
- Fix type checking decorated method overrides when decorator changes signature (PR 3918)
- Fix crash caused by attempted serialization of a partial type (PR 3954)
- Support user install on Windows (Ethan Smith, PR 4005)
- Fix crashes related to partial types (Ivan Levkivskyi, PR 3995)
- Don't crash on unexpected PY2 except syntax (PR 4011)
Dependencies Etc.
- This mypy release requires typed-ast version 1.1.0 or higher. Since mypy 0.52[01] required a version less than 1.1.0, this is a mandatory upgrade. The main reason for the upgrade is that the new version supports Python 3.3 and 3.4 on Windows, which required some API changes.
- This is the last mypy release with official support for running on Python 3.3, which reached its End Of Life last week. Note that future mypy versions will still support checking older Python versions — they just won’t run on 3.3 or earlier. It’s easy to have multiple Python versions installed, so this won’t prevent anyone from running mypy.
Other Changes
- Many more mypy unit tests are now run via pytest (but the runtests.py script is still not dead yet).
- Almost all of the mypy implementation is now type checked using strict optional checking (Ivan Levkiskyi).
- Improvements to error messages (Dominik Miedziński, PR 3515; Elazar Gershuni, PR 3783; Svyatoslav Ilinskiy, PR 3707; Jesse Lord, PR 3411; Ethan Smith, PR 3862; Daniel Watkins, PR 3873, PR 3893, PR 3898, and PR 3910).
Acknowledgments
First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.
Thanks to all mypy contributors who contributed to this release:
- Brian Bruggeman
- Carl Meyer
- Chen Rotem Levy
- Daniel F Moisset
- Daniel Watkins
- David Ross
- Dominik Miedziński
- Edward Betts
- Elazar Gershuni
- Ethan Smith
- Greg Price
- Ivan Levkivskyi
- Jelle Zijlstra
- Jesse Lord
- lincolnq
- Mark Hahnenberg
- Max Moroz
- Michael R. Crusoe
- Roy Williams
- Shunsuke Shimizu
- Svyatoslav Ilinskiy
- Thomas Grainger
- Tin Tvrtković
- Tobin Yehle
Additional thanks to all contributors to typeshed:
- Adam Dangoor
- Alex Ellis
- Alex Grönholm
- Amandine Lee
- Artur Rodrigues
- Ashwini Chaudhary
- Ben Leslie
- Brian C. Lane
- Bruce Merry
- Chris Kerr
- Daniel Watkins
- David Novakovic
- Dominik Gabi
- Elazar Gershuni
- Ethan Smith
- Evan Krall
- FichteFoll
- Garrett
- Ivan Levkivskyi
- Jaen
- Jelle Zijlstra
- John Reese
- Jonathan Schoonhoven
- Liran Nuna
- Luka Sterbic
- Martin DeMello
- Matthias Kramm
- Melvyn Sopacua
- Michael Lee
- Mikołaj Biel
- Neil Pilgrim
- Øystein Olsen
- Pavel Pletenev
- Pieter-Jan Briers
- Robert T. McGibbon
- Semyon Proshev
- Svyatoslav Ilinskiy
- Thomas Grainger
- Tim Crawford
- Zack Hsi
- aubanel
- blokeley
- daid
- eujing
- rchen152
- toejough
- vim345
