I just uploaded mypy-lang 0.3 to PyPI! This release focuses on further PEP 484 compatibility and Python 2 compatibility, but it also contains many general improvements. The changes are too numerous to enumerate them all. Instead, I'll focus here on the most visible and useful ones.
I'm also happy to announce new mypy core developers Guido, David Fisher and Greg Price, who have joined the project thanks to support from our employer, Dropbox. This has given mypy development a big boost, and mypy is currently under very active development.
Mypy 0.3 breaks backward compatibility. You may have to modify your code to be PEP 484 compatible if you have been using mypy-lang from PyPI. As before, mypy is experimental and still has known bugs and limitations. In particular, we still don't have library stubs for many commonly used modules. Please report any new issues you encounter -- we try to respond quickly to all user requests, and bugs are often fixed rapidly.
Feature Highlights
- Python 2 support -- now basically all Python 2.7 syntax is supported in Python 2 mode. Mypy also supports a comment-based function annotation syntax for Python 2 that was added to PEP 484 recently. It also works for code that aims to be both Python 2 and 3 compatible. More about this below.
- Improved PEP 484 support. Mypy now supports most PEP 484 features (not yet, however, sys.version_info checks), and several non-standard features have been removed.
- Partial Python 3.5 support (though async/await are sadly still not supported).
- New experimental type checking modes (--silent-imports and --implicit-any command-line options) that make it much easier to type check subsets of a larger code base (see discussion of developer workflow improvements below).
- Improved type inference -- fewer variable annotations are required, and mypy is much less likely to give the annoying "Cannot determine type of x" errors.
- Many bug fixes and better Python compatibility.
- Numerous standard library stub improvements (via typeshed, see below).
GitHub Changes
- The mypy github repository has moved to the Python organization. The new URL is https://github.com/python/mypy.
- Library stubs now live in a separate repo called typeshed (https://github.com/python/typeshed). The idea of the repository is make stubs a shared resource between all Python type checking or inference related projects.
Python 2 Support
Mypy now has experimental Python 2.7 support! For code that needs to be Python 2 compatible, function type annotations are given in comments, since the function annotation syntax was introduced in Python 3. Mypy has had partial, unofficial Python 2 support for a while, but only now it is actually usable and standard (after a recent PEP 484 update).
Run mypy in Python 2 mode by using the --py2 option:
$ mypy --py2 program.py
To run your program, you must have the typing module in your module search path. Use pip install typing to install the module (it also works for Python 3).
Python 2 support was a lot of work and here is a summary of some of the most obvious changes:
- Support pretty much all of Python 2 only syntax, including the print statement and the except Ex, err: syntax.
- Provide Python 2 stubs for many library modules (now in typeshed, as mentioned earlier.)
- Better Python 2 implementation of typing.
- Document Python 2 support.
- Make stub generator support generating Python 2 stubs (the stub generator is still highly experimental and not at all well documented, but it has already been pretty useful anyway.)
The example below illustrates Python 2 function type annotation syntax. This is also valid in Python 3 mode:
from typing import List
items = [] # type: List[str]
def hello(): # type: () -> None
print 'hello'
class Example:
def method(self, lst, opt=0, *args, **kwargs):
# type: (List[str], int, *str, **bool) -> int
...
Here are some things to note:
- Type syntax for variables is the same as Python 3.
- Function annotations are in comments and look quite a bit like Python 3 annotations.
- The module typing is available in Python 2 mode and things like Any must be imported from it, even if they are only used in comments.
- You don't provide an annotation for the self/cls variable of methods.
- The annotation can be on the same line as the function header or on the following line.
- You don't need to use string literal escapes for forward references within comments.
Better PEP 484 Compatibility
Several changes improve PEP 484 compatibility. Mypy still doesn't implement quite everything in PEP 484, but we are pretty close.
Undefined Removed
As the final version of PEP 484 doesn't include Undefined, mypy no longer supports it. It's easy to migrate old code to not use Undefined -- just use type comments. For example, refactor this
from typing import Undefined
x = Undefined(int)
to this:
x = None # type: int
Support for Callable[..., t]
Mypy supports callable types with unspecified argument types. Just replace the arguments with an explicit ...:
from typing import Callable
def call_a_lot(func: Callable[..., int]) -> None:
kwargs = {'arg': 1}
func(1, x=y, **kwargs) # OK, no checking of arguments
Support for Homogeneous Tuples (Tuple[t, ...])
Mypy now has a type for homogeneous, variable-length tuples: Tuple[t, ...] (with an explicit ...). These can be used like sequences, but the concrete type must be a tuple. These are useful for several library functions that expect concrete tuples, and also let us give precise types for *args arguments, which are tuples.
@no_type_check
You can use the @no_type_check decorator to skip all type checks within a function. It also lets you use function annotations for non-mypy uses:
from typing import no_type_check
@no_type_check
def special_annotations(arg: {'value': 1}) -> 1234: # Fine!
1 + '' # We can do all sorts of crazy things here, and mypy is silent!
Changes to Name Visibility in Stubs
As per PEP 484, mypy no longer export names imported in stubs, unless they are imported via as. For example, this stub won't export Tuple any more:
from typing import Tuple
def my_function() -> Tuple[int, str]: ...
This is especially nice together if you import a module with a stub using from module import *, as your namespace won't get polluted with random names imported by the stub from typing.
If in a stub you import using from m import *, everything in m will get exported. This makes it easy for a stub to re-export all definitions from another module.
Using ... in Stubs
PEP 484 recommends using an ellipsis (...) instead of pass for empty function and class bodies in stubs. It can can also be used as variable initializers and default argument values. This example illustrates all of these:
# This is stub file
var = ... # type: int
def function(arg: str = ...) -> int: ...
class VeryBadError(Exception): ...
bytearray Is Compatible with bytes
Mypy considers bytearray to be compatible with bytes (as per PEP 484). This simplifies stub files and annotations, as previously we often had to use Union[bytes, bytearray] or something similar, which was hard to remember to do consistently.
Workflow Improvements
This release has several experimental improvements to running mypy against your code. These are subject to change in the future and not very well documented, but they are useful enough that I'll introduce them anyway:
- You can give an arbitrary number of files or directories to type check on the mypy command line (e.g. mypy file1.py file2.py dir). Mypy will recurse into a directory argument if it contains an __init__.py or __init__.pyi (stub) file, looking for .py and .pyi files. If the directory is not a package, it'll only look for .py / .pyi files within the given directory but not in subdirectories.
- You can ask mypy to ignore files outside those explicitly mentioned on the command line (or inferred by the directory traversal algorithm discussed above) via --silent-imports. Stubs are still consulted, but other than that mypy will not try to follow imports to other modules or complain about missing stubs. This makes it easy to start experimenting with mypy on a subset of a larger codebase without getting a huge number of errors about missing stubs and similar.
- By default mypy type checks only functions with annotations. You can give mypy a the option --implicit-any to implicitly give an annotation with all Any types for every unannotated function. This way mypy can do useful (though a bit limited) type checking for unannotated code. As you add function annotations, mypy will do an incrementally better job. In the future, something like this may be the default mode of operation, once we get this good enough.
Miscellanea
Here are some additional improvements:
- Mypy now continues processing files after errors and can report errors in multiple files in a single run
- Rudimentary __new__ support
- Better type inference for isinstance checks, including support for and and not operations involving isinstance and multiple checks per condition
- Using a value with Any type as a base class (this is very useful together with # type: ignore)
- Support for @abstractproperty
There are also some interesting work-in-progress features such as a new automatic stub generator.
Acknowledgements
First of all, I'd like to thank Dropbox for contributing very significant engineering resources to improving mypy, including some of my time! We've been experimenting with using mypy on Dropbox code and initial results are promising, even though there are some big unsolved challenges ahead, such as type checking performance and scaling to very large codebases (but we are working on these!).
Also, thanks to Guido and Matt Robben for feedback on earlier drafts of this article.
Many people contributed to this release. Thanks to everybody who helped! Apologies if I've missed anybody (this includes contributions to typeshed):
- Andrew Aldridge
- Anup Chenthamarakshan
- Ben Darnell
- Ben Longbons
- ctcutler
- Daniel Shaulov
- Darjus Loktevic
- David Fisher
- David Shea
- Della Anjeh
- Eric Price
- Gigi Sayfan
- Greg Price
- Guido van Rossum
- Howard Lee
- Ian Cordasco
- icoxfog417
- Igor Vuk
- ismail-s
- James Guthrie
- Jared Hance
- Jing Wang
- Jukka Lehtosalo
- Kyle Consalus
- Li Haoyi
- Matthew Wright
- Matthias Bussonnier
- Matthias Kramm
- Michael Walter
- Michal Pokorný
- Mihnea Giurgea
- Motoki Naruse
- Prayag Verma
- Robert T. McGibbon
- Roy Williams
- Ryan Gonzalez
- Sander Kersten
- Sebastian Reuße
- Seo Sanghyeon
- Tad Leonard
- Tim Abbott
- Vita Smid
- Vlad Shcherbina
- Wen Zhang
- wizzardx
- Yuval Langer
An up-to-date list of all mypy code contributions is available at the GitHub contributors page.
