Mypy has been bundling stubs for many third-party libraries from typeshed, a repository of type definitions for Python packages. Mypy uses the stubs to type check code that uses these libraries, and they allow more types to be inferred by mypy. Typeshed is not just for mypy — it’s also used by several other tools in the Python typing ecosystem.
Mypy will no longer include stubs for third-party libraries in the soon-to-be-released mypy 0.900 release. Instead, mypy will only ship stubs for standard library modules, while stubs for third-party libraries can be independently installed using pip. For example, if you use requests, you can install the stubs like this:
python3 -m pip install types-requests
The stub packages are named types-<project name>. Note that the project name may be different from the runtime package name. For example, stubs for the yaml package are available in types-PyYAML.
Steps for Mypy Users
When you update to mypy 0.900 (once it has been released), you’ll likely get errors about missing stubs. Mypy will suggest the stub package that should be installed. Here’s an example output from mypy:
a.py:1: error: Library stubs not installed for "requests"
a.py:1: note: Hint: "python3 -m pip install types-requests"
a.py:1: note: (or run "mypy --install-types" to install all
missing stub packages)
a.py:1: note: See https://mypy.readthedocs.io/en/...
You can now run mypy --install-types to install all the missing dependencies (unless you are using mypy daemon). Alternatively, you can install the stubs manually using pip as suggested by mypy.
As a final step, you may want to add the stub packages to the build or test dependencies of your project, often in requirements.txt. You can use pip freeze to help pin to the current installed versions of the stubs. For example, you might add this line to your requirements.txt:
types-requests==0.1.8
Currently stub packages are mostly versioned independently of the implementation package. However, it’s possible to update stub metadata in typeshed to have the versioning better reflect the target implementation package version.
Motivation
The bundling of stubs with mypy has been convenient, but it also has caused a lot of friction, especially for users with many third-party dependencies. Here are the main benefits of unbundling stubs:
- Updating stubs without updating mypy. Right now it’s not straightforward to update bundled stubs to a newer version without also updating mypy. Soon you can use pip to update to the latest stubs, without having to update to mypy as well.
- Updating mypy without updating stubs. Updating stubs or installing new stubs sometimes results in additional mypy errors, due to newly added but too restrictive types in stubs, or newly introduced but correct types that expose incorrect types in your code. Previously mypy users had to deal with these errors when updating mypy, which complicated the update process. Soon you’ll be able to update mypy without updating third-party stubs. Standard library stubs will still be bundled for convenience, since all mypy users need them, and they tend to be easier to validate for correctness.
- Faster access to new stubs and improvements. If new stubs or stub improvements are contributed to typeshed, previously you had to wait for a new mypy release that includes them, or deal with a custom, unsupported typeshed set-up that hasn’t been properly tested. With mypy 0.900, you will be able to install or update stub packages whenever you want. As soon as a change to a stub is merged, typeshed will automatically upload a fresh stub package to PyPI. If you are not happy with some stub changes or encounter an issue, you can easily contribute fixes and start using the updated stubs, or continue to use older stubs if you so wish.
- Lower barrier to contributing new stubs. Since previously every user would get all contributed stubs, each new stub was a somewhat risky proposition. If some types were incorrect, or if there were missing definitions in the stubs, this could result in many false positives. Modular typeshed lets each user individually decide whether to start using the contributed stubs, making the impact of glitches in stubs lower. The faster iteration speed should also make it more likely that issues will be fixed soon.
- Consistency. Previously some third-party library stubs were bundled with mypy, some were bundled with the main implementation (upstream) package, and some were available on PyPI as separate stub packages. Now there will be one fewer option: you’d either use stubs or types included with the implementation package, or install a stub package. Types included with the main package are the preferred option, but many packages still don’t have them. Stub packages will be common for some time. Installing, versioning and updating stubs and implementation packages will also be more similar.
- Support for multiple library versions. Libraries may introduce backward incompatible API changes, such as removing deprecated features. Sometimes, for various reasons, you may want to continue using an older version of a library. Now it will be easy to also continue using an older version of stubs, since library stubs are independently versioned. This also opens the option of maintaining separate stubs for versions 1.x and 2.x of a library, for example.
Vendoring Stubs
If you don’t want to deal with installing stub packages, mypy 0.900 will support vendoring stubs (i.e., maintaining a local copy in your repository). First, you can copy some stubs from typeshed into a directory, and point the MYPYPATH environment variable (or mypy_path in the mypy configuration file) to it. Mypy will look for stubs in that directory. For example, you could use a directory named stubs in your repository structured like this:
- stubs/toml.pyi
- stubs/yaml/__init__.pyi
- …
If your project supports both Python 2 and 3, you can have stubs for Python 2 in a @python2 subdirectory of the main stub directory:
- stubs/foobar.pyi (used for Python 3)
- stubs/@python2/foobar.pyi (used for Python 2)
If you include stubs in the mypy search path, stubs/@python2 will automatically be used for Python 2 stubs.
Feedback?
Feel free to open a GitHub issue if you have concerns about this new approach of distributing stubs.
Acknowledgements
Ivan Levkivskyi set up the infrastructure for automatically uploading stub packages to PyPI and implemented the necessary changes to typeshed.
