Friday, 8 February 2019

Mypy 0.670 Released

We’ve just uploaded mypy 0.670 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

(There are also new mypy-mypyc wheels; see the 0.660 release blog post.)

You can read the documentation for this release on Read the Docs.

New Feature: Variable Redefinition

As you might have experienced, PEP 484 and mypy require a variable to keep its initial type throughout a scope. Occasionally this can be annoying, as the traditional workaround is to rename the second (independent) use of the variable. We have introduced a new flag, --allow-redefinition, which relaxes this behavior in certain contexts. Here’s an example where this may be useful:

    def process(items: List[str]) -> None:
        # 'items' has type List[str]
        items = [item.split() for item in items]
        # 'items' now has type List[List[str]]
        ...

You can also enable this per module in the mypy.ini file using allow_redefinition = True. See the docs for the command-line flag and the config option.

Stubgen Improvements

We’ve done a fairly large overhaul of the stubgen utility, which can automatically generate draft stub files for almost any module or package:

  • Streamline the command-line interface; in particular, the source discovery/collection options now more closely resemble those of mypy itself: -m <module>, -p <package>, <file> or <directory>
  • Perform a lightweight semantic analysis to generate somewhat better stubs
  • Added documentation
  • When parsing types out of docstrings (this happens for C modules only), stubgen now handles overloaded signatures generated by pybind11 (Wiktor Niesiobędzki, PR 5975)

Other Improvements

  • Expand getting started docs to discuss type hints in more detail (Michael Lee, PR 6226)
  • Always infer in operator as returning bool (Joel Croteau, PR 5688)
  • Allow star args in ctypes.Array constructor (Alan Du, PR 6213)
  • Fix plugin invocation for __call__ methods (oremanj, PR 6334)
  • Implement the XDG directory spec for config files: $XDG_CONFIG_HOME/mypy/config is now included in the search path for config files (Ryan Delaney, PR 6304)
  • When using the --junit-xml flag, the Python version and platform in the junit.xml file are now formatted as mypy-py3_6-windows — previously this was mypy-py3.6-windows but the dot was misinterpreted by some tools (PR 6222)
  • Update the typed_ast dependency to version 1.3.1; this means we now officially support Python 3.7
  • Temporarily delete pyproject.toml from the repo in order to work around a pip bug (PR 6342)
  • Include mypy_bootstrap.ini in PyPI packages (John Reese, PR 6252)

Internal Improvements and Code Cleanup

  • Fix daemon overlapped I/O when more data needs to be read (Ethan Smith, PR 6272)
  • Move most message constants to mypy.message_registry (Chad Dombrova, PR 6194)
  • Fix all DeprecationWarning: invalid escape sequence (Mickaël Schoentgen, PR 6195)
  • Add strictness flags (for mypy itself) that can be added with no source code changes (Michael Lee, PR 6237)
  • Some daemon performance improvements for large code bases

Typeshed Updates

  • Create stubs for Flask (Pascal Corpet, PR 2740)
  • Fix type of indent in JSONEncoder (Vield, PR 2737)
  • Make metavar in argparse be Optional (cormoran, PR 2739)
  • As of Python 3.6 dump_stats() method allows PathLike object to be passed (Igor Davydenko, PR 2741)
  • Add back StopIteration.value in Python 3 (Jelle Zijlstra, PR 2744)
  • Fix logging.getLevelName() type hints (Michael Noseworthy, PR 2730)
  • Add missing explicit Optional to stubs for the xml.etree package (Michael R. Shannon, PR 2734)
  • logging: inherit TimedRotatingFileHandler from Handler (Евгений, PR 2738)
  • Add SSLCertVerificationError fields (Hynek Schlawack, PR 2745)
  • Fix six.raise_from() value type (Frazer McLean, PR 2746)
  • builtins.pyi: Update __iadd__() and __imul__() in class list (Utkarsh Gupta, PR 2754)
  • pkg_resources: Add PKG_INFO str attribute for Distribution class (Joachim Jablon, PR 2775)
  • pkg_resources: fix stub for get_metadata_lines() (Joachim Jablon, PR 2776)
  • Add type annotation for collections.deque.__iadd__() (Joel Rosdahl, PR 2774)

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Alan Du
  • Chad Dombrova
  • Ethan Smith
  • Joel Croteau
  • John Reese
  • Michael Lee
  • Mickaël Schoentgen
  • oremanj
  • Ryan Delaney
  • Sebastian Witowski
  • Wiktor Niesiobędzki

Additional thanks to all contributors to typeshed:

  • cormoran
  • Евгений
  • Frazer McLean
  • Hynek Schlawack
  • Igor Davydenko
  • Jelle Zijlstra
  • Joachim Jablon
  • Joel Rosdahl
  • Michael Noseworthy
  • Michael R. Shannon
  • Pascal Corpet
  • Utkarsh Gupta
  • Vield
— Guido van Rossum, on behalf of the mypy team

Wednesday, 16 January 2019

Mypy 0.660 Released

We’ve just uploaded mypy 0.660 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes an optional compiled version, new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the documentation for this release on Read the Docs.

Introducing an Optional Compiled Version of Mypy — mypy-mypyc

As part of our ongoing attempts to speed up mypy, we have built mypyc, an ahead-of-time compiler from type-annotated Python to CPython C extensions modules. We will be writing more about mypyc in the near future, but for now the main thing to know is that mypy can be compiled with mypyc, speeding it up by up to 3-4x!

To install a compiled mypy, instead of installing the mypy package, install the mypy-mypyc one (mypy-mypyc packages share version numbers with the corresponding mypy package):

    python3 -m pip uninstall -U mypy  # if mypy is installed
    python3 -m pip install -U mypy-mypyc

Binaries are available for 64-bit Windows, macOS, and Linux for Python 3.5 to 3.7.

Literal Types

This release introduces a new experimental kind of type: literal types. Literal types let you declare that some expression is equal to a specific primitive value. For example, if a variable is annotated with the type Literal["foo"], mypy will understand that variable is not only of type str, but is also equal to specifically the string "foo".

This feature is particularly useful when you want to annotate functions where the return type depends on the exact value provided. For example, suppose we have a function fetch_data(…) that returns bytes if the first argument is True, and str if it’s False. Previously, there was no good way of annotating the return type of fetch_data: the best we can do is Union[bytes, str]. With literal types, we can produce a more precise signature:

    from typing import overload
    from typing_extensions import Literal
    
    @overload
    def fetch_data(raw: Literal[True]) -> bytes: ...
    @overload
    def fetch_data(raw: Literal[False]) -> str: ...
    # Fallback overload if the user provides a regular bool
    @overload
    def fetch_data(raw: bool) -> Union[bytes, str]: ...
    
    def fetch_data(raw):
        # Implementation is omitted
        ...
    
    reveal_type(fetch_data(True))   # Revealed type is 'bytes'
    reveal_type(fetch_data(False))  # Revealed type is 'str'
Note 1: Literal types have some important nuances and limitations. For example, Literal[…] may contain only primitive literal values, and mypy will not attempt to understand complex expressions using literal types on a deep level. For more details, please see the documentation. Note 2: In order to use this feature, you must install the latest typing_extensions package, version 3.7.2:
    python3 -m pip install -U typing_extensions

Mypy Daemon Windows Support

Thanks to work from Ethan Smith, the mypy daemon, dmypy, is now fully supported on Windows!

Quick Mode Removed

The quick mode (--quick-and-dirty) has been deprecated for two mypy releases and has now been removed. The mypy daemon, dmypy, offers high speed without the correctness compromises of the quick mode.

Plugin Improvements

  • Add documentation for plugin system (PR 6057)
  • Make name lookup available to all plugin hooks (PR 6044)
  • Add more information to FunctionContext and MethodContext (Maxim Kurnikov, PR 5918)

Other Improvements and Notable Bugs Fixed

  • Introduce an optional sqlite backed incremental cache, enabled with --sqlite-cache (PR 6023)
  • Fix a daemon crash when there is a decode error (PR 6064)
  • Allow setting python_executable from config file (Ethan Smith, PR 5777)
  • Short-circuit if expression for always true/always false variables and MYPY/TYPE_CHECKING (Samer Masterson, PR 5965)
  • Don't map actual kwargs to formal *args (PR 6096)
  • Disable cache when producing reports (Maarten ter Huurne, PR 6076)
  • Fix issues with pointer arrays in the ctypes plugin (Jakub Stasiak, PR 6097)
  • Support kw_only=True in the attrs plugin (David Euresti, PR 6107)
  • Fix some daemon crash bugs (PR 6098)
  • Better error messages when __eq__ has unexpected signature (David Wobrock, PR 6106)
  • Collect additional timing stats and allow reporting them from the daemon (PR 6137)
  • Fix dmypy run when bad options passed to mypy (Ethan Smith, PR 6153)
  • Improve error messages from multiple inheritance compatibility checks (Maxim Kurnikov, PR 5926)
  • Fix an incremental mode crash that can occur in situations with import cycles and star imports (PR 6179)

mypy_extensions Moved to a Separate Repository

This is likely only of interest to downstream packagers, but the mypy_extensions module has been moved from the extensions subdirectory of the mypy repository to its own repository, https://github.com/python/mypy_extensions.

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Chad Dombrova
  • Chris Philip
  • David Euresti
  • David Wobrock
  • Ethan Smith
  • Jakub Stasiak
  • Jared Hance
  • Jonathan Striebel
  • Maarten ter Huurne
  • Maxim Kurnikov
  • Michael Lee
  • Mykhailo Havelia
  • Samer Masterson
  • Shashank Parekh
  • Vincent Perez

Additional thanks to all contributors to typeshed:

  • Alexander Lyon
  • Alex Sarkesian
  • Andrew Gaul
  • Andrew Svetlov
  • Brandt Bucher
  • Dan Čermák
  • Daniel Mouritzen
  • Dave Halter
  • David Euresti
  • Diogo Magalhães Martins
  • Dominik Gabi
  • Ethan Smith
  • gnattishness
  • Hynek Schlawack
  • Ilya Konstantinov
  • Jared Hance
  • Jelle Zijlstra
  • Juan Gonzalez
  • Kai Willadsen
  • Kostya Esmukov
  • Michael Lee
  • Philipp Hahn
  • Sander Voerman
  • Savo Kovačević
  • Sebastian Rittau
  • Tomer Keren
  • Utkarsh Gupta
  • Ville Skyttä
— Michael J. Sullivan, on behalf of the mypy team