Friday, 13 April 2018

Mypy 0.590 Released

We’ve just uploaded mypy 0.590 to the Python Package Index (PyPI). This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the documentation for this release on ReadTheDocs.

New Features

PEP 561

We are releasing experimental support for PEP 561, which allows packages installed through pip (or other packaging tools, as long as packages end up on sys.path, typically in site-packages) to indicate that they contain type annotations (either inline or in stubs), and also allows packages that just provide stubs, overriding typeshed. We’re grateful to PEP 561 author Ethan Smith for the design and implementation. (Note that normally mypy doesn’t process packages installed in site-packages or elsewhere on sys.path, because realistically most packages, whether third-party or in the standard library, cause a myriad of spurious type checking errors.)

A summary of the new functionality: 1. Packages can include a marker file named py.typed to indicate that they have inline annotations or stubs next to the sources. For packages marked this way, if foo.py and foo.pyi both exist, mypy will use the annotations from foo.pyi (ignoring foo.py completely), while if only bar.py exists, mypy will type-check and use the annotations in bar.py. 2. Instead of distributing annotations or stubs as part of a package, it’s also possible to have a separately installable package that just provides stubs; e.g. for package baz, if a package named baz-stubs is installed, mypy will use the annotations from that package.

For more details read the documentation. The full specification is in PEP 561. Note that the PEP hasn’t been accepted yet, hence the labeling of mypy’s support as experimental. (The mypy implementation is considered a reference implementation which will be used to gain experience to feed back into the PEP review process. We’re expecting minimal changes to the basics as described above though.)

Incremental mode is on by default

After over a year of “burn-in” we’re finally confident enough about incremental mode to make it the default. For larger codebases this can dramatically speed up mypy runs. If for some reason you want to turn it off, you can use --no-incremental.

Flags for always true/false

New command line flags --always-true NAME and --always-false NAME can be used to treat variables named NAME als compile-time constants. This allows you to define your own flags for things like platform checks, version checks, and other environmental checks. Example:

    ON_WINDOWS = (sys.platform == 'win32')
    def foo():
        if ON_WINDOWS:
            <do it the Windows way>
        else:
            <do it the UNIX way>

By default mypy will check both branches of the if statement. When this example is checked with --always-true ON_WINDOWS only the first branch is checked; with --always-false ON_WINDOWS only the second branch is checked. Note that in all cases mypy does not consider the initial value given to the variable. (This is by design, since mypy doesn’t assume that variables remain constant over the lifetime of the program.)

Note that mypy only checks the last component of the variable name — module and class attributes whose name matches one of these will also be affected. (This is a blessing and a curse, and a reminder to use this feature sparingly and only for the most obvious cases.)

These flags may be repeated on the command line, e.g. --always-true FOO --always-true BAR --always-true BAZ assumes FOO and BAR are always true and BAZ is always false.

These flags may also be given in the config file (e.g. mypy.ini). There the syntax is

    always_true = FOO, BAR
    always_false = BAZ

The may also occur in per-module sections, in order to limit the scope of their effect.

Add a per-module follow_imports_for_stubs options

Setting this flag to true causes follow_imports to affect stub files. Example:

    [mypy-foo]
    follow_imports = skip
    follow_imports_for_stubs = true
will force skipping the import of module foo even if a stub file (foo.pyi) is found. See the documentation for a full explanation.

Notable Bugs Fixed

  • Fix error message when assigning to read-only property (PR 4863 by Michael Lee, fixes issue 4756)
  • Respect --follow-imports=silent in --warn-unused-ignores (PR 4857, fixes issue 3949)
  • Don't produce spurious “unused type ignore” messages in incremental mode (PR 4841, fixes issue 2960)
  • Fix operator access on Type[...] (PR 4823, fixes issue 4789)
  • Remove incorrect override check on overloaded methods (PR 4810 by Michael Lee, fixes issue 4565)
  • Handle NotImplemented returned from __ne__ (PR 4770 by Peter Ludemann, fixes issue 4767)
  • Options in mypy.ini should be spelled with underscores, not hyphens (PR 4754 by Emil Hessman, fixes issue 4753)

Documentation Improvements

Stubgen Improvements

  • Emit Any type annotations for unannotated arguments (PR 4774 by Matt Gilson, fixes issue 4766)
  • Fix stubgen --recursive to only generate stubs for requested submodules. (PR 4845 by Matt Gilson, fixes issue 4844)
  • Properly format Callables (PR 4650 by Evan Hubinger, fixes issue 4640)

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • Emil Hessman
  • Eric Wieser
  • Ethan Smith
  • Evan Hubinger
  • Evan Krall
  • Jelle Zijlstra
  • Matt Gilson
  • Michael Lee
  • Peter Ludemann
  • Rob Day
  • Viet Hung Nguyen

Additional thanks to all contributors to typeshed:

  • Alan Du
  • Ashwini Chaudhary
  • Bryce Guinta
  • Christopher Schramm
  • Danny Weinberg
  • David Euresti
  • dgelessus
  • Eric Wieser
  • Evan Krall
  • Jelle Zijlstra
  • Josh Holland
  • João Santos
  • Leonardo Fedalto
  • Luka Sterbic
  • Łukasz Langa
  • Matt Gilson
  • matthewfranglen
  • NODA, Kai
  • Nipunn Koorapati
  • Omar Sandoval
  • Patrick Meyer
  • Russell Cloran
  • Sebastian Rittau
  • Semyon Proshev
  • Tuomas Suutari
  • z33ky
— Guido van Rossum, on behalf of the mypy team

Friday, 23 March 2018

Mypy 0.580 released

We’ve just uploaded mypy 0.580 to the Python Package Index (PyPI). This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the documentation for this release on ReadTheDocs.

New Features

  • Add support for multiple packages on the command line alongside modules (Eric Wieser, PR 4733). Now you can run mypy with several packages on the command line, for example:
    mypy --package p.a --package p.b --module c
  • Set python_version to default to sys.version_info (Ethan Smith, PR 4686). This makes the implementation match docs and makes it easier to implement PEP 561.

Notable Bugs Fixed

  • Fix issues related to indirect imports in import cycles (PR 4695)
  • Fix frozen subclass and callable issues in attrs plugin (David Euresti, PR 4755)
  • stubgenc: Generate PEP 526 style variable annotations (gclinch, PR 4716)
  • Fix XML report generation (PR 4745)

Improved Error Messages

  • Helpful error message when strict is found in configuration file, which is unsupported (gclinch, PR 4706)

Other Changes

  • Refuse to run under Python 3.5.0 or anything before Python 3.4 (PR 4721). This is caused by the fact that the version of typing module shipped with Python 3.5.0 has several bugs and incompatible/old API.

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • David Euresti
  • Eric Wieser
  • Ethan Smith
  • gclinch
  • Jelle Zijlstra

Additional thanks to all contributors to typeshed:

  • Dan Sully
  • David Euresti
  • Ethan Smith
  • Josh Holland
  • Kenny Do
  • Martin DeMello
  • Matthias Kramm
  • Miguel Gaiowski
  • Nipunn Koorapati
  • rchen152
  • Rob Day
  • Roy Williams
  • Salvo 'LtWorf' Tomaselli
  • Sebastian Rittau
  • Sergey Passichenko
  • Svyatoslav Ilinskiy
  • Takuya Akiba
  • yed podtrzitko

— Ivan Levkivskyi, on behalf of the mypy team

Friday, 2 March 2018

Mypy 0.570 Released

We’ve just uploaded mypy 0.570 to the Python Package Index (PyPI). This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:

    python3 -m pip install -U mypy

You can read the documentation for this release on ReadTheDocs.

We're also happy to welcome long-time mypy core dev Ivan Levkivskyi to the mypy team at Dropbox.

New Features

  • Type check class decorators (David Euresti, PR 4544)
  • Plugin for typechecking classes generated by attrs (Documentation) (David Euresti)
  • Hooks into the semantic analyzer for plugins (PR 4328, PR 4397) (David Euresti)

Notable Bugs Fixed

  • Allow calling instances of tuple subclasses that define __call__ (PR 4367)
  • Improve handling of callable() (PR 4343)
  • Fix crashes due to improper handling of type variables (PR 4384)
  • Various stubgen bugs fixed (PR 4407, PR 4467, PR 4518) (hoefling)
  • Allow subtypes to define more overloads than their supertype (Ryan Gonzalez, PR 3263)
  • Fix typing of assignment to list (e.g. [a, b] = foo()) (Elazar Gershuni, PR 4340)
  • Don’t generate --warn-return-any warnings from binary magic methods that raise NotImplementedError (Vlad Starostin, PR 4545)
  • Fix false negatives in incremental mode due to missing dependencies on parent classes (PR 4566)
  • Fix type of forward reference to a decorated class method (PR 4486)
  • Have divmod() properly dispatch to __[r]divmod__ methods (David Euresti, PR 4585)
  • Fix setup.py breakage (yed podtrzitko, PR 4570)
  • Fix platforms in setup.py (Thomas Grainger, PR 3765)
  • Make psutil an optional dependency (Dominik Miedziński, PR 4634)

Improved Error Messages

  • Give faster feedback by streaming error messages incrementally during type checking (PR 4396)
  • Add variable name to "Need type annotation for variable" error (Elliott Beach, PR 4496)
  • Report columns as 1-based, not 0-based (Elliott Beach, PR 4482)
  • Display argument name, not number, for functions called with keyword arguments (Elliott Beach, PR 4504)

Other Changes

  • Document Generator/Iterator distinction (Carl Meyer, PR 4424)
  • Consistently exit with code 2 on blocking errors (PR 4449)
  • Allow unrecognized flags beginning with x_ in mypy.ini, for use in configuring linters and other tools that work with mypy (PR 4521)
  • Refactor and clean up test discovery (Elazar Gershuni, PR 3973)
  • Remove the home-rolled myunit testing framework (Elazar Gershuni, PR 4369)
  • Test cases for enum iteration (Nick Gaya, PR 4400)

Acknowledgments

First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.

Thanks to all mypy contributors who contributed to this release:

  • a4ff7810
  • Amit D
  • Ashwini Chaudhary
  • Carl Meyer
  • David Euresti
  • Dominik Miedziński
  • Elazar Gershuni
  • Elliott Beach
  • Ethan Smith
  • Germano Gabbianelli
  • Guilherme Caminha
  • hoefling
  • Jelle Zijlstra
  • Nick Gaya
  • Ryan Gonzalez
  • Thomas Grainger
  • Vlad Starostin
  • yed podtrzitko

Additional thanks to all contributors to typeshed:

  • Alan Du
  • Alex Dehnert
  • Anthony Sottile
  • Ashwini Chaudhary
  • Asim Goheer
  • Can Berk Güder
  • Carl Meyer
  • Chad Dombrova
  • Daniel Bowman
  • Daniel Bowtell
  • Daniel Li
  • David Euresti
  • David Zbarsky
  • Elazar Gershuni
  • Elliott Beach
  • Erwin Janssen
  • Ethan Smith
  • Garrett
  • George King
  • gossrock
  • Han Song
  • Ilya Kulakov
  • James M. Allen
  • Jason Fried
  • Jelle Zijlstra
  • Karl Ramm
  • Łukasz Langa
  • Mark Amery
  • Martin DeMello
  • Marti Raudsepp
  • Matt Bogosian
  • Matthias Kramm
  • Michael Lee
  • Michel Albert
  • Nathan Henrie
  • Neil Conway
  • Nick Gaya
  • NODA, Kai
  • PeterJCLaw
  • Pradyun Gedam
  • rchen152
  • Rhys Parry
  • Rick Kilgore
  • Roy Williams
  • Salvo 'LtWorf' Tomaselli
  • Sebastian Rittau
  • Semyon Proshev
  • shahin
  • Thomas Broadley
  • Yuri Khan

--Michael J. Sullivan, on behalf of the mypy team

Monday, 5 February 2018

Dropbox releases mypy-protobuf

At Dropbox we use Google’s Protocol Buffers a lot. So much, in fact, that we sorely needed a way to generate mypy stub files (.pyi files) that match the generated .py files. And out of sheer enthusiasm we implemented the stub file generation code twice! Once in Go, once in Python. Go figure. :-)

We’ve now open-sourced both versions, so if you’re a Python shop, you can use the Python version, while if you’re a Go shop that occasionally uses Python, or you want a faster stub generator, you can use the Go version.

Ready to give it a try? Here’s the repo: https://github.com/dropbox/mypy-protobuf; instructions for both versions are in the top-level README.md. If you’re going for the Python version there’s no need to clone the repo: just pip install mypy-protobuf and pass --mypy_out=<somewhere> to your protoc invocation.