Tuesday, 8 November 2022

Mypy 0.991 Released

We’ve just uploaded mypy 0.991 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, performance improvements and bug fixes. You can install it as follows: 

    python3 -m pip install -U mypy

You can read the full documentation for this release on Read the Docs.

Basic Python 3.11 Support

Mypy and mypyc now have partial official support for Python 3.11. We provide compiled wheels for 3.11, and many 3.11 stdlib features are supported.

Most new language and typing features are still unsupported or are experimental. Adding support for the remaining 3.11 features is a priority for the mypy team.

Breaking Change: No Implicit Optional Types for Arguments

Mypy now disables implicit optional types such as these by default to fix a long-standing incompatibility with PEP 484, and they will have to be explicitly enabled: 

    def foo(s: str = None) -> int: ...  # Error!

Mypy will now only accept the example with an explicit optional type (e.g. str | None) for the s argument, when using the default options: 

    # s: Optional[str] also works
    def foo(s: str | None = None) -> int: ...  # OK

You can still explicitly enable implicit optional types via --implicit-optional or no_implicit_optional = False in your mypy config file. We don’t recommend relying on implicit optional types, since they are a non-standard feature, but we have no plans to stop supporting them.

You can also use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase to explicit optional types.

This was contributed by Shantanu:

  • Use --no-implicit-optional by default (Shantanu, PR 13401)
  • Suggest codemod for --no-implicit-optional (Shantanu, PR 13747)

Breaking Change: Namespace Packages Enabled by Default

Mypy now enables --namespace-packages by default (PEP 420), so that namespace packages (packages with a missing __init__.py file) are supported without any extra configuration.

Refer to the documentation of --namespace-packages and to Mapping file paths to modules in mypy documentation for more information.

In some cases you may need to use --explicit-package-bases and to explicitly declare additional package roots that mypy can’t figure out automatically by using the MYPYPATH environment variable or using mypy_path = <dirs> in your config file. Otherwise you may get errors about missing imports.

You can explicitly disable the option by using --no-namespace-packages or namespace_packages = False in your config file to return to the old behavior. We recommend first trying --explicit-package-bases discussed above if you encounter any issues.

This was contributed by Shantanu (PR 9636).

Recursive Types Enabled By Default

Mypy support for general recursive types is now official. These were available in mypy 0.981 through the --enable-recursive-aliases flag. Now you no longer need to explicitly enable them. Example: 

    # This is now allowed by default
    NaiveJSON = str | list["NaiveJSON"] | dict[str, "NaiveJSON"]
    test_data: NaiveJSON = {"foo": {"bar": "baz"}}  # OK

Note that you will need to use string escapes on the right hand side (e.g. "NaiveJSON") if you define recursive types outside of stub files, since Python doesn’t support forward references at runtime. Also, mypy generally doesn’t infer recursive types by default. Similar to TypedDict types, you will often need to explicitly annotate a variable with a recursive type or mypy will try to infer a non-recursive type instead.

You can still disable recursive types using --disable-recursive-aliases if they cause any problems. Please open an issue on the mypy issue tracker if this is the case.

See “Experimental Support for General Recursive Types” in the mypy 0.981 release blog post for more details.

This change was contributed by Ivan Levkivskyi in (PR 13516), (PR 13852) and (PR 13754).

Error Codes Shown by Default

Mypy now shows error codes at the end of each error message by default, within square brackets. Example: 

    error: 
    Unsupported operand types for + ("int" and "str")  [operator]

This makes it easier to silence only specific error codes on a line by using # type: ignore[<error-code>]. Example: 

    print(1 + "x")  # type: ignore[operator]

You can find helpful information about specific errors by looking up the error code in the documentation.

You can hide error codes by using --hide-error-codes or by adding show_error_codes = False to your configuration file.

This was contributed by Marc Mueller (PR 13542).

Safe Handling of Empty Function Bodies

For historical reasons, mypy used to always allow trivial function/method bodies (e.g. a body with only a single pass statement and/or a docstring), even if this is not type safe. Example: 

    # Error: Missing return statement (no error before 0.990)
    def some_func(arg: str) -> int:
        pass

This could hide bugs and was a generally confusing behavior. Mypy is now handling such empty bodies safely. In the example above you will now get an error (unless you are using --no-strict-optional). Mypy only allows an empty/trivial body in a method with a non-optional return type if the method is abstract: 

    class Base(ABC):
        @abstractmethod
        def must_implement_in_subclass(self, arg: str) -> int:
            pass  # Still OK

Additionally, mypy now flags unsafe super() calls to a method with an empty body that’s decorated with @abstractmethod. Mypy will now issue an error in this case.

To disable this functionality, use the --allow-empty-bodies flag.

This change was contributed by Ivan Levkivskyi in (PR 13729).

Enabling Experimental Features Individually

Experimental mypy features can now be enabled individually. With this change, using --enable-incomplete-features to enable all of them becomes deprecated and the recommended way forward is to enable experimental features individually with --enable-incomplete-features=FEATURE.

This change was contributed by Ivan Levkivskyi and Nikita Sobolev in (PR 13790).

Configuring Packages/Modules in the Config File

It is now possible to tell mypy to implicitly type check specific packages by using the options modules and packages in the configuration file, without having to explicitly include these on the command line. These options take a comma separated list of packages which will be checked by mypy if none are given on the command line.

Mypy will not recursively type check any submodules of the provided modules by modules and will recursively type check any submodules of the provided modules by packages.

These options may only be set in the global section. See also the documentation.

This feature was contributed by sameer-here (PR 12639).

Warn about Variable Annotations in Unchecked Functions

Mypy will no generate a note if you use a type annotation in a function without a type annotation (and you are not using --check-untyped-defs), since these annotations have no effect. Example: 

    def foo():
        x: int = bar()
        print(x)

This is treated as a warning, so it doesn’t cause a mypy run to fail: 

    $ mypy prog.py
    prog.py:2: note: By default the bodies of untyped functions 
        are not checked, consider using --check-untyped-defs  
        [annotation-unchecked]
    Success: no issues found in 1 source file

This was contributed by Ivan Levkivskyi (PR 13851).

Error Code for Using an Abstract Class as type[T]

Mypy disallows using an abstract type object (ABC) if type type[T] is expected. This would be unsafe, since mypy always allows calling type[T] values to instantiate objects, and ABCs can’t be instantiated.

Now this error has the dedicated error code type-abstract, and you can use --disable-error-code type-abstract to disable this check, since it can generate undesirable errors in some use cases. Example: 

    import abc
    
    class A(abc.ABC):
        @abc.abstractmethod
        def m(self) -> None: pass
    
    class C(A):
        def m(self) -> None: print('m')
    
    def check(o: object t: type[A]) -> bool:
        return isinstance(o, t)
    
    # Error: Only concrete class can be given here [type-abstract]
    check(C(), A)
    check(C(), C)  # OK

This was contributed by Ivan Levkivskyi (PR 13785).

Performance Improvements

Mypy includes a potential fix for a performance regression edge case introduced in mypy 0.982:

  • Revert literal type change in builtins.sum (Shantanu, PR 13961)

Changes to Error Reporting and Messages

  • Return exit status 0 if there are only notes and no errors (Ivan Levkivskyi, PR 13879)
  • Generate an error if a function object is used in a boolean context, with the truthy-function error code, even if the truthy-bool error code is disabled (Marc Mueller, PR 13686)
  • Silence errors from third-party packages in mypy daemon (Jukka Lehtosalo, PR 13768)
  • Preserve file order of messages during successive daemon runs (Jukka Lehtosalo, PR 13780)
  • Suggest additional types-* packages from typeshed (Jukka Lehtosalo, PR 13698)
  • Better diagnostic for conditional function mismatch (Shantanu, PR 13604)
  • Use consistent capitalization for TypeVar (Marti Raudsepp, PR 13687)
  • Suggest using upper bound for unbound type variable (Shantanu, PR 13730)
  • Show error codes for some notes (Ivan Levkivskyi, PR 13880)
  • Suggest using a protocol if trying to use a module as a type (Nikita Sobolev, PR 13861)
  • Improve error message for --strict-concatenate (Shantanu, PR 13777)
  • Improve error message for implicitly abstract functions (Shantanu, PR 13776)
  • Mention implicit export on missing attribute access (Shantanu, PR 13917)
  • Always mention explicit export when relevant (Shantanu, PR 13925)
  • Fix error code when reporting an invalid Literal type (Shantanu, PR 13763)
  • Replace invalid __set__ with __get__ in messages (Ilya Konstantinov, PR 13913)

Mypyc Fixes and Improvements

  • Add support for building mypyc code on WASM (Ethan Smith and Shantanu, PR 13446)
  • Fix C errors about shifting negative integers (Jukka Lehtosalo, PR 13876)

Documentation Updates

Several parts of the documentation received major updates.

  • Major update to "Using mypy with existing codebase" (Shantanu, PR 13683)
  • Update cheat sheet (PR 13873, PR 13679) (Shantanu)
  • Update “Runtime troubles” (Shantanu, PR 13680)
  • Mention files config file option in "Specifying code to check" (Shantanu, PR 13682)
  • Make language more consistent (Shantanu, PR 13677)
  • Update “Getting started” (Shantanu, PR 13734)
  • Update “Type inference and annotations” (Shantanu, PR 13736)
  • Discuss user defined protocols before built-in protocols (Shantanu, PR 13737)
  • Reorder sections in “Running mypy” (Shantanu, PR 13738)
  • Improve "Precise typing of alternative constructors" example (Jelle Zijlstra)
  • Improve “Getting started” docs (Shantanu, PR 13875)
  • Update “Extending mypy” docs (Nikita Sobolev, PR 13924)
  • Add clear warning about stubtest code execution (Nikita Sobolev, PR 13927)

Stubgen Improvements

Stubgen is a tool for automatically generating draft stubs for libraries.

  • Introduce an object-oriented system for extracting function signatures (Chad Dombrova, PR 13473)
  • Add known return types to magic methods (Nikita Sobolev, PR 13909)

Stubtest Improvements

Stubtest is a tool for testing that stubs conform to the implementations.

  • Detect abstract properties mismatches (Nikita Sobolev, PR 13647)
  • Catch SyntaxError from inspect.getsourcelines (Shantanu, PR 13848)

Other Notable Fixes and Improvements

  • Detect invalid ParamSpec annotations used with *args and **kwargs (Nikita Sobolev, PR 13892)
  • Fix crash with generic class definition in function (Michael Lee, PR 13678)
  • Fix joining a function against metaclass-using object constructors (Michael Lee, PR 13648)
  • Fixes to attrs magic attribute handling (Tin Tvrtković and Nikita Sobolev, PR 13522)
  • Support __attrs_init__ method in attrs classes (Spencer Brown, PR 13865)
  • Fix crash on match statement with value restricted TypeVar (Shantanu, PR 13728)
  • Fix crash in match statement with unmatchable class pattern (Shantanu)
  • Fix unsound variance (Shantanu, PR 13714)
  • Fix crash with report generation on namespace packages (Shantanu, PR 13733)
  • Remove use of LiteralString in builtins (Shantanu, PR 13743)
  • Treat __dataclass_fields__ as a ClassVar (Shantanu, PR 13721)
  • Respect per-module follow_import for empty folders (Shantanu, PR 13758)
  • Fix a crash related to dataclasses and type aliases (Ivan Levkivskyi, PR 13759)
  • Add install-types extra that depends on pip (David Runge, PR 13739)
  • Restore Type vs Callable special-casing that was broken in refactoring (Ivan Levkivskyi, PR 13784)
  • Fix module and protocol subtyping and hasattr with a module (Shantanu, PR 13778)
  • Make join of recursive types more robust (Ivan Levkivskyi, PR 13808)
  • Fix minor issues with **kwargs that use TypedDict unpacking (Ivan Levkivskyi, PR 13854)
  • Fix crash on missing indirect dependencies (Ivan Levkivskyi, PR 13847)
  • Ignore promotions when simplifying unions (Ivan Levkivskyi, PR 13781)
  • Fall back to FORCE_COLOR environment variable if MYPY_FORCE_COLOR is not present (Alex Waygood, PR 13814)

Typeshed Updates

Typeshed is now modular and distributed as separate PyPI packages for everything except the standard library stubs. Please see git log for full list of typeshed changes.

Acknowledgements

Thanks to all mypy contributors who contributed to this release:

  • Alex Waygood
  • Bas van Beek
  • Ben Raz
  • Chad Dombrova
  • David Runge
  • Ethan Smith
  • Ilya Konstantinov
  • Ivan Levkivskyi
  • iyanging
  • Jared Hance
  • Jelle Zijlstra
  • Jeroen Van Goey
  • Jinze Wu
  • Jukka Lehtosalo
  • Kevin Kirsche
  • KotlinIsland
  • Marc Mueller
  • Marti Raudsepp
  • Matt Wozniski
  • Michael Lee
  • Nikita Sobolev
  • pranavrajpal
  • Richard Si
  • Ryan Soklaski
  • sameer-here
  • Shantanu
  • Spencer Brown
  • Stas Ilinskiy
  • Tim D. Smith
  • Tin Tvrtković
  • Valentin Stanciu

We’d also like to thank our employer, Dropbox, for funding the mypy core team.

Posted by at
Subscribe to: Posts (Atom)