Static type checking for Python
We mentioned mypy as a must in a previous post about Python Best Practices — here, we want to introduce it in more detail.
sleepy, as the docs explain, is a “static type checker for Python”. This means that it adds type annotations and controls to the Python language, which is typed dynamically by design (types are inferred at runtime, unlike, say, C++). Doing so allows you to find errors in your code at compile time, which is a huge help and a must for any semi-pro Python project, as explained in my previous post.
In this post, we will introduce mypy using various examples. Disclaimer: This post will not present all of mypy’s features (or even close to them). Instead, I’ll try to find a good balance between enough detail to allow you to write almost all the code you want, and building a steep learning curve from scratch to a solid understanding of mypy. For more details I’d like to check the official docs or any other great tutorials out there.
To install mypy, just run: pip install mypy
However, I would recommend using some sort of dependency management system, such as poetry. It explains how to include this and mypy in a larger software project. here.
Let’s motivate the use of mypy with a first example. Consider the following code:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
multiply expects two numbers and returns your product. Thus, multiply(3, 10) works fine and returns the desired result. But the second statement fails and blocks execution, since we can’t multiply strings. Because Python was typed dynamically, nothing prevented us from coding/executing that statement, and we only found the problem at runtime, which is problematic.
Here, mypy comes to the rescue. Now we can write down the arguments and also the return type of the function, like so:
def multiply(num_1: int, num_2: int) -> int:
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
This annotation will not change the execution in any way, in particular, you can still run this faulty program. However, before doing so and submitting our program, we can now run mypy and check for errors via: mypy .
Running this command will fail and correctly flag that we cannot pass strings to multiply. The above command is meant to be executed from the main application folder, . will check all files in the current folder and subdirectories. But you can also check specific files via mypy file_to_check.py.
Hopefully this motivated the need and use of mypy, now let’s dig deeper.
mypy can be configured in many different ways; Without going into details, you just need to find a configuration file (like mypy.ini, pyproject.toml, …) with a “mypy” section in it. Here, we will create the default file. mypy.iniwhich should live in the main folder of the project.
Now, let’s move on to the possible configuration options. To do this, let’s go back to our initial example:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
Just running mypy doesn’t actually produce errors! That is, because type hints are optional by default, and mypy only checks types where an annotation is provided. We can disable this via the flag. — disallow-untyped-defs. Also, there are a multitude of other flags one can use (see here). However, in keeping with the general format of this post, we will not go into details of all of these, but will only present the strict mode. This mode activates basically all optional checks. And in my experience, the best way to use mypy is to simply request the strictest verification possible, and then fix (or selectively ignore) any problems that arise.
To do this, we are going to fill the mypy.ini file like this:
[mypy]
strict = true
The section header [mypy] it is required for any mypy-related configuration, and the next line is self-explanatory.
When we now run mypy as usual, we get errors complaining about missing type annotations, which only go away once everything is written and we remove the bad string call.
Now let’s take a closer look at how to annotate with mypy.
In this section, we will describe the most common mypy keywords and type annotations.
We can annotate primitive types simply by using their Python type, i.e. bool, int, float, str…:
def negate(value: bool) -> bool:
return not valuedef multiply(multiplicand: int, multiplier: int) -> int:
return multiplicand * multiplier
def divide(dividend: float, divisor: float) -> float:
return dividend / divisor
def concat(str_a: str, str_b: str) -> str:
return str_a + " " + str_b
print(negate(True))
print(multiply(3, 10))
print(divide(10, 3))
print(concat("Hello", "world"))
From Python 3.9 onwards, also built-in collection types can be used as type annotations. That is list, set, dict…:
def add_numbers(numbers: list[int]) -> int:
return sum(numbers)def cardinality(numbers: set[int]) -> int:
return len(numbers)
def concat_values(value_dict: dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
As we can see, we have to specify the content of the containers (for example, int). For mixed types, please see below.
Older versions of Python
For earlier versions of Python, one had to use types inherited from the typing module:
from typing import Dict, List, Setdef add_numbers(numbers: List[int]) -> int:
return sum(numbers)
def cardinality(numbers: Set[int]) -> int:
return len(numbers)
def concat_values(value_dict: Dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
content mix
As mentioned above, we may want to create containers that hold different types of data. For this, we can use the Union keyword, which allows us to annotate a type as a union of types:
from typing import Uniondef scan_list(elements: list[Union[str | int]]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")
scan_list([1, "hello", "world", 100])
Similar to the simplifications made in Python 3.9, Python 3.10 (specifically PEP 604) introduces a shorthand notation for the Union write using the logical operator or (|):
def scan_list(elements: list[str | int]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")scan_list([1, "hello", "world", 100])
In this section, we will introduce more types and essential keywords.
None
Nonejust like in “normal” Python, it denotes a None value: most commonly used to annotate functions without a return type:
def print_foo() -> None:
print("Foo")print_foo()
Optional
We can often run into situations where we want to implement branching code based on whether we pass a value for a parameter or not, and we often use None to indicate its absence. For this, we can use typing.Optional[X] – which denotes exactly this: annotates type Xbut it also allows None:
from typing import Optionaldef square_number(x: Optional[int]) -> Optional[int]:
return x**2 if x is not None else None
print(square_number(14))
After Python 3.10 and later, introduced PEP 604, Optional can be shortened back to X | None:
def square_number(x: int | None) -> int | None:
return x**2 if x is not None else Noneprint(square_number(14))
Please note that this does not apply to required or optional parameters, which is often confused! An optional parameter is one that we don’t have to specify when calling a function, whereas mypy’s Optional indicates a parameter that can be of some type, but also None. A possible source of confusion could be that a common default value for optional parameters is None.
Any
Any, as the name suggests (sorry I keep repeating this sentence…) simply allows all types, thus disabling any kind of type checking. So try to avoid this whenever you can.
from typing import Anydef print_everything(val: Any) -> None:
print(val)
print_everything(0)
print_everything(True)
print_everything("hello")
Variable Annotation
So far, we’ve used mypy only to annotate function parameters and return types. It is natural to extend this to any type of variables:
int_var: int = 0
float_var: float = 1.5
str_var: str = "hello"
However, this is used a bit less (and doesn’t apply in the strict version of mypy either), since the types of variables are mostly clear from the context. Typically you would only do this when the code is relatively ambiguous and difficult to read.
In this section, we’ll discuss class annotation, but also annotation with your own class and other complex classes.
annotation classes
Class annotation can be handled fairly quickly: just annotate the class functions like any other function, but don’t annotate the argument itself in the constructor:
class SampleClass:
def __init__(self, x: int) -> None:
self.x = xdef get_x(self) -> int:
return self.x
sample_class = SampleClass(5)
print(sample_class.get_x())
Annotating with custom/complex classes
With our class defined, we can now use its name like any other type annotation:
sample_class: SampleClass = SampleClass(5)
In fact, mypy works with most classes and types out of the box, for example:
import pathlibdef read_file(path: pathlib.Path) -> str:
with open(path, "r") as file:
return file.read()
print(read_file(pathlib.Path("mypy.ini")))
In this section, we’ll see how to deal with external libraries that aren’t writable and selectively disable type checking for certain lines that cause problems, building on a slightly more complex example involving numb and matplotlib.
Let’s start with a first version of the code:
import matplotlib.pyplot as plt
import numpy as np
import numpy.typing as nptdef calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x)
x = np.linspace(0, 10, 100)
y = calc_np_sin(x)
plt.plot(x, y)
plt.savefig("plot.png")
We define a simple function that calculates the sine of a numpy array and apply it to the input values xspanning space [0, 10]. Then, we plot the sinus curve using matplotlib.
In this code, we also see the correct writing of numpy arrays using numpy.typing.
However, if we run mypy on this, we’ll get two errors. The first is:
error: Return any of the function declared to return “ndarray[Any, dtype[floating[_32Bit]]]”
This is a relatively common pattern in mypy. Actually, we didn’t do anything wrong, but mypy would like to make it a bit more explicit, and here, as in other situations, we need to “force” mypy to accept our code. We can do this, for example, by introducing a proxy variable of the correct type:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
y: npt.NDArray[np.float32] = np.sin(x)
return y
The next error is:
error: Skip parsing “matplotlib”: module found but no type hints or library stubs
This is because matplotlib It’s not written (yet). So we need to tell mypy to exclude it from verification. We do this by adding the following to our mypy.ini file:
[mypy-matplotlib.*]
ignore_missing_imports = True
ignore_errors = True
Lastly, note that you can also selectively ignore any line of code by adding # type: ignore it. Do this, if there really is an unsolvable problem with mypy, or if you want to silence some known but irrelevant warnings/errors. We could also have hidden our first error above through this:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x) # type: ignore
In this post, we introduce mypy, which is a static type checker for Python. Using mypy, we can (and should) annotate variable types, parameters, and return values, giving us a way to check the sanity of our program at compile time. mypy is very widespread and is recommended for any medium-large software project.
We start by installing and configuring mypy. Next, we introduce how to annotate primitive and complex types, such as lists, dicts, or arrays. Next, we discuss other important annotators, such as Union, Optional, Noneeither Any. Eventually, we show that mypy supports a wide range of complex types, such as custom classes. We end the tutorial by showing how to debug and fix mypy errors.
That’s it for mypy. I hope you liked this post, thanks for reading!
