In this post, we will be taking on Python type hinting and discuss the subtle differences between
Any types and how static type checkers behave based on our instructions.
There are a few static type checkers for Python like
pyright to name a few, that help bring the assurance of static typing over to Python while retaining the convenience of dynamic typing. We picked
mypy to illustrate our examples here.
Generally, we see the
Any type used pretty often, even when
object would be better. In this post, we are going to focus on the difference between
Any when utilizing Python type hinting and static type checking.
A primer on type hinting and static type checkers
At its core, Python is a dynamically or duck typed language. But as your projects grow and more people start working on the same project, static typing becomes more attractive.
Starting from Python 3.5, we have the option to statically indicate the type of a value in our Python code. Or in other words, type hint it.
Let’s see how we can type hint a variable that holds an integer and then assign the value 42 to it:
>>> my_integer: int >>> my_integer = 42 # we can also do it in one line: >>> my_integer: int = 42
From now on, whoever sees that variable, either by first locating its declaration or by hovering her mouse on it in the IDE, knows it is supposed to be an integer.
Nevertheless, type hinting on its own will not enforce any integrity. In the example above, we could assign a string to the
my_integer variable and would not face any errors. That’s why we require some additional help from static type checkers.
Static type checkers like
mypy leverage type hinting and take on the responsibility of evaluating our code and making sure the assigned values to variables are what we intended them to be.
Benefits of incorporating type hinting and static type checkers
Below are some of the advantages that static typing brings to the table:
- Improves readability by making function/method signatures more descriptive, so you can often see a method’s intent at a glance.
- Keeps your codebase clean and less prone to having bugs.
- Reduces human error
- Supercharges your IDE
Let’s see how much of a difference it makes when we use one type over the other.
object vs Any
Since type hints and static type checking are additions to Python, it is common for projects to adopt them gradually. At Inmanta, we try to maximize type coverage but there are still parts of the codebase that are untyped. This gradual adoption is where the
Any type comes in.
The purpose of the
Any type is to indicate to the type checker that a part of the program should not be checked. A variable (or function parameter) that is annotated with the
Any type accepts any value, and the type checker allows any operation on it.
On the other way around, a variable of any type accepts an object of type
Any. This means the following snippet will type check correctly, even though it will not run correctly:
from typing import Any my_int: Any = "This is a string!" my_int.my_method() my_bool: bool = my_int
Annotating parts of your program with
Any can be useful to have static checking without getting a whole bunch of type errors for the sections that haven’t been annotated yet. The semantics of the
Any type outlined above make sure the type checker never raises an error for an
Any typed variable/value.
However, there is a pitfall: due to its name, we have found that the semantics are often misinterpreted. Aside from being used with the intent to blind the type checker to a part of the program, it is often used when the programmer wants to indicate that a variable/parameter accepts any object.
While this may sound similar at first sight, there are some very important differences. When the
Any type is used for situations like this, even though it wasn’t the intention, it still indicates to the type checker not to bother with that part of the code involving that variable/parameter.
Below, we will supply an example scenario, illustrating the issues that might arise when using
Any and propose an alternative that matches closely with what the programmer intends.
Consider a simple (and rather redundant) print function:
def myprint(obj): print(obj)
The function is meant to handle any value we pass to it. A naive approach to adding type annotations would then be:
def myprint(obj: Any) -> None: print(obj)
At the moment the consequences of this misinterpretation of the semantics of
Any are limited. But now suppose we decide to expand on the behavior of our print function. We don’t want it to just print the object, but to say hello to it as well.
Easy! A simple string concatenation will do:
def myprint(obj: Any) -> None: print("hello " + obj) myprint("world!") myprint(42)
Let’s type check it:
/home/Python$ mypy main.py Success: no issues found in 1 source file
The type checker reports no errors. Great! Let’s give it a go:
/home/Python$ python main.py "hello world!" Traceback (most recent call last): File "/home/Python/main.py", line 9, in <module> myprint(42) File "/home/Python/main.py", line 5, in myprint print("hello " + obj) TypeError: can only concatenate str (not "int") to str
Why didn’t the type checker warn us about this? Because we inadvertently told it not to!
obj parameter is of type
Any, so the type checker allows us to do anything we feel like with it, for instance using the
+ operator on it. This was not the intention when adding type annotations; the goal is to have the type checker catch mistakes like this, not hide them behind an
The message we want to send to the type checker is that
obj might be a value of any type and we don’t know which, so please don’t make any assumptions. Instead, we’ve sent: it can be of any type, please assume anything I do with it is valid, because it could be.
object to the rescue
So, we have determined that the
Any type is not appropriate in this case. The
object type on the other hand is exactly what we need. It is the supertype of all types, so if we write:
def myprint(obj: object) -> None: print("hello " + obj) myprint("world!") myprint(42)
And run the type checker:
/home/Python$ mypy main.py main.py:4: error: Unsupported operand types for + ("str" and "object") Found 1 error in 1 file (checked 1 source file)
The type checker is still satisfied with us passing it “world!” and 42 because both
int are subtypes of
object. Now, contrarily, it tells us Unsupported operand types for + (“str” and “object”), discovering the mistake we accidentally put in our program.
Let’s backtrack a bit and have another look at a snippet we looked at earlier, this time using
/home/Python$ cat main.py my_int: object = "This is a string!" my_int.my_method() my_bool: bool = my_int
/home/Python$ mypy main.py main.py:2: error: "object" has no attribute "my_method" main.py:3: error: Incompatible types in assignment (expression has type "object", variable has type "bool") Found 2 errors in 1 file (checked 1 source file)
The first line still type checks correctly: a
string is an
object so that assignment is fine.
The next two lines give type errors:
my_int can be any object, and not all objects have a
my_method method, nor are all objects booleans.
The same is true for function return values. If a function is typed as returning
Any, the type checker will not make any checks on what the caller does with it.
How does it affect us?
At Inmanta we place a great emphasis on type hinting our codebase. We closely monitor the coverage of our codebase and constantly improve it. Also, we use a handful of open-source tools, and whenever possible we contribute back to the community.
During one of our sessions revolving around the type hinting of the Python logging module in Typeshed, we observed the use of type
Any in places where utilizing the
object type would be more appropriate. It motivated us to make a contribution to the Typeshed project to improve the typing of the logging module.
The improvement of type coverage introduced by this change is evident in the graph below:
If you want to learn more about this change, you can find our pull request here.
This post illustrated the nuances of Python type hinting. The
Any type is very useful for gradually moving from an untyped codebase to one with type annotations used for static type checking. It hides untyped parts of the program from the type checker. In practice, however, it’s often (mis)used in situations where the intention is to state that a variable should accept any type of value. This has the side effect of hiding that part of the program for the type checker as well. It results in a weaker guarantee provided by the type checker.
The solution is to use
object instead. Rather than hiding a part of the program from the type checker, it signals that a variable can be of any type. So anything we do with it, has to be sufficiently generic that we can apply to all types.
Special thanks to Sander Van Balen and Wouter De Borger for co-authoring this post.