Python type narrowing using TypeGuard
🏠 Go home
Type narrowing is a technique we use to instruct the type-checker (mypy
in my case) that in certain scope the variable of a broader type A
has actually a more specific type B
. For example if we allow users of an API to provide as an input two different JSONs, let's say something like this.
from pydantic import BaseModel
class SearchByIdRequest(BaseModel):
id: int
class SearchByNameRequest(BaseModel):
name: str
class SearchResponse(BaseModel):
id: int
name: str
SearchRequest = SearchByIdRequest | SearchByNameRequest
And let's assume that at some point in the call-stack of our application we need to distinguish which request we are processing. For the sake of completeness, let's assume following exception and functions are defined:
class UnexpectedRequestError(Exception):
pass
def search_by_id(id: int) -> list[SearchResponse]:
raise NotImplemented
def search_by_name(name: str) -> list[SearchResponse]:
raise NotImplemented
The DB handler function might look as follows.
def is_by_id_request(request: SearchRequest) -> bool:
return isinstance(request, SearchByIdRequest)
def is_by_name_request(request: SearchRequest) -> bool:
return isinstance(request, SearchByNameRequest)
def run_database_query(request: SearchRequest) -> list[SearchResponse]:
if is_by_id_request(request):
return search_by_id(request.id)
elif is_by_name_request(request):
return search_by_name(request.name)
raise UnexpectedRequestError
Such a code works but mypy will be very unsatisfied and as result will produce following errors.
typguard_example.py:32: error: Item "SearchByNameRequest" of "Union[SearchByIdRequest, SearchByNameRequest]" has no attribute "id"
typguard_example.py:34: error: Item "SearchByIdRequest" of "Union[SearchByIdRequest, SearchByNameRequest]" has no attribute "name"
Found 2 errors in 1 file (checked 1 source file)
Actually, if we didn't hide isinstance
checks in a function, mypy would be happy because isinstance
is one of supported expressions for type narrowing. But we are responsible developers, we follow SLAP and give our expressions meaningful names by hiding them into nice, small and composable functions.
We can instruct mypy
to type-check the code by making our is_by_name_request
and is_by_id_request
functions behave the same way the isinstance
(or issubclass
, type
and callable
) does. This is exactly what TypeGuard
type constructor is for. It is available in Python >=3.10 typing
module or in typing_extensions
module.
from typing import TypeGuard
def is_by_id_request(request: SearchRequest) -> TypeGuard[SearchByIdRequest]:
return isinstance(request, SearchByIdRequest)
def is_by_name_request(request: SearchRequest) -> TypeGuard[SearchByNameRequest]:
return isinstance(request, SearchByNameRequest)
Now, try to hover over the request
variable in your IDE / editor with mypy enabled.
...
if is_by_id_request(request):
return search_by_id(request.id) # <- mypy will say this request is SearchByIdRequest
...
You can think of TypeGuard[T]
being a special, magical and unicornish version of basic bool
. During the runtime, it still produces bool and mypy doesn't have any problems accepting result of narrowing function on places where bool is expected. For example, the following code type-checks.
def i_need_bool(is_it_true_bro: bool) -> None:
pass
request = SearchByIdRequest(id=69)
i_need_bool(is_by_id_request(request))
For the given example, Python 3.10's match
also works
Let's just note that in Python 3.10, it is possible to solve the problem using match
statement. Big advantage of this approach is that mypy checks for exhaustiveness and it produces an error if the match doesn't handle all cases. Therefore, we can omit the raise statement.
def run_database_query_differently(request: SearchRequest) -> list[SearchResponse]:
match request:
case SearchByNameRequest(name=name):
return search_by_name(name)
case SearchByIdRequest(id=id):
return search_by_id(id)
Be careful with type narrowing!
Be very careful, mypy doesn't enforce any relation between the input and narrowed type. Also you are in charge of defining the condition correctly.
def narrow_the_type(v: str) -> TypeGuard[dict]:
return True
value = "test"
if narrow_the_type(value):
print(value.items())
Mypy is happy, you are happy but the Python interpreter is not!
AttributeError: 'str' object has no attribute 'items'
Also, it is possible to accept more arguments in the narrowing function, the narrowing will apply for the first argument only.