Python type narrowing using TypeGuard

🏠 Go home
Mypy Python Typing 2022-08-09

Type narrowing is a technique we use to instruct the type-checker (mypy in my case) that in certain scope the variable of a broader type A has actually a more specific type B. For example if we allow users of an API to provide as an input two different JSONs, let's say something like this.

from pydantic import BaseModel

class SearchByIdRequest(BaseModel):
    id: int

class SearchByNameRequest(BaseModel):
    name: str

class SearchResponse(BaseModel):
    id: int
    name: str

SearchRequest = SearchByIdRequest | SearchByNameRequest

And let's assume that at some point in the call-stack of our application we need to distinguish which request we are processing. For the sake of completeness, let's assume following exception and functions are defined:

class UnexpectedRequestError(Exception):
    pass

def search_by_id(id: int) -> list[SearchResponse]:
    raise NotImplemented

def search_by_name(name: str) -> list[SearchResponse]:
    raise NotImplemented

The DB handler function might look as follows.

def is_by_id_request(request: SearchRequest) -> bool:
    return isinstance(request, SearchByIdRequest)

def is_by_name_request(request: SearchRequest) -> bool:
    return isinstance(request, SearchByNameRequest)

def run_database_query(request: SearchRequest) -> list[SearchResponse]:
    if is_by_id_request(request):
        return search_by_id(request.id)
    elif is_by_name_request(request):
        return search_by_name(request.name)

    raise UnexpectedRequestError

Such a code works but mypy will be very unsatisfied and as result will produce following errors.

typguard_example.py:32: error: Item "SearchByNameRequest" of "Union[SearchByIdRequest, SearchByNameRequest]" has no attribute "id"
typguard_example.py:34: error: Item "SearchByIdRequest" of "Union[SearchByIdRequest, SearchByNameRequest]" has no attribute "name"
Found 2 errors in 1 file (checked 1 source file)

Actually, if we didn't hide isinstance checks in a function, mypy would be happy because isinstance is one of supported expressions for type narrowing. But we are responsible developers, we follow SLAP and give our expressions meaningful names by hiding them into nice, small and composable functions.

We can instruct mypy to type-check the code by making our is_by_name_request and is_by_id_request functions behave the same way the isinstance (or issubclass, type and callable) does. This is exactly what TypeGuard type constructor is for. It is available in Python >=3.10 typing module or in typing_extensions module.

from typing import TypeGuard

def is_by_id_request(request: SearchRequest) -> TypeGuard[SearchByIdRequest]:
    return isinstance(request, SearchByIdRequest)

def is_by_name_request(request: SearchRequest) -> TypeGuard[SearchByNameRequest]:
    return isinstance(request, SearchByNameRequest)

Now, try to hover over the request variable in your IDE / editor with mypy enabled.

...
    if is_by_id_request(request):
        return search_by_id(request.id)  # <- mypy will say this request is SearchByIdRequest
...

You can think of TypeGuard[T] being a special, magical and unicornish version of basic bool. During the runtime, it still produces bool and mypy doesn't have any problems accepting result of narrowing function on places where bool is expected. For example, the following code type-checks.

def i_need_bool(is_it_true_bro: bool) -> None:
    pass

request = SearchByIdRequest(id=69)

i_need_bool(is_by_id_request(request))

For the given example, Python 3.10's match also works

Let's just note that in Python 3.10, it is possible to solve the problem using match statement. Big advantage of this approach is that mypy checks for exhaustiveness and it produces an error if the match doesn't handle all cases. Therefore, we can omit the raise statement.

def run_database_query_differently(request: SearchRequest) -> list[SearchResponse]:
    match request: 
        case SearchByNameRequest(name=name):
            return search_by_name(name)
        case SearchByIdRequest(id=id):
            return search_by_id(id)

Be careful with type narrowing!

Be very careful, mypy doesn't enforce any relation between the input and narrowed type. Also you are in charge of defining the condition correctly.

def narrow_the_type(v: str) -> TypeGuard[dict]:
    return True

value = "test"

if narrow_the_type(value):
    print(value.items())

Mypy is happy, you are happy but the Python interpreter is not!

AttributeError: 'str' object has no attribute 'items' 

Also, it is possible to accept more arguments in the narrowing function, the narrowing will apply for the first argument only.