Exception handling in Python asyncio code

Question

I am running the following example:

import asyncio


async def longwait(n):
    print('doing long wait')
    await asyncio.sleep(n)


async def main():
    long_task = asyncio.create_task(longwait(10))
    try:
        result = await asyncio.wait_for(asyncio.shield(long_task), timeout=1)  # line 'a'
        print(result)
    except asyncio.exceptions.TimeoutError:
        print('Taking longer than usual. Please wait.')
        result = await long_task  # line 'b'
        print(result)


asyncio.run(main())

# why not use the `result` from try block?
# maybe coz we tried to print result before it was assigned any value in try block?

What I don’t get is why we are writing line ‘b’ when we already have line ‘a’ where we are assigning a value to the result variable. We can directly use the result from try clause, no?

My reasoning is that if TimeoutError exception is raised before the future in line ‘a’ is completed, it is immediately handled in the except clause. The except clause will try to print result. If we don’t use line ‘b’ and since no value was assigned to result in the try clause, the except clause will raise UnboundLocalError. Is this correct reasoning? Or is there something I am missing?

Asked By: mayankkaizen

||

Source

Answer 1

[…] we already have line ‘a’ where we are assigning a value to the result variable.

Not really. To be more precise, we are doing multiple things in line a in that order:

First, we are calling asyncio.shield(long_task), which returns a new Future (let’s name it f).
Next, we are calling asyncio.wait_for with f, which returns a new coroutine object (let’s name it c).
Then, we are awaiting c, which returns whatever the underlying function wait_for returns. (Which in this case is what shield would return, which in turn is just what long_task returns.)
Lastly, we assign that return value from step 3 to the name result (which happens to be local to the main function).

As is the case with any combination of Python expressions/assignments, if any preceding operation raises an exception, the interpreter does not proceed with the following operations. The try–except-construct allows us to define what happens after that. But it does not change the fact that everything starting from the line that caused the exception up until our except-block is completely disregarded.

[…] TimeoutError exception is raised before the future in line ‘a’ is completed

Yes, but you need to be specific about which future you are talking about. In this case wait_for does in fact complete (with the exception). And before raising it cancels future f (returned by shield), but long_task (having been shielded from cancellation) is not done at this point.

In this concrete example, during step 3 the TimeoutError exception is raised. This means that whatever comes after that, i.e. the variable assignment in step 4 is not executed.

[…] since no value was assigned to result in the try clause, the except clause will raise UnboundLocalError.

Exactly. At runtime, the result name is never created in the try-block.

And this is what I think @user2357112 meant by everything relevant being the same with synchronous code.

It is just that in this case you may get confused by the nested futures because long_task is still running despite the exception having been raised. Nothing cancelled it, so in our except-block we can treat it the same as before. Thus we can for example await it directly (as you did) and assign its output to result. This is something we could not do in synchronous code of course.

PS

Regarding your follow-up question, yes, long_task is still the same task. Its execution began earlier, when you called create_task and it remained unaffected by the exception. wait_for actually cancels the task it was passed, if the timeout is reached, but this did not affect long_task because you shielded it from cancellation. Of course, if you don’t catch the exception, but allow it to bubble up, that will interrupt the entire program (including the event loop).

You can actually test fairly easily, that you are dealing with the same task:

from asyncio import create_task, run, shield, sleep, wait_for
from asyncio.exceptions import TimeoutError


async def wait(n: int) -> str:
    print("Starting to wait")
    for i in range(n):
        await sleep(1)
        print(f"Waited {i + 1} sec")
    print("Done waiting")
    return "foo"


async def main() -> None:
    long_task = create_task(wait(5))
    try:
        result = await wait_for(shield(long_task), timeout=1.5)
    except TimeoutError:
        print("Taking longer than usual. Please wait.")
        result = await long_task
    print(result)


if __name__ == "__main__":
    run(main())

Output:

Starting to wait
Waited 1 sec
Taking longer than usual. Please wait.
Waited 2 sec
Waited 3 sec
Waited 4 sec
Waited 5 sec
Done waiting
foo

Answered By: Daniil Fajnberg

Exception handling in Python asyncio code

Question:

Answers:

PS