"async with" in Python 3.4

Question

The Getting Started docs for aiohttp give the following client example:

import asyncio
import aiohttp

async def fetch_page(session, url):
    with aiohttp.Timeout(10):
        async with session.get(url) as response:
            assert response.status == 200
            return await response.read()

loop = asyncio.get_event_loop()
with aiohttp.ClientSession(loop=loop) as session:
    content = loop.run_until_complete(
        fetch_page(session, 'http://python.org'))
    print(content)

And they give the following note for Python 3.4 users:

If you are using Python 3.4, please replace await with yield from and
async def with a @coroutine decorator.

If I follow these instructions I get:

import aiohttp
import asyncio

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(10):
        async with session.get(url) as response:
            return (yield from response.text())

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    with aiohttp.ClientSession(loop=loop) as session:
        html = loop.run_until_complete(
            fetch(session, 'http://python.org'))
        print(html)

However, this will not run, because async with is not supported in Python 3.4:

$ python3 client.py 
  File "client.py", line 7
    async with session.get(url) as response:
             ^
SyntaxError: invalid syntax

How can I translate the async with statement to work with Python 3.4?

Asked By: Imran

||

Source

Answer 1

Just don’t use the result of session.get() as a context manager; use it as a coroutine directly instead. The request context manager that session.get() produces would normally release the request on exit, but so does using response.text(), so you could ignore that here:

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(10):
        response = yield from session.get(url)
        return (yield from response.text())

The request wrapper returned here doesn’t have the required asynchronous methods (__aenter__ and __aexit__), they omitted entirely when not using Python 3.5 (see the relevant source code).

If you have more statements between the session.get() call and accessing the response.text() awaitable, you probably want to use a try:..finally: anyway to release the connection; the Python 3.5 release context manager also closes the response if an exception occurred. Because a yield from response.release() is needed here, this can’t be encapsulated in a context manager before Python 3.4:

import sys

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(10):
        response = yield from session.get(url)
        try:
            # other statements
            return (yield from response.text())
        finally:
            if sys.exc_info()[0] is not None:
                # on exceptions, close the connection altogether
                response.close()
            else:
                yield from response.release()

Answered By: Martijn Pieters

Answer 2

aiohttp‘s examples implemented using 3.4 syntax. Based on json client example your function would be:

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(10):
        resp = yield from session.get(url)
        try:
            return (yield from resp.text())
        finally:
            yield from resp.release()

Upd:

Note that Martijn’s solution would work for simple cases, but may lead to unwanted behavior in specific cases:

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(5):
        response = yield from session.get(url)

        # Any actions that may lead to error:
        1/0

        return (yield from response.text())

# exception + warning "Unclosed response"

Besides exception you’ll get also warning “Unclosed response”. This may lead to connections leak in complex app. You will avoid this problem if you’ll manually call resp.release()/resp.close():

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(5):
        resp = yield from session.get(url)
        try:

            # Any actions that may lead to error:
            1/0

            return (yield from resp.text())
        except Exception as e:
            # .close() on exception.
            resp.close()
            raise e
        finally:
            # .release() otherwise to return connection into free connection pool.
            # It's ok to release closed response:
            # https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/client_reqrep.py#L664
            yield from resp.release()

# exception only

I think it’s better to follow official examples (and __aexit__ implementation) and call resp.release()/resp.close() explicitly.

Answered By: Mikhail Gerasimov

"async with" in Python 3.4

Question:

Answers: