Asyncio on long running task

Question

I just learned async with python 3.5 yesterday.

here is what I want to accomplish today.

import asyncio  
import time
import requests

async def foo():
    """Some Long Running Taks"""
    requests.get("http://stackoverflow.com/questions/41301031/asyncio-on-long-running-task")
    print("foo")

async def bar():
    """Some Quick Task"""
    print("bar")

while True:
    loop = asyncio.get_event_loop()
    loop.run_until_complete(asyncio.gather(foo(), bar()))
    loop.close()
    time.sleep(2)

#Expected output
"""
>>>bar
>>>bar
>>>bar
>>>bar
>>>bar
>>>foo
and so on
"""

Is this possible using python async/await?

Asked By: Rahul

||

Source

Answer 1

your time.sleep() is blocking; you need to use asyncio.sleep().

and why do you run your loop inside while True?

also note that asyncio will only help speedup io-bound tasks (e.g. get data from the net). it will not help you speed up cpu-bound tasks. for that you need threads or multiplrocessing.

seeing that you have updated your question, here a small update in my answer: also requests is blocking and does not play nice with asyncio. aiohttp is similiar in its functionality but does play nice with asyncio

Answered By: hiro protagonist

Answer 2

You have a few issues in your code:

requests does not support asyncio, use aiohttp instead.
Your couroutines will run only when the loop is running: Call loop.run_until_complete() only once in your code, and schedule many (short or long) tasks using await (or asycio.ensure_future() or loop.create_task()).

Here is an example doing something similar to what you were trying to do:

# make sure to run `pip install aiohttp` first!!!

import asyncio

import aiohttp


async def slow_fetch(delay):
    url = "http://httpbin.org/delay/{}".format(delay)
    print("Getting ", url)
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            text = await resp.text()
    print("Got {}, {} chars".format(url, len(text)))


async def quick_loop():
    print("enter quick_loop")
    for i in range(10):
        await asyncio.sleep(1)
        print("quick_loop", i)
    print("exit quick_loop")


loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
    slow_fetch(3),
    slow_fetch(4),
    quick_loop(),
))
loop.close()

Answered By: Udi

Answer 3

Here’s a better approach to the ensure_future / create_task:
USE A DECORATOR

def async_wrap(func):
    @wraps(func)
    async def run(*args, loop=None, executor=None, **kwargs):
        if loop is None:
            loop = asyncio.get_event_loop()
        pfunc = partial(func, *args, **kwargs)
        return await loop.run_in_executor(executor, pfunc)

    return run

Now if you have a long running function like some function that uses requests you can do this:

@async_wrap
def long_running_function():
    return requests.get(_URL)

Answered By: ArminMz

Asyncio on long running task

Question:

Answers: