asyncio.ensure_future vs. BaseEventLoop.create_task vs. simple coroutine?
Question:
I’ve seen several basic Python 3.5 tutorials on asyncio doing the same operation in various flavours.
In this code:
import asyncio
async def doit(i):
print("Start %d" % i)
await asyncio.sleep(3)
print("End %d" % i)
return i
if __name__ == '__main__':
loop = asyncio.get_event_loop()
#futures = [asyncio.ensure_future(doit(i), loop=loop) for i in range(10)]
#futures = [loop.create_task(doit(i)) for i in range(10)]
futures = [doit(i) for i in range(10)]
result = loop.run_until_complete(asyncio.gather(*futures))
print(result)
All the three variants above that define the futures variable achieve the same result; the only difference I can see is that with the third variant the execution is out of order (which should not matter in most cases). Is there any other difference? Are there cases where I can’t just use the simplest variant (plain list of coroutines)?
Answers:
create_task()
- accepts coroutines,
- returns Task,
- it is invoked in context of the loop.
ensure_future()
- accepts Futures, coroutines, awaitable objects,
- returns Task (or Future if Future passed).
- if the given arg is a coroutine it uses
create_task,
- loop object can be passed.
As you can see the create_task is more specific.
async function without create_task or ensure_future
Simple invoking async function returns coroutine
>>> async def doit(i):
... await asyncio.sleep(3)
... return i
>>> doit(4)
<coroutine object doit at 0x7f91e8e80ba0>
And since the gather under the hood ensures (ensure_future) that args are futures, explicitly ensure_future is redundant.
Similar question What's the difference between loop.create_task, asyncio.async/ensure_future and Task?
Actual info:
Starting from Python 3.7 asyncio.create_task(coro) high-level function was added for this purpose.
You should use it instead other ways of creating tasks from coroutimes. However if you need to create task from arbitrary awaitable, you should use asyncio.ensure_future(obj).
Old info:
ensure_future vs create_task
ensure_future is a method to create Task from coroutine. It creates tasks in different ways based on argument (including using of create_task for coroutines and future-like objects).
create_task is an abstract method of AbstractEventLoop. Different event loops can implement this function different ways.
You should use ensure_future to create tasks. You’ll need create_task only if you’re going to implement your own event loop type.
Upd:
@bj0 pointed at Guido’s answer on this topic:
The point of ensure_future() is if you have something that could
either be a coroutine or a Future (the latter includes a Task because
that’s a subclass of Future), and you want to be able to call a method
on it that is only defined on Future (probably about the only useful
example being cancel()). When it is already a Future (or Task) this
does nothing; when it is a coroutine it wraps it in a Task.
If you know that you have a coroutine and you want it to be scheduled,
the correct API to use is create_task(). The only time when you should
be calling ensure_future() is when you are providing an API (like most
of asyncio’s own APIs) that accepts either a coroutine or a Future and
you need to do something to it that requires you to have a Future.
and later:
In the end I still believe that ensure_future() is an appropriately
obscure name for a rarely-needed piece of functionality. When creating
a task from a coroutine you should use the appropriately-named
loop.create_task(). Maybe there should be an alias for that
asyncio.create_task()?
It’s surprising to me. My main motivation to use ensure_future all along was that it’s higher-level function comparing to loop’s member create_task (discussion contains some ideas like adding asyncio.spawn or asyncio.create_task).
I can also point that in my opinion it’s pretty convenient to use universal function that can handle any Awaitable rather than coroutines only.
However, Guido’s answer is clear: “When creating a task from a coroutine you should use the appropriately-named loop.create_task()“
When coroutines should be wrapped in tasks?
Wrap coroutine in a Task – is a way to start this coroutine “in background”. Here’s example:
import asyncio
async def msg(text):
await asyncio.sleep(0.1)
print(text)
async def long_operation():
print('long_operation started')
await asyncio.sleep(3)
print('long_operation finished')
async def main():
await msg('first')
# Now you want to start long_operation, but you don't want to wait it finised:
# long_operation should be started, but second msg should be printed immediately.
# Create task to do so:
task = asyncio.ensure_future(long_operation())
await msg('second')
# Now, when you want, you can await task finised:
await task
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
first
long_operation started
second
long_operation finished
You can replace asyncio.ensure_future(long_operation()) with just await long_operation() to feel the difference.
for your example, all the three types execute asynchronously. the only difference is that, in the third example, you pre-generated all 10 coroutines, and submitted to the loop together. so only the last one gives output randomly.
Note: Only valid for Python 3.7 (for Python 3.5 refer to the earlier answer).
From the official docs:
asyncio.create_task (added in Python 3.7) is the preferable way for spawning new tasks instead of ensure_future().
Detail:
So now, in Python 3.7 onwards, there are 2 top-level wrapper function (similar but different):
asyncio.create_task: which simply call event_loop.create_task(coro) directly. (see source code)ensure_future which also call event_loop.create_task(coro) if it is coroutine or else it is simply to ensure the return type to be a asyncio.Future. (see source code). Anyway, Task is still a Future due to its class inheritance (ref).
Well, utlimately both of these wrapper functions will help you call BaseEventLoop.create_task. The only difference is ensure_future accept any awaitable object and help you convert it into a Future. And also you can provide your own event_loop parameter in ensure_future. And depending if you need those capability or not, you can simply choose which wrapper to use.
I’ve seen several basic Python 3.5 tutorials on asyncio doing the same operation in various flavours.
In this code:
import asyncio
async def doit(i):
print("Start %d" % i)
await asyncio.sleep(3)
print("End %d" % i)
return i
if __name__ == '__main__':
loop = asyncio.get_event_loop()
#futures = [asyncio.ensure_future(doit(i), loop=loop) for i in range(10)]
#futures = [loop.create_task(doit(i)) for i in range(10)]
futures = [doit(i) for i in range(10)]
result = loop.run_until_complete(asyncio.gather(*futures))
print(result)
All the three variants above that define the futures variable achieve the same result; the only difference I can see is that with the third variant the execution is out of order (which should not matter in most cases). Is there any other difference? Are there cases where I can’t just use the simplest variant (plain list of coroutines)?
create_task()
- accepts coroutines,
- returns Task,
- it is invoked in context of the loop.
ensure_future()
- accepts Futures, coroutines, awaitable objects,
- returns Task (or Future if Future passed).
- if the given arg is a coroutine it uses
create_task, - loop object can be passed.
As you can see the create_task is more specific.
async function without create_task or ensure_future
Simple invoking async function returns coroutine
>>> async def doit(i):
... await asyncio.sleep(3)
... return i
>>> doit(4)
<coroutine object doit at 0x7f91e8e80ba0>
And since the gather under the hood ensures (ensure_future) that args are futures, explicitly ensure_future is redundant.
Similar question What's the difference between loop.create_task, asyncio.async/ensure_future and Task?
Actual info:
Starting from Python 3.7 asyncio.create_task(coro) high-level function was added for this purpose.
You should use it instead other ways of creating tasks from coroutimes. However if you need to create task from arbitrary awaitable, you should use asyncio.ensure_future(obj).
Old info:
ensure_future vs create_task
ensure_future is a method to create Task from coroutine. It creates tasks in different ways based on argument (including using of create_task for coroutines and future-like objects).
create_task is an abstract method of AbstractEventLoop. Different event loops can implement this function different ways.
You should use ensure_future to create tasks. You’ll need create_task only if you’re going to implement your own event loop type.
Upd:
@bj0 pointed at Guido’s answer on this topic:
The point of
ensure_future()is if you have something that could
either be a coroutine or aFuture(the latter includes aTaskbecause
that’s a subclass ofFuture), and you want to be able to call a method
on it that is only defined onFuture(probably about the only useful
example beingcancel()). When it is already aFuture(orTask) this
does nothing; when it is a coroutine it wraps it in aTask.If you know that you have a coroutine and you want it to be scheduled,
the correct API to use iscreate_task(). The only time when you should
be callingensure_future()is when you are providing an API (like most
of asyncio’s own APIs) that accepts either a coroutine or aFutureand
you need to do something to it that requires you to have aFuture.
and later:
In the end I still believe that
ensure_future()is an appropriately
obscure name for a rarely-needed piece of functionality. When creating
a task from a coroutine you should use the appropriately-named
loop.create_task(). Maybe there should be an alias for that
asyncio.create_task()?
It’s surprising to me. My main motivation to use ensure_future all along was that it’s higher-level function comparing to loop’s member create_task (discussion contains some ideas like adding asyncio.spawn or asyncio.create_task).
I can also point that in my opinion it’s pretty convenient to use universal function that can handle any Awaitable rather than coroutines only.
However, Guido’s answer is clear: “When creating a task from a coroutine you should use the appropriately-named loop.create_task()“
When coroutines should be wrapped in tasks?
Wrap coroutine in a Task – is a way to start this coroutine “in background”. Here’s example:
import asyncio
async def msg(text):
await asyncio.sleep(0.1)
print(text)
async def long_operation():
print('long_operation started')
await asyncio.sleep(3)
print('long_operation finished')
async def main():
await msg('first')
# Now you want to start long_operation, but you don't want to wait it finised:
# long_operation should be started, but second msg should be printed immediately.
# Create task to do so:
task = asyncio.ensure_future(long_operation())
await msg('second')
# Now, when you want, you can await task finised:
await task
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
first
long_operation started
second
long_operation finished
You can replace asyncio.ensure_future(long_operation()) with just await long_operation() to feel the difference.
for your example, all the three types execute asynchronously. the only difference is that, in the third example, you pre-generated all 10 coroutines, and submitted to the loop together. so only the last one gives output randomly.
Note: Only valid for Python 3.7 (for Python 3.5 refer to the earlier answer).
From the official docs:
asyncio.create_task(added in Python 3.7) is the preferable way for spawning new tasks instead ofensure_future().
Detail:
So now, in Python 3.7 onwards, there are 2 top-level wrapper function (similar but different):
asyncio.create_task: which simply callevent_loop.create_task(coro)directly. (see source code)ensure_futurewhich also callevent_loop.create_task(coro)if it is coroutine or else it is simply to ensure the return type to be a asyncio.Future. (see source code). Anyway,Taskis still aFuturedue to its class inheritance (ref).
Well, utlimately both of these wrapper functions will help you call BaseEventLoop.create_task. The only difference is ensure_future accept any awaitable object and help you convert it into a Future. And also you can provide your own event_loop parameter in ensure_future. And depending if you need those capability or not, you can simply choose which wrapper to use.