Tulip/asyncIO: why not all calls be async and specify when things should be synchronous?
Question:
I went to the SF Python meetup when Guido talked about Tulip, the future asyncIO library for asynchronous operations in Python.
The take away is that if you want something to be run asynchronously you can use the "yield from" + expression and a couple of decorators to specify that the call to what comes after yield from should be executed asynchronously. The nice thing about it is that you can read the statements in that function normally (as if it was synchronous) and it will behave as if it was synchronous with respect to the execution of that function (return values and error/exception propagation and handling).
My question is: why not have the opposite behavior, namely, have all function calls be by default async (and without the yield from) and have a different explicit syntax when you want to execute something synchronously?
(besides the need for another keyword/syntax spec)
Answers:
Note that the possible uses of yield from are a small part of the asynch PEP, and never need to be used. Maybe Guido oversold them in his talk 😉
As to why functions aren’t being changed to always be async by default, that’s just realism. Asynch gimmicks bring new overheads and semantic complications, and Python isn’t going to slow down and complicate life for everyone to make a few applications easier to write.
In short, “practicality beats purity” 😉
The real answer is that Guido likes the fact that asynchronous yield points are explicit in coroutines, because if you don’t realize that a call can yield, then that’s an invitation to concurrency problems — like with threads. But if you have to write an explicit yield from, it’s fairly easy to make sure it doesn’t land in the middle of two critical operations that should appear atomic to the rest of the code.
As he mentions in his PyCon 2013 keynote, there are other Python async frameworks like Gevent, which are async by default, and he doesn’t like that approach. (at 11:58):
And unfortunately you’re still not completely clear of the problem
that the scheduler could at a random moment interrupt your task and
switch to a different one. […] Any function that you call today that
you happen to know that it never switches, tomorrow someone could add
a logging statement or a lazy caching or a consulting of a settings
file. […]
I went to the SF Python meetup when Guido talked about Tulip, the future asyncIO library for asynchronous operations in Python.
The take away is that if you want something to be run asynchronously you can use the "yield from" + expression and a couple of decorators to specify that the call to what comes after yield from should be executed asynchronously. The nice thing about it is that you can read the statements in that function normally (as if it was synchronous) and it will behave as if it was synchronous with respect to the execution of that function (return values and error/exception propagation and handling).
My question is: why not have the opposite behavior, namely, have all function calls be by default async (and without the yield from) and have a different explicit syntax when you want to execute something synchronously?
(besides the need for another keyword/syntax spec)
Note that the possible uses of yield from are a small part of the asynch PEP, and never need to be used. Maybe Guido oversold them in his talk 😉
As to why functions aren’t being changed to always be async by default, that’s just realism. Asynch gimmicks bring new overheads and semantic complications, and Python isn’t going to slow down and complicate life for everyone to make a few applications easier to write.
In short, “practicality beats purity” 😉
The real answer is that Guido likes the fact that asynchronous yield points are explicit in coroutines, because if you don’t realize that a call can yield, then that’s an invitation to concurrency problems — like with threads. But if you have to write an explicit yield from, it’s fairly easy to make sure it doesn’t land in the middle of two critical operations that should appear atomic to the rest of the code.
As he mentions in his PyCon 2013 keynote, there are other Python async frameworks like Gevent, which are async by default, and he doesn’t like that approach. (at 11:58):
And unfortunately you’re still not completely clear of the problem
that the scheduler could at a random moment interrupt your task and
switch to a different one. […] Any function that you call today that
you happen to know that it never switches, tomorrow someone could add
a logging statement or a lazy caching or a consulting of a settings
file. […]