Difference between async await in python vs JavaScript
Question:
Note: this is not about multi threading or multi processing. This question is regarding a single process and single thread.
Python async.io and JavaScript async both are single thread concepts.
In python, async.io, we can use async await keywords to create a function so that when this function is invoked multiple times (via gather) they get executed concurrently. The way it works is that when an await keyword is encountered, other tasks can execute. As explained here we apply the async await keywords to function that we would like to execute concurrently. However while these tasks are running concurrently, the main thread is blocked.
In JavaScript async has evolved from callbacks, promise, async/await. In the main program, when async is encountered, then the function is sent to the event loop (where the function execution begins) and the main thread can continue working. Any subsequent async function also gets added to the event loop. Inside the event loop when the function execution encountered an await then other function is given a chance to execute untill await in encountered.
To get this behaviour in python, that is – allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing. Because once we start the child thread/process, and untill we call .join the main thread is not blocked.
Is there anyway by which the python’s async.io can make the main thread non blocking? If not, then is this the fundamental difference between async concept in JavaScript and python?
Answers:
when async is encountered, then the function is sent to the event loop and the main thread can continue working.
This is close, but not quite right. In Javascript, execution won’t stop until the callstack has been emptied – the await keyword will suspend the execution of a particular function until an event triggers, and in the mean time, control returns to its caller. This means the first part of any async function will execute as soon as it is called (it’s not immediately put into the event loop), and will only pause as soon as an await is hit.
To get this behaviour in python, that is – allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing.
The difference here is that by default, Javascript always has an event loop and python does not. In other words, python has an on/off switch for asynchronous programming while Javascript does not. When you run something such as loop.run_forever(), you’re basically flipping the event loop on, and execution won’t continue where you left off until the event loop gets turned back off. (calling it a "thread" isn’t quite the right word here, as it’s all single-threaded, as you already acknowledged. Instead, we generally call each task that we queue up, well, a "task")
You’re asking if there’s a way to let your code continue execution after starting up the event loop. I’m pretty sure the answer is no, nor should it be needed. Whatever you want to execute after the event loop has started can just be executed within the event loop.
If you want your python program to act more like Javascript, then the first thing you do can be to start up an event loop, and then any further logic can be placed within the first task that the event loop executes. In Javascript, this boiler plate essentially happens for you, and your source code is effectively that first task that’s queued up in the event loop.
Update:
Because there seems to be some confusion with how the Javascript event loop works, I’ll try to explain it a little further.
Remember that an event loop is simply a system where, when certain events happen, a block of synchronous code can be queued up to run as soon as the thread is not busy.
So let’s see what the event loop does for a simple program like this:
// This async function will resolve
// after the number of ms provided has passed
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
When Javascript begins executing the above program, it’ll begin with a single task queued up in it’s "run these things when you’re not busy" queue. This item is the whole program.
So, it’ll start at the top, defining whatever needs to be defined, executes console.log(1), call the main function, enters into it and runs console.log(2), calls wait() which will conceptually cause a background timer to start, wait() will return a promise which we then await, at which point we immediately go back to the caller of main, main wasn’t awaited so execution continues to console.log(3), until we finally finish at the end of the file. That whole path (from defining functions to console.log(3)) is a single, non-interruptible task. Even if another task got queued up, Javascript wouldn’t stop to handle that task until it finished this chunk of synchronous logic.
Later on, our countdown timer will finish, and another task will go into our queue, which will cause our main() function to continue execution. The same logic as before applies here – our execution path could enter and exit other async functions, and will only stop when it reaches the end of, in this case, the main function (even hitting an await keywords doesn’t actually make this line of syncrounous logic stop, it just makes it jump back to the caller). The execution of a single task doesn’t stop until the callstack has been emptied, and when execution is continuing from an async function, the first entry of the callstack starts at that particular async function.
Python’s async/await follows these same rules, except for the fact that in Python, the event loop isn’t running by default.
javascript
const wait = async (s) => {
setTimeout(() => {
console.log("wating " + s + "s")
}, s * 1000)
}
async function read_file() {
console.log("initial read_file sleep(2.1)")
await wait(2)
console.log("read_file 1/2 wait(2)")
await wait(0.1)
console.log("read_file 2/2 wait(0.1)")
}
async function read_api() {
console.log("initial read_api wait(2)")
await wait(2)
console.log("read_api whole wait(2)")
}
read_file()
console.log("does not block")
read_api()
console.log("the second time, won't block")
// initial read_file sleep(2.1)
// does not block
// initial read_api wait(2)
// the second time, won't block
// read_file 1/2 wait(2)
// read_api whole wait(2)
// read_file 2/2 wait(0.1)
// !!! Wait a moment
// wating 0.1s
// wating 2s
// wating 2s
python
import asyncio
async def read_file():
print("initial read_file asyncio.sleep(2 + 0.1)")
await asyncio.sleep(2)
print("read_file 1/2 asyncio.sleep(2)")
await asyncio.sleep(0.1)
print("read_file 2/2 asyncio.sleep(0.1)")
async def read_api():
print("initial read_api asyncio.sleep(2)")
await asyncio.sleep(2)
print("read_api whole asyncio.sleep(2)")
async def gather():
await asyncio.gather(
asyncio.create_task(read_file()),
asyncio.create_task(read_api()))
asyncio.run(gather())
"""
initial read_file asyncio.sleep(2.1)
initial read_api asyncio.sleep(2)
!!! Wait a moment
read_file 1/2 asyncio.sleep(2)
read_api whole asyncio.sleep(2)
read_file 2/2 asyncio.sleep(0.1)
"""
await scope:
- javascript: After the method is executed, wait for the Promise to resolve
await wait(2) Just wait(2) inside is guaranteed to be synchronous (or wait)
- python: Suspend method for other methods to execute
await asyncio.sleep(2) Method read_file will release resources and suspend
btw, javascript’s await/async is just Promise syntactic sugar
@scotty
Just for my clear understandng, how would you write:
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
in Python?
Note: this is not about multi threading or multi processing. This question is regarding a single process and single thread.
Python async.io and JavaScript async both are single thread concepts.
In python, async.io, we can use async await keywords to create a function so that when this function is invoked multiple times (via gather) they get executed concurrently. The way it works is that when an await keyword is encountered, other tasks can execute. As explained here we apply the async await keywords to function that we would like to execute concurrently. However while these tasks are running concurrently, the main thread is blocked.
In JavaScript async has evolved from callbacks, promise, async/await. In the main program, when async is encountered, then the function is sent to the event loop (where the function execution begins) and the main thread can continue working. Any subsequent async function also gets added to the event loop. Inside the event loop when the function execution encountered an await then other function is given a chance to execute untill await in encountered.
To get this behaviour in python, that is – allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing. Because once we start the child thread/process, and untill we call .join the main thread is not blocked.
Is there anyway by which the python’s async.io can make the main thread non blocking? If not, then is this the fundamental difference between async concept in JavaScript and python?
when async is encountered, then the function is sent to the event loop and the main thread can continue working.
This is close, but not quite right. In Javascript, execution won’t stop until the callstack has been emptied – the await keyword will suspend the execution of a particular function until an event triggers, and in the mean time, control returns to its caller. This means the first part of any async function will execute as soon as it is called (it’s not immediately put into the event loop), and will only pause as soon as an await is hit.
To get this behaviour in python, that is – allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing.
The difference here is that by default, Javascript always has an event loop and python does not. In other words, python has an on/off switch for asynchronous programming while Javascript does not. When you run something such as loop.run_forever(), you’re basically flipping the event loop on, and execution won’t continue where you left off until the event loop gets turned back off. (calling it a "thread" isn’t quite the right word here, as it’s all single-threaded, as you already acknowledged. Instead, we generally call each task that we queue up, well, a "task")
You’re asking if there’s a way to let your code continue execution after starting up the event loop. I’m pretty sure the answer is no, nor should it be needed. Whatever you want to execute after the event loop has started can just be executed within the event loop.
If you want your python program to act more like Javascript, then the first thing you do can be to start up an event loop, and then any further logic can be placed within the first task that the event loop executes. In Javascript, this boiler plate essentially happens for you, and your source code is effectively that first task that’s queued up in the event loop.
Update:
Because there seems to be some confusion with how the Javascript event loop works, I’ll try to explain it a little further.
Remember that an event loop is simply a system where, when certain events happen, a block of synchronous code can be queued up to run as soon as the thread is not busy.
So let’s see what the event loop does for a simple program like this:
// This async function will resolve
// after the number of ms provided has passed
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
When Javascript begins executing the above program, it’ll begin with a single task queued up in it’s "run these things when you’re not busy" queue. This item is the whole program.
So, it’ll start at the top, defining whatever needs to be defined, executes console.log(1), call the main function, enters into it and runs console.log(2), calls wait() which will conceptually cause a background timer to start, wait() will return a promise which we then await, at which point we immediately go back to the caller of main, main wasn’t awaited so execution continues to console.log(3), until we finally finish at the end of the file. That whole path (from defining functions to console.log(3)) is a single, non-interruptible task. Even if another task got queued up, Javascript wouldn’t stop to handle that task until it finished this chunk of synchronous logic.
Later on, our countdown timer will finish, and another task will go into our queue, which will cause our main() function to continue execution. The same logic as before applies here – our execution path could enter and exit other async functions, and will only stop when it reaches the end of, in this case, the main function (even hitting an await keywords doesn’t actually make this line of syncrounous logic stop, it just makes it jump back to the caller). The execution of a single task doesn’t stop until the callstack has been emptied, and when execution is continuing from an async function, the first entry of the callstack starts at that particular async function.
Python’s async/await follows these same rules, except for the fact that in Python, the event loop isn’t running by default.
javascript
const wait = async (s) => {
setTimeout(() => {
console.log("wating " + s + "s")
}, s * 1000)
}
async function read_file() {
console.log("initial read_file sleep(2.1)")
await wait(2)
console.log("read_file 1/2 wait(2)")
await wait(0.1)
console.log("read_file 2/2 wait(0.1)")
}
async function read_api() {
console.log("initial read_api wait(2)")
await wait(2)
console.log("read_api whole wait(2)")
}
read_file()
console.log("does not block")
read_api()
console.log("the second time, won't block")
// initial read_file sleep(2.1)
// does not block
// initial read_api wait(2)
// the second time, won't block
// read_file 1/2 wait(2)
// read_api whole wait(2)
// read_file 2/2 wait(0.1)
// !!! Wait a moment
// wating 0.1s
// wating 2s
// wating 2s
python
import asyncio
async def read_file():
print("initial read_file asyncio.sleep(2 + 0.1)")
await asyncio.sleep(2)
print("read_file 1/2 asyncio.sleep(2)")
await asyncio.sleep(0.1)
print("read_file 2/2 asyncio.sleep(0.1)")
async def read_api():
print("initial read_api asyncio.sleep(2)")
await asyncio.sleep(2)
print("read_api whole asyncio.sleep(2)")
async def gather():
await asyncio.gather(
asyncio.create_task(read_file()),
asyncio.create_task(read_api()))
asyncio.run(gather())
"""
initial read_file asyncio.sleep(2.1)
initial read_api asyncio.sleep(2)
!!! Wait a moment
read_file 1/2 asyncio.sleep(2)
read_api whole asyncio.sleep(2)
read_file 2/2 asyncio.sleep(0.1)
"""
await scope:
- javascript: After the method is executed, wait for the Promise to resolve
await wait(2)Justwait(2)inside is guaranteed to be synchronous (or wait)
- python: Suspend method for other methods to execute
await asyncio.sleep(2)Methodread_filewill release resources and suspend
btw, javascript’s
await/asyncis justPromisesyntactic sugar
@scotty
Just for my clear understandng, how would you write:
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
in Python?