Why using Asyncio is not reducing the overall execution time in Python and run functions concurrently?

Question

I am trying to run a piece of code using asyncio and reduce the time execution of the whole code. Below is my code which is taking around 6 seconds to fully execute itself

Normal function calls- (approach 1)

from time import time, sleep
import asyncio


def find_div(range_, divide_by):
    lis_ = []
    for i in range(range_):
        if i % divide_by == 0:
            lis_.append(i)
        
    print("found numbers for range {}, divided by {}".format(range_, divide_by))
    return lis_

if __name__ == "__main__":
    start = time()
    find_div(50800000, 341313)
    find_div(10005200, 32110)
    find_div(50000340, 31238)
    print(time()-start)

The output of the above code is just the total execution time which is 6 secs.

Multithreaded Approach- (approach 2)
Used multithreading in this, but surprisingly the time increased

from time import time, sleep
import asyncio
import threading


def find_div(range_, divide_by):
    lis_ = []
    for i in range(range_):
        if i % divide_by == 0:
            lis_.append(i)
        
    print("found numbers for range {}, divided by {}".format(range_, divide_by))
    return lis_

if __name__ == "__main__":
    start = time()
    t1 = threading.Thread(target=find_div, args=(50800000, 341313)) 
    t2 = threading.Thread(target=find_div, args=(10005200, 32110)) 
    t3 = threading.Thread(target=find_div, args=(50000340, 31238)) 
  
    t1.start() 
    t2.start() 
    t3.start()

    t1.join() 
    t2.join() 
    t3.join()
    print(time()-start)

The output of the above code is 12 secs.

Multiprocessing approach- (approach 3)

from time import time, sleep
import asyncio
from multiprocessing import Pool

def multi_run_wrapper(args):
   return find_div(*args)

def find_div(range_, divide_by):
    lis_ = []
    for i in range(range_):
        if i % divide_by == 0:
            lis_.append(i)
        
    print("found numbers for range {}, divided by {}".format(range_, divide_by))
    return lis_

if __name__ == "__main__":
    start = time()
    with Pool(3) as p:
        p.map(multi_run_wrapper,[(50800000, 341313),(10005200, 32110),(50000340, 31238)])
    
    
    print(time()-start)

The output of the multiprocessing code is 3 secs which is better than the normal function call approach.

Asyncio Approach- (approach 4)

from time import time, sleep

import asyncio

async def find_div(range_, divide_by):
    lis_ = []
    for i in range(range_):
        if i % divide_by == 0:
            lis_.append(i)
        
    print("found numbers for range {}, divided by {}".format(range_, divide_by))
    return lis_


async def task():

    tasks = [find_div(50800000, 341313),find_div(10005200, 32110),find_div(50000340, 31238)]
    result = await asyncio.gather(*tasks)
    print(result)

if __name__ == "__main__":
    start = time()
    asyncio.run(task())
    print(time()-start)

The above code is also taking around 6 seconds which is the same as the normal execution function call that is the Approach 1.

Problem-
Why is my Asyncio approach not working as expected and reducing the overall time?
What is wrong in the code?

Asked By: john mich

||

Source

Answer 1

You have code that exclusively uses the CPU.
Code like this cannot be sped up using async.

Async shines when you have tasks that are waiting on something not CPU related, e.g. a network request or reading from disk. This is generally true for all languages.

In python, also the thread based approach will not help you, as this still restricts you to a single core and not true parallel execution. This is due to the Global Interpreter Lock (GIL). The overhead of starting and switching between threads makes it slower than the simple version.
In this regard, threads are similar to async in python, it only helps if the time you are waiting is not spend mainly on the CPU or if you are calling code that’s not bound by the GIL, e.g. c extensions.

Using multiprocessing really uses multiple cpu cores, so it is faster than the normal solution.

Answered By: MaxNoe

Answer 2

asyncio def run(time):
     await asyncio.sleep(time)

This code takes 1 min 40 seconds

from datetime import datetime
now = datetime.now()

task=[]
for i in range(10):
    await run(10)  

now1=datetime.now()
print(now1-now)

OPTIMIZED USING async–>

THis takes 10 seconds only

from datetime import datetime
now = datetime.now()

task=[]
for i in range(10):
    task.append(asyncio.create_task(run(10)))
await asyncio.gather(*task)   

now1=datetime.now()
print(now1-now)

Answered By: 07satyam07

Why using Asyncio is not reducing the overall execution time in Python and run functions concurrently?

Question:

Answers: