what's Python asyncio.Lock() for?

Question

Is it because coroutines may be preempted in the future? Or it allows people to use yield from in critical section (which IMO shouldn’t be encouraged)?

Asked By: user3761759

||

Source

Answer 1

You use it for the same reason you’d use a lock in threaded code: to protect a critical section. asyncio is primarily meant for use in single-threaded code, but there is still concurrent execution happening (any time you hit a yield from or await), which means sometimes you need synchronization.

For example, consider a function that fetches some data from a web server, and then caches the results:

async def get_stuff(url):
    if url in cache:
        return cache[url]
    stuff = await aiohttp.request('GET', url)
    cache[url] = stuff
    return stuff

Now assume that you’ve got multiple co-routines running concurrently that might potentially need to use the return value of get_stuff:

async def parse_stuff():
    stuff = await get_stuff("www.example.com/data")
    # do some parsing

async def use_stuff():
    stuff = await get_stuff("www.example.com/data")
    # use stuff to do something interesting

async def do_work():
     out = await aiohttp.request("www.awebsite.com")
     # do some work with out
   

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
    parse_stuff(),
    use_stuff(),
    do_work(),
))

Now, pretend that fetching data from url is slow. If both parse_stuff and use_stuff run concurrently, each will be hit with the full cost of going over the network to fetch stuff. If you protect the method with a lock, you avoid this:

stuff_lock = asyncio.Lock()

async def get_stuff(url):
    async with stuff_lock:
        if url in cache:
            return cache[url]
        stuff = await aiohttp.request('GET', url)
        cache[url] = stuff
        return stuff

One other thing to note is that while one coroutine is inside get_stuff, making the aiohttp call, and another waits on stuff_lock, a third coroutine that doesn’t need to call get_stuff at all can also be running, without being affected by the coroutine blocking on the Lock.

Obviously this example is a little bit contrived, but hopefully it gives you an idea of why asyncio.Lock can useful; it allows you to protect a critical section, without blocking other coroutines from running which don’t need access to that critical section.

Answered By: dano

Answer 2

one example is when you only want some code run once, yet is requested by many(when in a web app for example)

async def request_by_many():
    key = False
    lock = asyncio.Lock()
    async with lock:
        if key is False:
            await only_run_once()

async def only_run_once():
    while True:
        if random()>0.5:
            key = True
            break
        await asyncio.sleep(1)

Answered By: eastonsuo

what's Python asyncio.Lock() for?

Question:

Answers: