Python/Pyscript asyncio order of events not as expected

Question

I am calling the code below from HTML using py-script. I am seeing that the order of events in this simple example is not what I expected. In the output below the displayed events are not in sequence. What am I missing to make my code wait until getWords() completes before I continue to process the output?

from request import request
import asyncio
import json

async def getWords():
    print("2. INSIDE GETWORDS:")
    baseurl = "http://localhost:4000/wordscape"

    body = json.dumps({"sqlline": sqlline})
    headers = {"Content-type": "application/json"}
    response = await request(f"{baseurl}/read", body=body, method="POST", headers=headers)

    my_Words = await response.string()
    print("3. MYWORDS inside getWords:", my_Words)


sqlline = "select upper(word) as word from dictionary.dictionary_words_20230103 where word = 'help'n"
print("1. BUILD SQL:", sqlline)
my_Words = ''
asyncio.ensure_future(getWords())
print("4. ####WORDS RETURNED FROM getWords:",my_Words)

Output:

1. BUILD SQL: select upper(word) as word from dictionary.dictionary_words_20230103 where word = 'help'

4. ####WORDS RETURNED FROM getWords: 
2. INSIDE GETWORDS:
3. MYWORDS inside getWords: [
  {
    "word": "HELP"
  }
]

Asked By: 99Valk

||

Source

Answer 1

There are a lot of things wrong with that code. The main problem you are describing comes from the fact that you don’t await the future that you schedule, when you call asyncio.ensure_future on your getWords function. It starts running, but you don’t wait for it to finish and jump to the print call right away.

In fact, you cannot await it in the global scope because await statements are not allowed outside of async functions. This means you need to create some sort of main coroutine function for that section of your code. That can be run from "the outside" via asyncio.run.

This leads right into the next big issue, which is the fact that you are using global variables (or more precisely mutating global state). This is almost always a terrible idea.

It is much more prudent to design your getWords function in such a way that it takes your sqlline as an argument and returns the words instead of writing them to a global variable. That way you can just await it from any other async function and continue processing its output.

Also, as the documentation mentions quite explicitly, you should probably avoid using ensure_future and use create_task instead. But in this case, you need neither of those because you can just await the coroutine directly. I see no use for scheduling it as a task (at least in your example).

Here is a simplified mock-version of your script with some changes:

import asyncio
import json


async def fake_request(_url: str, **_kwargs: object) -> str:
    await asyncio.sleep(2)
    return "foo"


async def get_words(sql_line: str) -> str:
    print("2. INSIDE get_words:")
    baseurl = "http://localhost:4000/wordscape"
    body = json.dumps({"sqlline": sql_line})
    headers = {"Content-type": "application/json"}
    my_words = await fake_request(
        f"{baseurl}/read", body=body, method="POST", headers=headers
    )
    print("3. my_words INSIDE get_words:", my_words)
    return my_words


async def main() -> None:
    sql = """
    select upper(word) as word from dictionary.dictionary_words_20230103 
    where word = 'help'n """
    print("1. BUILD SQL:", sql)
    words = await get_words(sql)
    print("4. #### WORDS RETURNED FROM get_words:", words)


if __name__ == "__main__":
    asyncio.run(main())

I also added type annotations and made the code PEP 8 compliant. Both of those are considered best practices.

Output:

1. BUILD SQL: 
    select upper(word) as word from dictionary.dictionary_words_20230103 
    where word = 'help'
 
2. INSIDE get_words:
3. my_words INSIDE get_words: foo
4. #### WORDS RETURNED FROM get_words: foo

Your question and code show a lot of gaps in fundamental understanding of asyncio. I recommend you read up on the basics because you will likely continue to run into such problems, unless you actually grasp the underlying concepts. There is a lot of good and free material out there.

Answered By: Daniil Fajnberg

Python/Pyscript asyncio order of events not as expected

Question:

Answers: