Python async Playwright pass data outside function

Question

I’m quite new to aynschronous programming and I just can’t get the json data out of the function. Is there some kind of special way to pass on data from async functions? I would like to use the json data to extract other data.

async def main():
    async with async_playwright() as p:

        async def handle_response(response): 
        # the endpoint we are insterested in 
            if ('eindpoint/name' in response.url): 
                json_data = await response.json()

                print((json_data))

        browser = await p.chromium.launch()
        page = await browser.new_page()
                
        # go to directly to searchpage
        await page.goto("website_url", wait_until='networkidle')
        
        page.on('response', handle_response)

        await page.fill('input[id=zoeklocatie]', 'search_query')
        
        # Use two enters to first make button visible
        await page.keyboard.press("Enter")
        await page.keyboard.press("Enter")
        
        await page.wait_for_timeout(3000)
        
        await browser.close()
    
await main()

The result right now is that the JSON data is printed. But how can I get this JSON data outside the function and use it further on for other stuff.

I tried to return the data and also the variable. Using global variables. But the return value keeps being empty and I think it has something to do with the asynchronomous working of the code. So the return comes earlier than the result.

Anyone an idea if I am correct and how I can solve this?

Thanks for the help!

Asked By: grietbroek

||

Source

Answer 1

you can use a Future (just like a Promise in JS)

async def main():
    async with async_playwright() as p:
        myRespPromise = asyncio.Future()
        async def handle_response(response): 
        # the endpoint we are insterested in 
            if ('eindpoint/name' in response.url): 
                json_data = await response.json()
                # "resolve the promise"
                myRespPromise.set_result(json_data)

        browser = await p.chromium.launch()
        page = await browser.new_page()
                
        # go to directly to searchpage
        await page.goto("website_url", wait_until='networkidle')
        
        page.on('response', handle_response)
        print("Made call, now await response...")
        result_json = await myRespPromise
        print("GOT RESULT:",result_json)

        await page.fill('input[id=zoeklocatie]', 'search_query')
        
        # Use two enters to first make button visible
        await page.keyboard.press("Enter")
        await page.keyboard.press("Enter")
        
        await page.wait_for_timeout(3000)
        
        await browser.close()
    
await main()

Answered By: Joran Beasley

Answer 2

So with the help of Joran I was able to make this code work, tx!

I used his suggestion for using a future twice to get the data outside the main() function.

mainRespPromise = asyncio.Future()

async def main():
    async with async_playwright() as p:
        myRespPromise = asyncio.Future()
        async def handle_response(response): 
        # the endpoint we are insterested in 
            if ('eindpoint/name' in response.url): 
                json_data = await response.json()
                # "resolve the promise"
                myRespPromise.set_result(json_data)

        browser = await p.chromium.launch()
        page = await browser.new_page()
                
        # go to directly to searchpage
        await page.goto("website_url", wait_until='networkidle')
        
        page.on('response', handle_response)
        print("Made call, now await response...")

        await page.fill('input[id=zoeklocatie]', 'search_query')
        
        # Use two enters to first make button visible
        await page.keyboard.press("Enter")
        await page.keyboard.press("Enter")
        
        result_json = await myRespPromise
        print("GOT RESULT:",result_json)

        await page.wait_for_timeout(3000)
        
        await browser.close()
        mainRespPromise.set_result(result_json)

await main()

json_data = await mainRespPromise

Answered By: grietbroek

Python async Playwright pass data outside function

Question:

Answers: