How can I run a simple twisted client on top of asyncio?

Question

I have the following client code that I borrowed from twisted’s docs:

https://docs.twistedmatrix.com/en/twisted-20.3.0/web/howto/client.html#the-agent

And I am trying to run it with asyncio since I am building an asyncio project that requires compatibility with twisted. Here is the code:

import asyncio

from twisted.internet import asyncioreactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers


asyncioreactor.install()


async def request():
    agent = Agent(asyncioreactor.AsyncioSelectorReactor)

    d = agent.request(
        b'GET',
        b'http://httpbin.com/anything',
        Headers({'User-Agent': ['Twisted Web Client Example']}),
        None)

    def cbResponse(ignored):
        print('Response received')

    d.addCallback(cbResponse)

    def cbShutdown(ignored):
        asyncioreactor.AsyncioSelectorReactor.stop()

    d.addBoth(cbShutdown)
    print("This is where it always get stuck")
    res = await d.asFuture(asyncio.get_event_loop())
    print("SUCCESSS!!!!")


if __name__ == "__main__":
    asyncio.run(request())

I saved this as a request.py file and run it with python request.py, but it always hangs when reaching this line:

    print("This is where it always get stuck")
    res = await d.asFuture(asyncio.get_event_loop())

Is it possible to run this with asyncio? I am not too familiar with twisted and my ultimate goal is to be able to run a twisted client with asyncio.

Asked By: Ruben Quinones

||

Source

Answer 1

Your example code uses twisted.internet.asyncioreactor.AsyncioSelectorReactor as if it were a reactor. Instead, it is a class that implements a reactor.

The reactor is twisted.internet.reactor. Use it like this:

from twisted.internet import asyncioreactor
asyncioreactor.install()
from twisted.internet import reactor

Also, your example calls asyncio.run but it should call twisted.internet.reactor.run instead:

reactor.run()

If you want the reactor to stop in response to some event, call reactor.stop in the handler for that event.

twisted.internet.task.react is a convenient reactor start/stop helper function.

Here’s an example from https://meejah.ca/blog/python3-twisted-and-asyncio

from twisted.internet.task import react
from twisted.internet.defer import ensureDeferred
# our "real" main
async def _main(reactor):
    await some_deferred_returning_function()
# a wrapper that calls ensureDeferred
def main():
    return react(
        lambda reactor: ensureDeferred(
            _main(reactor)
        )
    )
if __name__ == '__main__':
    main()

Answered By: Jean-Paul Calderone

How can I run a simple twisted client on top of asyncio?

Question:

Answers: