Async generator is not an iterator?

Question

In Python you can write a generator that is iterable like:

def generate(count):
    for x in range(count):
        yield x

# as an iterator you can apply the function next() to get the values.
it = generate(10)
r0 = next(it)
r1 = next(it) ...

When trying to use an async iterator, you get the ‘yield inside async’ error.
The suggested solution is to implement your own generator:

class async_generator:
    def __aiter__(self):
        return self

    async def __anext__(self):
        await asyncio.sleep()
        return random.randint(0, 10)
        
# But when you try to get the next element
it = async_generator(10)
r0 = next(it)

You get the error "async_generator" object is not an iterator

I think that if you are going to call something an Iterator it’s because it has exactly the same interface, so I can just write async iterators and use on a framework that relies heavily on next() calls.
Any new Python capability is pointless if you need to rewrite your entire code to be able to use async.

Am I missing something?

Thanks!

Asked By: user1275011

||

Source

Answer 1

I believe a new statement was introduced for async generators:

async for TARGET in ITER:
    BLOCK
else:
    BLOCK2

according to PEP 492.

Basically, this would mean you should do:

async for number in generate(10):
        print(number)

Also, check the Differences from generators:

Native coroutine objects do not implement iter and next
methods. Therefore, they cannot be iterated over or passed to iter() ,
list() , tuple() and other built-ins. They also cannot be used in a
for..in loop. An attempt to use iter or next on a native
coroutine object will result in a TypeError .

Answered By: bosnjak

Answer 2

I used this to async loop through a list

class AsyncRange(object):
    def __init__(self, length):
        self.length = length
        self.i = 0

    async def __aiter__(self):
        return self

    async def __anext__(self):
        index = self.i
        self.i += 1
        if self.i <= self.length:
            return index
        else:
            raise StopAsyncIteration

then simply:

async for i in AsyncRange(my_list):
    # your code

Answered By: Juggernaut

Answer 3

So, as @bosnjak said, you can use async for:

async for ITEM in A_ITER:
    BLOCK1
else: # optional
    BLOCK2

But if you want to iterate manually, you can simply write:

it = async_iterator()
await it.__anext__()

But I wouldn’t recommend to do that.

I think that if you are going to call something an Iterator its because it has exactly the same interface, so I can just write async iterators and use on a framework that relies heavily on next() calls

No, acutally it’s not the same. There is a difference between regular synchronous iterators and asynchronous ones. And there few reasons for that:

Python coroutines are built on top of generators internally
According to Zen of python, explicit is better than implicit. So that you will actually see, where code can be suspended.

That’s why it’s impossible to use iter and next with asynchronous iterators. And you cannot use them with frameworks that expects synchronous iterators. So if you are going to make your code asynchronous, you have to use asynchronous frameworks as well. Here are few of them.

Also, I would like to say a few words about iterators and generators. Iterator is a special object that has __iter__ and __next__ methods. Whereas generator is a special function containing yield expression. Every generator is an iterator, but not vice versa. The same thing is acceptable to asynchronous iterators and generators. Yes, since python 3.6 you are able to write asynchronous generators!

async def ticker(delay, to):
    for i in range(to):
        yield i
        await asyncio.sleep(delay)

You can read PEP 525 for more details

Answered By: Michael Ihnatenko

Answer 4

In python 3.10+ you can use anext https://docs.python.org/3/library/functions.html#anext

I modified previous examples to illustrate how anext works:

import asyncio


async def ticker(to):
    for i in range(to):
        await asyncio.sleep(1)
        yield i


async def main():

    # get only first, failed if nothing yield
    await anext(ticker(3))

    # get only first or default (77)
    await anext(ticker(3), 77)

    # iterate through all
    async for i in ticker(3):
        print(i)

asyncio.run(main())

Answered By: rfedorov

Async generator is not an iterator?

Question:

Answers: