How to print a generator expression?
Question:
In the Python shell, if I enter a list comprehension such as:
>>> [x for x in string.letters if x in [y for y in "BigMan on campus"]]
I get a nicely printed result:
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
Same for a dictionary comprehension:
>>> {x:x*2 for x in range(1,10)}
{1: 2, 2: 4, 3: 6, 4: 8, 5: 10, 6: 12, 7: 14, 8: 16, 9: 18}
If I enter a generator expression, I get not such a friendly response:
>>> (x for x in string.letters if x in (y for y in "BigMan on campus"))
<generator object <genexpr> at 0x1004a0be0>
I know I can do this:
>>> for i in _: print i,
a c g i m n o p s u B M
Other than that (or writing a helper function) can I easily evaluate and print that generator object in the interactive shell?
Answers:
You can just wrap the expression in a call to list:
>>> list(x for x in string.letters if x in (y for y in "BigMan on campus"))
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
>>> list(x for x in string.letters if x in (y for y in "BigMan on campus"))
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
Quick answer:
Doing list() around a generator expression is (almost) exactly equivalent to having [] brackets around it. So yeah, you can do
>>> list((x for x in string.letters if x in (y for y in "BigMan on campus")))
But you can just as well do
>>> [x for x in string.letters if x in (y for y in "BigMan on campus")]
Yes, that will turn the generator expression into a list comprehension. It’s the same thing and calling list() on it. So the way to make a generator expression into a list is to put brackets around it.
Detailed explanation:
A generator expression is a “naked” for expression. Like so:
x*x for x in range(10)
Now, you can’t stick that on a line by itself, you’ll get a syntax error. But you can put parenthesis around it.
>>> (x*x for x in range(10))
<generator object <genexpr> at 0xb7485464>
This is sometimes called a generator comprehension, although I think the official name still is generator expression, there isn’t really any difference, the parenthesis are only there to make the syntax valid. You do not need them if you are passing it in as the only parameter to a function for example:
>>> sorted(x*x for x in range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Basically all the other comprehensions available in Python 3 and Python 2.7 is just syntactic sugar around a generator expression. Set comprehensions:
>>> {x*x for x in range(10)}
{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
>>> set(x*x for x in range(10))
{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
Dict comprehensions:
>>> dict((x, x*x) for x in range(10))
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
>>> {x: x*x for x in range(10)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
And list comprehensions under Python 3:
>>> list(x*x for x in range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> [x*x for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Under Python 2, list comprehensions is not just syntactic sugar. But the only difference is that x will under Python 2 leak into the namespace.
>>> x
9
While under Python 3 you’ll get
>>> x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
This means that the best way to get a nice printout of the content of your generator expression in Python is to make a list comprehension out of it! However, this will obviously not work if you already have a generator object. Doing that will just make a list of one generator:
>>> foo = (x*x for x in range(10))
>>> [foo]
[<generator object <genexpr> at 0xb7559504>]
In that case you will need to call list():
>>> list(foo)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Although this works, but is kinda stupid:
>>> [x for x in foo]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Or you can always map over an iterator, without the need to build an intermediate list:
>>> _ = map(sys.stdout.write, (x for x in string.letters if x in (y for y in "BigMan on campus")))
acgimnopsuBM
Unlike a list or a dictionary, a generator can be infinite. Doing this wouldn’t work:
def gen():
x = 0
while True:
yield x
x += 1
g1 = gen()
list(g1) # never ends
Also, reading a generator changes it, so there’s not a perfect way to view it.
To see a sample of the generator’s output, you could do
g1 = gen()
[g1.next() for i in range(10)]
Generator object do not store actual data, it is basically just an expression. Program can not print what will be value of an expression without evaluating it. Generator object(generator expression) can be evaluated by typecasting into any iterable data type.
eg.
list(genexpr)
dict(genexpr)
set(genexpr)
for data in genexpr:
Additional
generating generator expression and then typecasting is 20% slower than directly creating required datatype object. So if we require whole data it’s better to use
data=[x for x in range(0,10)]
than using
genexpr=(x for x in range(0,10))
data=list(genexpr)
print(i for i in range(9))
if we run this we will get output as: – <generator object at 0x000001F01A153E40>
one simple way to print generator is converting it to list.
so simply if we modify our code as
print(*[i for i in range(9)])
So we will get output as : 0 1 2 3 4 5 6 7 8
You could also just do something like this:
gen = (i for i in 'abcde')
print( *gen ) # => a b c d e
In the Python shell, if I enter a list comprehension such as:
>>> [x for x in string.letters if x in [y for y in "BigMan on campus"]]
I get a nicely printed result:
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
Same for a dictionary comprehension:
>>> {x:x*2 for x in range(1,10)}
{1: 2, 2: 4, 3: 6, 4: 8, 5: 10, 6: 12, 7: 14, 8: 16, 9: 18}
If I enter a generator expression, I get not such a friendly response:
>>> (x for x in string.letters if x in (y for y in "BigMan on campus"))
<generator object <genexpr> at 0x1004a0be0>
I know I can do this:
>>> for i in _: print i,
a c g i m n o p s u B M
Other than that (or writing a helper function) can I easily evaluate and print that generator object in the interactive shell?
You can just wrap the expression in a call to list:
>>> list(x for x in string.letters if x in (y for y in "BigMan on campus"))
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
>>> list(x for x in string.letters if x in (y for y in "BigMan on campus"))
['a', 'c', 'g', 'i', 'm', 'n', 'o', 'p', 's', 'u', 'B', 'M']
Quick answer:
Doing list() around a generator expression is (almost) exactly equivalent to having [] brackets around it. So yeah, you can do
>>> list((x for x in string.letters if x in (y for y in "BigMan on campus")))
But you can just as well do
>>> [x for x in string.letters if x in (y for y in "BigMan on campus")]
Yes, that will turn the generator expression into a list comprehension. It’s the same thing and calling list() on it. So the way to make a generator expression into a list is to put brackets around it.
Detailed explanation:
A generator expression is a “naked” for expression. Like so:
x*x for x in range(10)
Now, you can’t stick that on a line by itself, you’ll get a syntax error. But you can put parenthesis around it.
>>> (x*x for x in range(10))
<generator object <genexpr> at 0xb7485464>
This is sometimes called a generator comprehension, although I think the official name still is generator expression, there isn’t really any difference, the parenthesis are only there to make the syntax valid. You do not need them if you are passing it in as the only parameter to a function for example:
>>> sorted(x*x for x in range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Basically all the other comprehensions available in Python 3 and Python 2.7 is just syntactic sugar around a generator expression. Set comprehensions:
>>> {x*x for x in range(10)}
{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
>>> set(x*x for x in range(10))
{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
Dict comprehensions:
>>> dict((x, x*x) for x in range(10))
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
>>> {x: x*x for x in range(10)}
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81}
And list comprehensions under Python 3:
>>> list(x*x for x in range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> [x*x for x in range(10)]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Under Python 2, list comprehensions is not just syntactic sugar. But the only difference is that x will under Python 2 leak into the namespace.
>>> x
9
While under Python 3 you’ll get
>>> x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
This means that the best way to get a nice printout of the content of your generator expression in Python is to make a list comprehension out of it! However, this will obviously not work if you already have a generator object. Doing that will just make a list of one generator:
>>> foo = (x*x for x in range(10))
>>> [foo]
[<generator object <genexpr> at 0xb7559504>]
In that case you will need to call list():
>>> list(foo)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Although this works, but is kinda stupid:
>>> [x for x in foo]
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Or you can always map over an iterator, without the need to build an intermediate list:
>>> _ = map(sys.stdout.write, (x for x in string.letters if x in (y for y in "BigMan on campus")))
acgimnopsuBM
Unlike a list or a dictionary, a generator can be infinite. Doing this wouldn’t work:
def gen():
x = 0
while True:
yield x
x += 1
g1 = gen()
list(g1) # never ends
Also, reading a generator changes it, so there’s not a perfect way to view it.
To see a sample of the generator’s output, you could do
g1 = gen()
[g1.next() for i in range(10)]
Generator object do not store actual data, it is basically just an expression. Program can not print what will be value of an expression without evaluating it. Generator object(generator expression) can be evaluated by typecasting into any iterable data type.
eg.
list(genexpr)
dict(genexpr)
set(genexpr)
for data in genexpr:
Additional
generating generator expression and then typecasting is 20% slower than directly creating required datatype object. So if we require whole data it’s better to use
data=[x for x in range(0,10)]
than using
genexpr=(x for x in range(0,10))
data=list(genexpr)
print(i for i in range(9))
if we run this we will get output as: – <generator object at 0x000001F01A153E40>
one simple way to print generator is converting it to list.
so simply if we modify our code as
print(*[i for i in range(9)])
So we will get output as : 0 1 2 3 4 5 6 7 8
You could also just do something like this:
gen = (i for i in 'abcde')
print( *gen ) # => a b c d e