Non-lazy evaluation version of map in Python3?

Question

I’m trying to use map in Python3. Here’s some code I’m using:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    map(writer.writerow, data)

However, since map in Python3 returns an iterator, this code doesn’t work in Python3 (but works fine in Python2 since that version of map always return a list)

My current solution is to add a list function call over the iterator to force the evaluation. But it seems odd (I don’t care about the return value, why should I convert the iterator into a list?)

Any better solutions?

Asked By: yegle

||

Source

Answer 1

If you don’t care about the return value, then map is not the best tool for the job. A simple for would be better:

for d in data:
    writer.writerow(d)

That’ll work fine in Python 2.x and 3.x. Notice that map is useful when you want to create a new list, if you’re traversing over an iterable just for the effect, then use a for.

Answered By: Óscar López

Answer 2

Using map for its side-effects (eg function call) when you’re not interested in returned values is undesirable even in Python2.x. If the function returns None, but repeats a million times – you’d be building a list of a million Nones just to discard it. The correct way is to either use a for-loop and call:

for row in data:
    writer.writerow(row)

or as the csv module allows, use:

writer.writerows(data)

If for some reason you really, really wanted to use map, then you can use the consume recipe from itertools and generate a zero length deque, eg:

from collections import deque
deque(map(writer.writerow, data), maxlen=0)

Answered By: Jon Clements

Answer 3

You can set up a zero length deque to do this:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    collections.deque(map(writer.writerow, data),0)

This is the same way that itertools.consume(iterator, None) recipe works. It functionally will exhaust the iterator without building the list.

You can also just use the consume recipe from itertools.

But a loop is more readable and Pythonic to me, but YMMV.

Answered By: dawg

Answer 4

I would use a function to extract data from the iterable using something like this:

def rake(what, where=None):
    for i in what: 
        if where: where.append(i)

rake(map(writer.writerow, data))

If you know up front that you won’t ever be collecting the output of the mapped function then you could simplify that to just:

for i in what: pass

But neither approach keeps excess data around unless you supply a list to put it in. And this approach should work equally well with map, filter, reduce, generators, range, and anything else that you could pass in to the rake function that for can iterate over.

Answered By: user1637146

Answer 5

You can also use list comprehension, as suggested in the official FAQ:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    [writer.writerow(elem) for elem in data]

List comprehension will force the evaluation of each element even if you don’t assign the newly created list to any variable.

Note, however, that the list may still be created behind the scenes, creating a potential performance pitfall, so while relatively succinct, it should not be used if the input sequence can get very long.

Answered By: undercat

Answer 6

list(map(lambda x: do(x),y))

will trigger an evaluation and stay in english pen-and-paper math semantics. Some like this more than others, but I’ve had a coder who spoke English as a second language tell me this is harder to understand than:

[ do(x) for x in y ]

¯(ツ)/¯

Anyhow, if you want to flush the map stack, trigger it with a list type conversion or a list comprehension.

With context:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    list(map(writer.writerow, data))
    # or: [ writer.writerow(x) for x in data ]

Answered By: Chris

Non-lazy evaluation version of map in Python3?

Question:

Answers: