Behaviour I don't understand in Python mixings comprehension lists and lambda-functions

Question

I don’t understand the behaviour of a piece of code, involving a comprehension list over lambda functions that call a method in different objects. It happened in a large program, so for this question I made a nonsensical toy-case to show my point:

class Evaluator(object):
    def __init__(self, lft, rgt):
        self.lft = lft
        self.rgt = rgt

    def eval(self, x):
        return self.lft + x * (self.rgt - self.lft)


if __name__ == "__main__":
    ev1 = Evaluator(2, 3)
    ev2 = Evaluator(4, 5)
    ev3 = Evaluator(6, 7)
    funcs = [lambda x:ev.eval(x+0.1) for ev in (ev1, ev2, ev3)]
    print([f(0.5) for f in funcs])

The output I get is [6.6, 6.6, 6.6], which means that it is the method in ev3 the one that is being evaluated all the time. Instead of [2.6, 4.6, 6.6], as I would have expected. But what really surprises me is that if I get rid of the lambda-function, the behaviour is fine:

class Evaluator(object):
    def __init__(self, lft, rgt):
        self.lft = lft
        self.rgt = rgt

    def eval(self, x):
        return self.lft + x * (self.rgt - self.lft)

if __name__ == "__main__":
    ev1 = Evaluator(2, 3)
    ev2 = Evaluator(4, 5)
    ev3 = Evaluator(6, 7)
    funcs = [ev.eval for ev in (ev1, ev2, ev3)]
    print([f(0.5) for f in funcs])

returns [2.5, 4.5, 6.5]. Can anyone explain what is going on here? And how should I code this in a Pythoninstic way?

Asked By: zeycus

||

Source

Answer 1

The problem is because of the way lambdas are evaluated. All three lambdas are this:

ev.eval(x+0.1)

But after the comprehension, the value of ev is ev3, and so all three lambdas are the same.

You can observe this by deleting ev before evaluating the functions. You will get an attribute error.

Answered By: Jayanth Koushik

Answer 2

Why aren't python nested functions called closures?

These lambdas are closures that use the name ev from the enclosing scope. When the lambdas are executed, ev is bound to ev3.

Observe:

a = 1

def x():
    print a

a = 2

x()   # prints 2

Here ev is evaluated at the lambda’s creation:

funcs = [lambda x, e=ev: e.eval(x + 0.1) for ev in (ev1, ev2, ev3)]
print([f(0.5) for f in funcs])

Answered By: Pavel Anossov

Answer 3

The problem is that you are only ever evaluating ev at the time you call the function. Thus, it’s using whatever value ev has only when you start the print statement. Of course, by that time ev has the value of the last function in the list.

This is no different than if you had done this:

funcs = [lambda x: ev.eval(x+0.1),
         lambda x: ev.eval(x+0.1),
         lambda x: ev.eval(x+0.1)]

Notice how they all use ev, and they will all use the same ev at the time you run the functions.

To do what you want, you need to bind ev to its current value in the list comprehension at the time you define the comprehension, which you can do by passing in the value through the lambda arguments:

funcs = [lambda x, ev=ev: ev.eval(x+0.1) for ev in (ev1, ev2, ev3)]

However, I strongly suggest you do not do this. As you have just experienced, such code is very hard to understand and debug. You win no points for cramming as much functionality into a single line as possible.

The technical term for this is a closure. For more information, check out this question on stackoverflow:
Why aren't python nested functions called closures?

Answered By: Bryan Oakley

Behaviour I don't understand in Python mixings comprehension lists and lambda-functions

Question:

Answers: