Making a sequence of functions in Python with different parameter values?
Question:
I have a function get_knng_graph that takes two parameters; a set of points and an integer k. I want to generate a sequence of functions, each of which only accepts the set of points, but with the value k of the parameter embedded inside different for every function.
Consider the code below:
# definition of get_knng_graph(....) here
graph_fns = []
for k in range(1,5):
def knng(pts):
return get_knng_graph(pts,k)
graph_fns.append(knng);
Is this reasonable code? By which I mean can I be assured that the values of the parameter k embedded inside each of the elements of graph_fns will continue to be different?
In the Haskell world, of course, this is nothing but currying, but this is the first time I am doing something like this in Python.
I tried it, and the code doesn’t work. If I place a print(k) in the code above, then when I execute successive functions in the array, it keeps prints out 4 for all function runs.
Answers:
The problem you are seeing is because Python creates that reference to the name k and doesn’t capture the value, so your code is equivalent to this code:
graph_fns = []
def knng(pts):
return get_knng_graph(pts,k)
for k in range(1,5):
graph_fns.append(knng);
If you want to bind the value of k to the function, there are a couple of solutions.
The most trivial code change is to add an extra argument with a default parameter:
graph_fns = []
for k in range(1,5):
def knng(pts, k=k):
return get_knng_graph(pts, k)
graph_fns.append(knng)
You might also find it a bit cleaner to use functools.partial:
from functools import partial
graph_fns = []
for k in range(1,5):
knng = partial(get_knng_graph, k=k)
graph_fns.append(knng)
and by that time you could just use a list comprehension:
from functools import partial
graph_fns = [partial(get_knng_graph, k=k) for k in range(1, 5)]
There are some other options discussed on this page, like creating a class for this.
In Python, scopes are function wide, that is using a for loop does not introduce a new nested scope. Thus in this example, k is rebound every iteration, and the k in every knng closure refers to that same variable, and if you call any of them after the loop has run its course will show its last value (4 in this case). The standard Python way to deal with this is to shadow it with a default argument:
graph_fns = []
for k in range(1,5):
def knng(pts, k=k):
return get_knng_graph(pts,k)
graph_fns.append(knng)
This works because default arguments are bound when the definition is executed and the closure is created.
Seems to me, this is a good case for using partial from the functools module.
So say I have a function that takes an input, squares it and adds some variable to it before returning the result:
def x_squared_plus_n(x,n):
return (x**2)+n
If I want to curry, or redefine that function, modifying it so that a fixed number (say 5) is always squared, and has a number n added to it, I can do so by using partial
from functools import partial
five_squared_plus_n = partial(x_squared_plus_n,5)
Now, I have a new function five_squared_plus_n for which the first x parameter in the original function’s parameter signature is fixed to x=5. The new function has a parameter signature containing only the remaining parameters, here n.
So calling:
five_squared_plus_n(15)
or equivalently,
five_squared_plus_n(n=15)
The answer of 40 is returned.
Any combination of parameters can be fixed like this and the resulting "curried" function be assigned to a new function name. It’s a very powerful tool.
In your example, you could wrap your partial calls in a loop, over which the values of different values could be fixed, and assign the resultant functions to values in a dictionary. Using my simple example, that might look something like:
func_dict = {}
for k in range(1,5):
func_dict[k]=partial(x_squared_plus_n,k)
Which would prepare a series of functions, callable by reference to that dictionary – so:
func_dict[1](5)
Would return 12+5=6 , while
func_dict[3](12)
Would return 32+12=21 .
It is possible to assign proper python names to these functions, but that’s probably for a different question – here, just imagine that the dictionary hosts a series of functions, accessible by key. I’ve used a numeric key, but you could assign strings or other values to help access the function you’ve prepared in this way.
Python’s support for Haskell-style "functional" programming is fairly strong – you just need to dig around a little to access the appropriate hooks. I think,m subjectively, there’s perhaps less purity in terms of functional design, but for most practical purposes, there is a functional solution.
I have a function get_knng_graph that takes two parameters; a set of points and an integer k. I want to generate a sequence of functions, each of which only accepts the set of points, but with the value k of the parameter embedded inside different for every function.
Consider the code below:
# definition of get_knng_graph(....) here
graph_fns = []
for k in range(1,5):
def knng(pts):
return get_knng_graph(pts,k)
graph_fns.append(knng);
Is this reasonable code? By which I mean can I be assured that the values of the parameter k embedded inside each of the elements of graph_fns will continue to be different?
In the Haskell world, of course, this is nothing but currying, but this is the first time I am doing something like this in Python.
I tried it, and the code doesn’t work. If I place a print(k) in the code above, then when I execute successive functions in the array, it keeps prints out 4 for all function runs.
The problem you are seeing is because Python creates that reference to the name k and doesn’t capture the value, so your code is equivalent to this code:
graph_fns = []
def knng(pts):
return get_knng_graph(pts,k)
for k in range(1,5):
graph_fns.append(knng);
If you want to bind the value of k to the function, there are a couple of solutions.
The most trivial code change is to add an extra argument with a default parameter:
graph_fns = []
for k in range(1,5):
def knng(pts, k=k):
return get_knng_graph(pts, k)
graph_fns.append(knng)
You might also find it a bit cleaner to use functools.partial:
from functools import partial
graph_fns = []
for k in range(1,5):
knng = partial(get_knng_graph, k=k)
graph_fns.append(knng)
and by that time you could just use a list comprehension:
from functools import partial
graph_fns = [partial(get_knng_graph, k=k) for k in range(1, 5)]
There are some other options discussed on this page, like creating a class for this.
In Python, scopes are function wide, that is using a for loop does not introduce a new nested scope. Thus in this example, k is rebound every iteration, and the k in every knng closure refers to that same variable, and if you call any of them after the loop has run its course will show its last value (4 in this case). The standard Python way to deal with this is to shadow it with a default argument:
graph_fns = []
for k in range(1,5):
def knng(pts, k=k):
return get_knng_graph(pts,k)
graph_fns.append(knng)
This works because default arguments are bound when the definition is executed and the closure is created.
Seems to me, this is a good case for using partial from the functools module.
So say I have a function that takes an input, squares it and adds some variable to it before returning the result:
def x_squared_plus_n(x,n):
return (x**2)+n
If I want to curry, or redefine that function, modifying it so that a fixed number (say 5) is always squared, and has a number n added to it, I can do so by using partial
from functools import partial
five_squared_plus_n = partial(x_squared_plus_n,5)
Now, I have a new function five_squared_plus_n for which the first x parameter in the original function’s parameter signature is fixed to x=5. The new function has a parameter signature containing only the remaining parameters, here n.
So calling:
five_squared_plus_n(15)
or equivalently,
five_squared_plus_n(n=15)
The answer of 40 is returned.
Any combination of parameters can be fixed like this and the resulting "curried" function be assigned to a new function name. It’s a very powerful tool.
In your example, you could wrap your partial calls in a loop, over which the values of different values could be fixed, and assign the resultant functions to values in a dictionary. Using my simple example, that might look something like:
func_dict = {}
for k in range(1,5):
func_dict[k]=partial(x_squared_plus_n,k)
Which would prepare a series of functions, callable by reference to that dictionary – so:
func_dict[1](5)
Would return 12+5=6 , while
func_dict[3](12)
Would return 32+12=21 .
It is possible to assign proper python names to these functions, but that’s probably for a different question – here, just imagine that the dictionary hosts a series of functions, accessible by key. I’ve used a numeric key, but you could assign strings or other values to help access the function you’ve prepared in this way.
Python’s support for Haskell-style "functional" programming is fairly strong – you just need to dig around a little to access the appropriate hooks. I think,m subjectively, there’s perhaps less purity in terms of functional design, but for most practical purposes, there is a functional solution.