What is the scope of a defaulted parameter in Python?
Question:
When you define a function in Python with an array parameter, what is the scope of that parameter?
This example is taken from the Python tutorial:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
Prints:
[1]
[1, 2]
[1, 2, 3]
I’m not quite sure if I understand what’s happening here. Does this mean that the scope of the array is outside of the function? Why does the array remember its values from call to call? Coming from other languages, I would expect this behavior only if the variable was static. Otherwise it seems it should be reset each time. And actually, when I tried the following:
def f(a):
L = []
L.append(a)
return L
I got the behavior I expected (the array was reset on each call).
So it seems to me that I just need the line def f(a, L=[]): explained – what is the scope of the L variable?
Answers:
The scope is as you would expect.
The perhaps surprising thing is that the default value is only calculated once and reused, so each time you call the function you get the same list, not a new list initialized to [].
The list is stored in f.__defaults__ (or f.func_defaults in Python 2.)
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
print f.__defaults__
f.__defaults__ = (['foo'],) # Don't do this!
print f(4)
Result:
[1]
[1, 2]
[1, 2, 3]
([1, 2, 3],)
['foo', 4]
The scope of the L variable is behaving as you expect.
The “problem” is with the list you’re creating with []. Python does not create a new list each time you call the function. L gets assigned the same list each time you call which is why the function “remembers” previous calls.
So in effect this is what you have:
mylist = []
def f(a, L=mylist):
L.append(a)
return L
The Python Tutorial puts it this way:
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.
and suggests the following way to code the expected behaviour:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
There’s even less “magic” than you might suspect. This is equivalent to
m = []
def f(a, L=m):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
m is only created once.
The “problem” here is that L=[] is only evaluated once, that is, when the file is compiled. Python steps through each line of the file and compiles it. By the time it reaches the def with the default parameter, it creates that list instance once.
If you put L = [] inside the function code, the instance is not created at “compile time” (actually compile time can also be called part of the run time) because Python compiles the function’s code but does not call it. So you will get a new list instance because the creation is done every time you call the function (instead of once during compilation).
A solution for that problem is not to use mutable objects as default parameters, or only fixed instances like None:
def f(a, L = None):
if l == None:
l = []
L.append(a)
return L
Note that in both cases you described, the scope of L is the function scope.
The explaination is given in answers to this question. To sum it up here:
Functions in Python are a kind of object. Because they are a kind of object, they act like objects when instantiated. A function, if defined with a mutable attribute as a default argument, is exactly the same as a class with a static attribute that is a mutable list.
Lennart Regebro has a good explanation and the answer to the question by Roberto Liffredo is excellent.
To adapt Lennart’s answer … if I have a BananaBunch class:
class BananaBunch:
bananas = []
def addBanana(self, banana):
self.bananas.append(banana)
bunch = BananaBunch()
>>> bunch
<__main__.BananaBunch instance at 0x011A7FA8>
>>> bunch.addBanana(1)
>>> bunch.bananas
[1]
>>> for i in range(6):
bunch.addBanana("Banana #" + i)
>>> for i in range(6):
bunch.addBanana("Banana #" + str(i))
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5']
// And for review ...
//If I then add something to the BananaBunch class ...
>>> BananaBunch.bananas.append("A mutated banana")
//My own bunch is suddenly corrupted. :-)
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5', 'A mutated banana']
How does this apply to functions? Functions in Python are objects. This bears repeating. Functions in Python are objects.
So when you create a function, you are creating an object. When you give a function a mutable default value, you are populating that object’s attribute with a mutable value, and every time you call that function you are operating on the same attribute. So if you are using a mutable call (like append), then you are modifying the same object, just as if you were adding bananas to the bunch object.
You have to keep in mind that python is an interpreted language. What is happening here is when the function “f” is defined, it creates the list and assigns it to the default parameter “L” of function “f”. Later, when you call this function, the same list is used as the default parameter. In short, the code on the “def” line, only gets executed once when the function is defined. This is a common python pitfall, of which I have fallen in myself.
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
There have been suggestions for idioms in other answers here to fix this issue. The one I would suggest is as follows:
def f(a, L=None):
L = L or []
L.append(a)
return L
This uses the or short circuit to either take the “L” that was passed, or create a new list.
The answer to your scope question is the “L” only has a scope within the function “f”, but because the default parameters are only assigned once to a single list instead of every time you call the function it behaves as if the default parameter “L” has a global scope.
Say you have the following code:
def func(a=[]):
a.append(1)
print("A:", a)
func()
func()
func()
You can use python’s indentation to help you understand what’s going on. Everything that is flush to the left margin is executed when the file gets executed. Everything that’s indented is compiled into a code object which gets executed when func() is called. So the function is defined and its default arguments set once, when the program gets executed (because the def statement is flush left).
What it does with the default arguments is an interesting issue. In python 3, it puts most of the information about a function in two places: func.__code__ and func.__defaults__. In python 2, func.__code__ was func.func_code func.__defaults__ was func.func_defaults. Later versions of python 2, including 2.6 have both sets of names, to aid the transition from python 2 to python 3. I will use the more modern __code__ and __defaults__. If you’re stuck on an older python, the concepts are the same; just the names differ.
The default values are stored in func.__defaults__, and retrieved each time the function is called.
Thus when you define the function above, the body of the function gets compiled and stored in variables under __code__, to be executed later, and the default arguments get stored in __defaults__. When you call the function, it uses the values in __defaults__. If those values get modified for any reason, it only has the modified version available to use.
Play around defining different functions in the interactive interpreter, and see what you can figure out about how python creates and uses functions.
When you define a function in Python with an array parameter, what is the scope of that parameter?
This example is taken from the Python tutorial:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
Prints:
[1]
[1, 2]
[1, 2, 3]
I’m not quite sure if I understand what’s happening here. Does this mean that the scope of the array is outside of the function? Why does the array remember its values from call to call? Coming from other languages, I would expect this behavior only if the variable was static. Otherwise it seems it should be reset each time. And actually, when I tried the following:
def f(a):
L = []
L.append(a)
return L
I got the behavior I expected (the array was reset on each call).
So it seems to me that I just need the line def f(a, L=[]): explained – what is the scope of the L variable?
The scope is as you would expect.
The perhaps surprising thing is that the default value is only calculated once and reused, so each time you call the function you get the same list, not a new list initialized to [].
The list is stored in f.__defaults__ (or f.func_defaults in Python 2.)
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
print f.__defaults__
f.__defaults__ = (['foo'],) # Don't do this!
print f(4)
Result:
[1]
[1, 2]
[1, 2, 3]
([1, 2, 3],)
['foo', 4]
The scope of the L variable is behaving as you expect.
The “problem” is with the list you’re creating with []. Python does not create a new list each time you call the function. L gets assigned the same list each time you call which is why the function “remembers” previous calls.
So in effect this is what you have:
mylist = []
def f(a, L=mylist):
L.append(a)
return L
The Python Tutorial puts it this way:
The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes.
and suggests the following way to code the expected behaviour:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
There’s even less “magic” than you might suspect. This is equivalent to
m = []
def f(a, L=m):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
m is only created once.
The “problem” here is that L=[] is only evaluated once, that is, when the file is compiled. Python steps through each line of the file and compiles it. By the time it reaches the def with the default parameter, it creates that list instance once.
If you put L = [] inside the function code, the instance is not created at “compile time” (actually compile time can also be called part of the run time) because Python compiles the function’s code but does not call it. So you will get a new list instance because the creation is done every time you call the function (instead of once during compilation).
A solution for that problem is not to use mutable objects as default parameters, or only fixed instances like None:
def f(a, L = None):
if l == None:
l = []
L.append(a)
return L
Note that in both cases you described, the scope of L is the function scope.
The explaination is given in answers to this question. To sum it up here:
Functions in Python are a kind of object. Because they are a kind of object, they act like objects when instantiated. A function, if defined with a mutable attribute as a default argument, is exactly the same as a class with a static attribute that is a mutable list.
Lennart Regebro has a good explanation and the answer to the question by Roberto Liffredo is excellent.
To adapt Lennart’s answer … if I have a BananaBunch class:
class BananaBunch:
bananas = []
def addBanana(self, banana):
self.bananas.append(banana)
bunch = BananaBunch()
>>> bunch
<__main__.BananaBunch instance at 0x011A7FA8>
>>> bunch.addBanana(1)
>>> bunch.bananas
[1]
>>> for i in range(6):
bunch.addBanana("Banana #" + i)
>>> for i in range(6):
bunch.addBanana("Banana #" + str(i))
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5']
// And for review ...
//If I then add something to the BananaBunch class ...
>>> BananaBunch.bananas.append("A mutated banana")
//My own bunch is suddenly corrupted. :-)
>>> bunch.bananas
[1, 'Banana #0', 'Banana #1', 'Banana #2', 'Banana #3', 'Banana #4', 'Banana #5', 'A mutated banana']
How does this apply to functions? Functions in Python are objects. This bears repeating. Functions in Python are objects.
So when you create a function, you are creating an object. When you give a function a mutable default value, you are populating that object’s attribute with a mutable value, and every time you call that function you are operating on the same attribute. So if you are using a mutable call (like append), then you are modifying the same object, just as if you were adding bananas to the bunch object.
You have to keep in mind that python is an interpreted language. What is happening here is when the function “f” is defined, it creates the list and assigns it to the default parameter “L” of function “f”. Later, when you call this function, the same list is used as the default parameter. In short, the code on the “def” line, only gets executed once when the function is defined. This is a common python pitfall, of which I have fallen in myself.
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
There have been suggestions for idioms in other answers here to fix this issue. The one I would suggest is as follows:
def f(a, L=None):
L = L or []
L.append(a)
return L
This uses the or short circuit to either take the “L” that was passed, or create a new list.
The answer to your scope question is the “L” only has a scope within the function “f”, but because the default parameters are only assigned once to a single list instead of every time you call the function it behaves as if the default parameter “L” has a global scope.
Say you have the following code:
def func(a=[]):
a.append(1)
print("A:", a)
func()
func()
func()
You can use python’s indentation to help you understand what’s going on. Everything that is flush to the left margin is executed when the file gets executed. Everything that’s indented is compiled into a code object which gets executed when func() is called. So the function is defined and its default arguments set once, when the program gets executed (because the def statement is flush left).
What it does with the default arguments is an interesting issue. In python 3, it puts most of the information about a function in two places: func.__code__ and func.__defaults__. In python 2, func.__code__ was func.func_code func.__defaults__ was func.func_defaults. Later versions of python 2, including 2.6 have both sets of names, to aid the transition from python 2 to python 3. I will use the more modern __code__ and __defaults__. If you’re stuck on an older python, the concepts are the same; just the names differ.
The default values are stored in func.__defaults__, and retrieved each time the function is called.
Thus when you define the function above, the body of the function gets compiled and stored in variables under __code__, to be executed later, and the default arguments get stored in __defaults__. When you call the function, it uses the values in __defaults__. If those values get modified for any reason, it only has the modified version available to use.
Play around defining different functions in the interactive interpreter, and see what you can figure out about how python creates and uses functions.