Change default arguments of function in python
Question:
I have a function that converts a numpy array to a array containing True or False based on a condition then groups the True or False entries that are adjacent to one another and calculates the length of each group. This is to determine the length of dry spells or wet spells in a given month of precipitation data.
This is the function:
import itertools
def spell(X, kind='wet', how='mean', threshold=0.5):
if kind=='wet':
condition = X>threshold
else:
condition = X<=threshold
length = [sum(1 if x==True else nan for x in group) for key,group in itertools.groupby(condition) if key]
if not length:
res = 0
elif how=='mean':
res = np.mean(length)
else:
res = np.max(length)
return res
So basically there is the option to determine the mean length or maximum length of wet or dry spells given a numpy array of precipitation data with the default parameters set to the mean length of wet spells.
I use this function with pandas to apply it to each month of a historical record:
#Create example dataframe
np.random.seed(1324)
idx = pd.DatetimeIndex(start='1960-01-01', periods=100, freq='d')
values = np.random.random(100)
df = pd.DataFrame(values, index=idx)
#Apply function
df.resample('M', how=spell)
and what I get is:
0
1960-01-31 1.555556
1960-02-29 1.500000
1960-03-31 1.777778
1960-04-30 6.000000
Which is perfect, however I want to be able to change the default values of this function somewhat on the fly so that I can use it’s other options with df.resample(). I’ve looked into functools.partial() however this is only a solution for cases where the input arguments are explicitly set ie. spell(kind='dry', how='max', threshold=0.7). Is there a way to change the default arguments of the function in a way they will not need to be explicitly set afterwords so that I can used it with df.resample()?
Answers:
This sounds like a job for a function wrapper!
def spellwrapper(newkind, newhow, newthreshold):
def wrapped_spell_func(X):
spell(X, kind=newkind, how=newhow, threshold=newthreshold)
return wrapped_spell_func
You would call this function with
new_spell_func = spellwrapper(newkind, newhow, newthreshold)
And it would return a wrapped version of the spell function that uses your new arguments as “defaults” instead of the ones created at the function definition. Then you would use
df.resample('M', how=new_spell_func)
The default values for a function are stored in that function’s func_defaults attribute, which is a tuple of values which pair up with the trailing elements of the function’s func_code.co_varnames tuple. For example:
>>> def foo(x, y=5):
... return x, y
...
>>> foo(10)
(10, 5)
>>> foo.func_code.co_varnames
('x', 'y')
>>> foo.func_defaults
(5,)
>>> foo.func_defaults = (7,)
>>> foo(10)
(10, 7)
You can even give a parameter a default value after the fact:
>>> foo.func_defaults = (2, 3)
>>> foo()
(2, 3)
Warning: I had thought to (ab)use the mock library to allow temporarily overriding the function defaults, in a similar manner to a recent answer of mine. However, it seems to leave the defaults set to None afterward, which means either there is a bug in (or I misunderstand the behavior of) mock, or that messing with functions like this is a little dangerous.
def foo(x=5):
return x
assert foo() == 5
with mock.patch.object(foo, 'func_defaults', (10,)):
assert foo() == 10
assert foo() == 5 # Oops; I'm observing foo.func_defaults to be None now
Manually saving and restoring the defaults seems to work fine, though, as you might expect.
orig_defaults = foo.func_defaults
foo.func_defaults = (10,)
assert foo() == 10
foo.func_defaults = orig_defaults
assert foo() == 5
You actually can use the partial function to modify the default keyword arguments so that they don’t need to be explicitly set later. The trick is to set the original function to be equal to the resulting partial function.
Generic example of partial
Define our generic function
def originalFunction(x,defaultValue='DEFAULT'):
print('x:', x, 'defaultValue:', defaultValue,)
## shows the original function outputing x and the defaultValue
originalFunction(1)
x: 1 defaultValue: DEFAULT
Use partial to overwrite function signature (THESE TOP TWO LINES ARE THE ONLY IMPORTANT PART)
from functools import partial
originalFunction = partial(originalFunction,defaultValue=999) ## Overwites the function with partial
## Use function with updated defaults
originalFunction(1)
x: 1 defaultValue: 999
To reset the default values you can either repeat the call to partial
originalFunction = partial(originalFunction,defaultValue="whateverYouWant") ##sets default value
## Use function with updated defaults
originalFunction(3)
x: 3 defaultValue: whateverYouWant
Or you can modify the partial function’s keywords dictionary like this
## Show dict containing keywords
originalFunction.keywords
{'defaultValue': 'whateverYouWant'}
originalFunction.keywords['defaultValue'] = "ThePossabilitesAreEndless" ##sets default value
## Use function with updated defaults
originalFunction("x")
x: x defaultValue: ThePossabilitesAreEndless
To revert to the original kwargs, you can overwrite the variable to the original function address found in .func
originalFunction = originalFunction.func ##resets the default
## shows the original function outputing x and the defaultValue
originalFunction(3)
x: 3 defaultValue: DEFAULT
Implamenting Partial in Original Question
Showing normal use of our function "spell"
import numpy as np
import itertools
import pandas as pd
def spell(X, kind='wet', how='mean', threshold=0.5):
print(kind,how,threshold) ##adding print to show kwargs
if kind=='wet':
condition = X>threshold
else:
condition = X<=threshold
length = [sum(1 if x==True else nan for x in group) for key,group in itertools.groupby(condition) if key]
if not length:
res = 0
elif how=='mean':
res = np.mean(length)
else:
res = np.max(length)
return res
np.random.seed(1324)
idx = pd.date_range(start='1960-01-01', periods=100, freq='d')
values = np.random.random(100)
df = pd.DataFrame(values, index=idx)
#Apply function
print(df.resample('M').agg(lambda x: spell(x)).to_markdown())
wet mean 0.5
wet mean 0.5
wet mean 0.5
wet mean 0.5
| | 0 |
|:--------------------|--------:|
| 1960-01-31 00:00:00 | 1.55556 |
| 1960-02-29 00:00:00 | 1.5 |
| 1960-03-31 00:00:00 | 1.77778 |
| 1960-04-30 00:00:00 | 6 |
updating spell kwargs with functools partial function
from functools import partial
spell = partial(spell,kind='dry', how='max', threshold=0.6)
#Apply function
print(df.resample('M').agg(lambda x: spell(x)).to_markdown())
dry max 0.6
dry max 0.6
dry max 0.6
dry max 0.6
| | 0 |
|:--------------------|----:|
| 1960-01-31 00:00:00 | 4 |
| 1960-02-29 00:00:00 | 5 |
| 1960-03-31 00:00:00 | 5 |
| 1960-04-30 00:00:00 | 2 |
Using functools vs editing __defaults__
The functools partial method is also useful when you are using a function that you have imported because functions within modules do not have __defaults__ like functions that are described in the current python script
Because we defined originalFunction, we can access the default kwargs using .defaults
##defaults from function we wrote
originalFunction.__defaults__
('DEFAULT',)
##editing defaults
originalFunction.__defaults__ = ('Edited',)
originalFunction(1)
x: 1 defaultValue: Edited
This method won’t work on an imported module like np.array
np.array.__defaults__ ## module functions do not have __defaults__
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
~AppDataLocalTemp/ipykernel_15784/3587101803.py in <module>
----> 1 np.array.__defaults__ ## module functions do not have __defaults__
AttributeError: 'builtin_function_or_method' object has no attribute '__defaults__'
so instead we can use the partial function to update the np.array kwargs
## how numpy.array normally works
np.array([1,1,1]).T
array([1, 1, 1])
## modify numpy.array ndmin default
from functools import partial
np.array = partial(np.array,ndmin = 2)
## how it works now
np.array([1,1,1]).T
array([[1],
[1],
[1]])
Using functools vs passing on kwargs
It is possible to pass on args and kwargs to another function instead of using partial
## definition of originalFuction
def originalFunction(x,defaultValue='DEFAULT'):
print('x:', x, 'defaultValue:', defaultValue,)
## shows the original function outputing x and the defaultValue
originalFunction(1)
x: 1 defaultValue: DEFAULT
We can pass on the incoming args and kwargs to another function while overwriting the defaultValue kwarg with the following method
def newFunction(*args,defaultValue='NEW_DEFAULT',**kwargs):
return originalFunction(*args,defaultValue = defaultValue, **kwargs)
newFunction(5)
x: 5 defaultValue: NEW_DEFAULT
One downside to this approach, is that you MUST NOT use this new function signature to overwrite the original function!
## add display() to show infinite loop
def newFunction(*args,defaultValue='NEW_DEFAULT',**kwargs):
display('Passing vals to OriginalFunction')
return originalFunction(*args,defaultValue = defaultValue, **kwargs)
newFunction(5)
'Passing vals to OriginalFunction'
x: 5 defaultValue: NEW_DEFAULT
If you do, it will create an INFINITE LOOP
originalFunction = newFunction
originalFunction(1)
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
I have a function that converts a numpy array to a array containing True or False based on a condition then groups the True or False entries that are adjacent to one another and calculates the length of each group. This is to determine the length of dry spells or wet spells in a given month of precipitation data.
This is the function:
import itertools
def spell(X, kind='wet', how='mean', threshold=0.5):
if kind=='wet':
condition = X>threshold
else:
condition = X<=threshold
length = [sum(1 if x==True else nan for x in group) for key,group in itertools.groupby(condition) if key]
if not length:
res = 0
elif how=='mean':
res = np.mean(length)
else:
res = np.max(length)
return res
So basically there is the option to determine the mean length or maximum length of wet or dry spells given a numpy array of precipitation data with the default parameters set to the mean length of wet spells.
I use this function with pandas to apply it to each month of a historical record:
#Create example dataframe
np.random.seed(1324)
idx = pd.DatetimeIndex(start='1960-01-01', periods=100, freq='d')
values = np.random.random(100)
df = pd.DataFrame(values, index=idx)
#Apply function
df.resample('M', how=spell)
and what I get is:
0
1960-01-31 1.555556
1960-02-29 1.500000
1960-03-31 1.777778
1960-04-30 6.000000
Which is perfect, however I want to be able to change the default values of this function somewhat on the fly so that I can use it’s other options with df.resample(). I’ve looked into functools.partial() however this is only a solution for cases where the input arguments are explicitly set ie. spell(kind='dry', how='max', threshold=0.7). Is there a way to change the default arguments of the function in a way they will not need to be explicitly set afterwords so that I can used it with df.resample()?
This sounds like a job for a function wrapper!
def spellwrapper(newkind, newhow, newthreshold):
def wrapped_spell_func(X):
spell(X, kind=newkind, how=newhow, threshold=newthreshold)
return wrapped_spell_func
You would call this function with
new_spell_func = spellwrapper(newkind, newhow, newthreshold)
And it would return a wrapped version of the spell function that uses your new arguments as “defaults” instead of the ones created at the function definition. Then you would use
df.resample('M', how=new_spell_func)
The default values for a function are stored in that function’s func_defaults attribute, which is a tuple of values which pair up with the trailing elements of the function’s func_code.co_varnames tuple. For example:
>>> def foo(x, y=5):
... return x, y
...
>>> foo(10)
(10, 5)
>>> foo.func_code.co_varnames
('x', 'y')
>>> foo.func_defaults
(5,)
>>> foo.func_defaults = (7,)
>>> foo(10)
(10, 7)
You can even give a parameter a default value after the fact:
>>> foo.func_defaults = (2, 3)
>>> foo()
(2, 3)
Warning: I had thought to (ab)use the mock library to allow temporarily overriding the function defaults, in a similar manner to a recent answer of mine. However, it seems to leave the defaults set to None afterward, which means either there is a bug in (or I misunderstand the behavior of) mock, or that messing with functions like this is a little dangerous.
def foo(x=5):
return x
assert foo() == 5
with mock.patch.object(foo, 'func_defaults', (10,)):
assert foo() == 10
assert foo() == 5 # Oops; I'm observing foo.func_defaults to be None now
Manually saving and restoring the defaults seems to work fine, though, as you might expect.
orig_defaults = foo.func_defaults
foo.func_defaults = (10,)
assert foo() == 10
foo.func_defaults = orig_defaults
assert foo() == 5
You actually can use the partial function to modify the default keyword arguments so that they don’t need to be explicitly set later. The trick is to set the original function to be equal to the resulting partial function.
Generic example of partial
Define our generic function
def originalFunction(x,defaultValue='DEFAULT'):
print('x:', x, 'defaultValue:', defaultValue,)
## shows the original function outputing x and the defaultValue
originalFunction(1)
x: 1 defaultValue: DEFAULT
Use partial to overwrite function signature (THESE TOP TWO LINES ARE THE ONLY IMPORTANT PART)
from functools import partial
originalFunction = partial(originalFunction,defaultValue=999) ## Overwites the function with partial
## Use function with updated defaults
originalFunction(1)
x: 1 defaultValue: 999
To reset the default values you can either repeat the call to partial
originalFunction = partial(originalFunction,defaultValue="whateverYouWant") ##sets default value
## Use function with updated defaults
originalFunction(3)
x: 3 defaultValue: whateverYouWant
Or you can modify the partial function’s keywords dictionary like this
## Show dict containing keywords
originalFunction.keywords
{'defaultValue': 'whateverYouWant'}
originalFunction.keywords['defaultValue'] = "ThePossabilitesAreEndless" ##sets default value
## Use function with updated defaults
originalFunction("x")
x: x defaultValue: ThePossabilitesAreEndless
To revert to the original kwargs, you can overwrite the variable to the original function address found in .func
originalFunction = originalFunction.func ##resets the default
## shows the original function outputing x and the defaultValue
originalFunction(3)
x: 3 defaultValue: DEFAULT
Implamenting Partial in Original Question
Showing normal use of our function "spell"
import numpy as np
import itertools
import pandas as pd
def spell(X, kind='wet', how='mean', threshold=0.5):
print(kind,how,threshold) ##adding print to show kwargs
if kind=='wet':
condition = X>threshold
else:
condition = X<=threshold
length = [sum(1 if x==True else nan for x in group) for key,group in itertools.groupby(condition) if key]
if not length:
res = 0
elif how=='mean':
res = np.mean(length)
else:
res = np.max(length)
return res
np.random.seed(1324)
idx = pd.date_range(start='1960-01-01', periods=100, freq='d')
values = np.random.random(100)
df = pd.DataFrame(values, index=idx)
#Apply function
print(df.resample('M').agg(lambda x: spell(x)).to_markdown())
wet mean 0.5
wet mean 0.5
wet mean 0.5
wet mean 0.5
| | 0 |
|:--------------------|--------:|
| 1960-01-31 00:00:00 | 1.55556 |
| 1960-02-29 00:00:00 | 1.5 |
| 1960-03-31 00:00:00 | 1.77778 |
| 1960-04-30 00:00:00 | 6 |
updating spell kwargs with functools partial function
from functools import partial
spell = partial(spell,kind='dry', how='max', threshold=0.6)
#Apply function
print(df.resample('M').agg(lambda x: spell(x)).to_markdown())
dry max 0.6
dry max 0.6
dry max 0.6
dry max 0.6
| | 0 |
|:--------------------|----:|
| 1960-01-31 00:00:00 | 4 |
| 1960-02-29 00:00:00 | 5 |
| 1960-03-31 00:00:00 | 5 |
| 1960-04-30 00:00:00 | 2 |
Using functools vs editing __defaults__
The functools partial method is also useful when you are using a function that you have imported because functions within modules do not have __defaults__ like functions that are described in the current python script
Because we defined originalFunction, we can access the default kwargs using .defaults
##defaults from function we wrote
originalFunction.__defaults__
('DEFAULT',)
##editing defaults
originalFunction.__defaults__ = ('Edited',)
originalFunction(1)
x: 1 defaultValue: Edited
This method won’t work on an imported module like np.array
np.array.__defaults__ ## module functions do not have __defaults__
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
~AppDataLocalTemp/ipykernel_15784/3587101803.py in <module>
----> 1 np.array.__defaults__ ## module functions do not have __defaults__
AttributeError: 'builtin_function_or_method' object has no attribute '__defaults__'
so instead we can use the partial function to update the np.array kwargs
## how numpy.array normally works
np.array([1,1,1]).T
array([1, 1, 1])
## modify numpy.array ndmin default
from functools import partial
np.array = partial(np.array,ndmin = 2)
## how it works now
np.array([1,1,1]).T
array([[1],
[1],
[1]])
Using functools vs passing on kwargs
It is possible to pass on args and kwargs to another function instead of using partial
## definition of originalFuction
def originalFunction(x,defaultValue='DEFAULT'):
print('x:', x, 'defaultValue:', defaultValue,)
## shows the original function outputing x and the defaultValue
originalFunction(1)
x: 1 defaultValue: DEFAULT
We can pass on the incoming args and kwargs to another function while overwriting the defaultValue kwarg with the following method
def newFunction(*args,defaultValue='NEW_DEFAULT',**kwargs):
return originalFunction(*args,defaultValue = defaultValue, **kwargs)
newFunction(5)
x: 5 defaultValue: NEW_DEFAULT
One downside to this approach, is that you MUST NOT use this new function signature to overwrite the original function!
## add display() to show infinite loop
def newFunction(*args,defaultValue='NEW_DEFAULT',**kwargs):
display('Passing vals to OriginalFunction')
return originalFunction(*args,defaultValue = defaultValue, **kwargs)
newFunction(5)
'Passing vals to OriginalFunction'
x: 5 defaultValue: NEW_DEFAULT
If you do, it will create an INFINITE LOOP
originalFunction = newFunction
originalFunction(1)
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'
'Passing vals to OriginalFunction'