How do I elegantly/efficiently write the __init__ function for a Python class that takes lots of instance variables?
Question:
Let’s say you have a class that takes many (keyword) arguments, most of which are meant to be stored as instance variables:
class ManyInitVariables():
def __init__(a=0, b=2, c=1, d=0, e=-1, ... , x=100, y=0, z=9):
How would you initialize them in __init__? You could do something like this:
class ManyInitVariables():
def __init__(a=0, b=2, c=1, d=0, e=-1, ... , x=100, y=0, z=9):
self.a = a
self.b = b
self.c = c
...
self.z = z
…but it would take a lot of typing! How could I get __init__ to automatically some of the arguments it takes, noting that other arguments may not need to be assigned as instance variables?
Answers:
I’m sure there are many other similar solutions out there on the web for this very common issue, but this is one, for example:
import functools
import inspect
def absorb_args(f):
args, _, _, defs = inspect.getargspec(f)
args = args[1:] # ignore the leading `self`
@functools.wraps(f)
def do_absorb(self, *a, **k):
ndefs = len(args) - len(a) + 2
for name, value in zip(args, a + defs[-ndefs:]):
setattr(self, name, value)
for name, value in k.items():
setattr(self, name, value)
return f(self, *a, **k)
return do_absorb
Added: I’ve been asked to explain this further, but, there’s a lot going on here if you’re not skilled at Python!-).
functools.wraps is a decorator to help make better decorators, see https://docs.python.org/2/library/functools.html#functools.wraps — not directly germane to the question but useful to support interactive help and tools based on functions’ docstrings. Get into the habit of always using it when writing a function decorator that (the most common case) wraps the decorated function, and you won’t regret it.
The inspect module is the only right way to do introspection in modern Python. inspect.getargspec in particular gives you information on what arguments a function accepts, and what the default values for them are, if any (the two bits of info I’m ignoring, by assigning them to _, are about *a and **k special args, which this decorator doesn’t support). See https://docs.python.org/2/library/inspect.html?highlight=getargspec#inspect.getargspec for more.
self, by convention, is always the first arg to a method (and this decorator is meant for methods only:-). So, the first for loop deals with positional args (whether explicitly given in the call or defaulting to default values); then, the second for loop deals with named args (that one, I hope, is simpler to grasp:-). setattr of course is the precious built-in function which sets an attribute with a variable name, https://docs.python.org/2/library/functions.html?highlight=setattr#setattr for more.
Incidentally, if you only care to use this in __init__ (as you see in the example below, absorb_attrs per se has no such constraint), then write a class decorator which singles out the class’s __init__ for this treatment, and apply that class decorator to the class itself.
Also, if your class’s __init__ has no work left to do once args are “absorbed” in this way, you must still define the (decorated) __init__, but its body can be limited to a docstring explaining the arguments (I personally prefer to always also have a pass statement in such cases, but that’s a personal style issue).
And now, back to the original answer, with an example…!-)
And then, e.g, something like
class Struggle(object):
@absorb_args
def __init__(self, a, b, c, bab='bo', bip='bop'):
self.d = a + b
@absorb_args
def setit(self, x, y, z, u=23, w=45):
self.t = x + y
def __str__(self):
attrs = sorted(self.__dict__)
r = ['%s: %s' % (a, getattr(self, a)) for a in attrs]
return ', '.join(r)
s = Struggle('fee', 'fie', 'foo', bip='beeeeeep')
s.setit(1, 2, 3, w=99)
print(s)
would print
a: fee, b: fie, bab: bo, bip: beeeeeep, c: foo, d: feefie, t: 3, u: 23, w: 99, x: 1, y: 2, z: 3
as desired.
My only excuse for “reinventing the wheel” this way (rather than scouring the web for a solution) is that the other evening my wife and co-author Anna (only ever woman winner of the Frank Willison Memorial Award for contribution to the Python community, BTW:-) asked me about it (we are, slowly alas!, writing the 3rd edition of “Python in a Nutshell”) — it took me 10 minutes to code this (sorry, no tests yet:-) while in the same 10 minutes she (despite being a very skilled web-searcher:-) could not locate an existing solution on the web. And, this way I need not worry about copyright issues if I want to post it here, include it in the next Nutshell, present about it at OSCON or Pycon, and so forth…:-)
A couple of years later, Python 3.7 introduced the dataclass, which allows you to define a class like this:
from dataclasses import dataclass
@dataclass
class ManyInitVariables:
a: int = 0
b: int = 2
...
z: 'typing.Any' = 9
This will automatically take care of setting the instance attributes.
There’s also make_dataclass:
from dataclasses import make_dataclass, field
attributes = [('a', int, 0), ('b', int, 2), ..., ('z', 'typing.Any', 9)]
ManyInitVariables = make_dataclass(
'ManyInitVariables',
[(attr_name, attr_type, field(default=attr_default))
for attr_name, attr_type, attr_default in attributes])
Arguments that should not be assigned as instance attributes can be marked as "init-only variables" using dataclasses.InitVar. These can then be processed in a __post_init__ method.
Let’s say you have a class that takes many (keyword) arguments, most of which are meant to be stored as instance variables:
class ManyInitVariables():
def __init__(a=0, b=2, c=1, d=0, e=-1, ... , x=100, y=0, z=9):
How would you initialize them in __init__? You could do something like this:
class ManyInitVariables():
def __init__(a=0, b=2, c=1, d=0, e=-1, ... , x=100, y=0, z=9):
self.a = a
self.b = b
self.c = c
...
self.z = z
…but it would take a lot of typing! How could I get __init__ to automatically some of the arguments it takes, noting that other arguments may not need to be assigned as instance variables?
I’m sure there are many other similar solutions out there on the web for this very common issue, but this is one, for example:
import functools
import inspect
def absorb_args(f):
args, _, _, defs = inspect.getargspec(f)
args = args[1:] # ignore the leading `self`
@functools.wraps(f)
def do_absorb(self, *a, **k):
ndefs = len(args) - len(a) + 2
for name, value in zip(args, a + defs[-ndefs:]):
setattr(self, name, value)
for name, value in k.items():
setattr(self, name, value)
return f(self, *a, **k)
return do_absorb
Added: I’ve been asked to explain this further, but, there’s a lot going on here if you’re not skilled at Python!-).
functools.wraps is a decorator to help make better decorators, see https://docs.python.org/2/library/functools.html#functools.wraps — not directly germane to the question but useful to support interactive help and tools based on functions’ docstrings. Get into the habit of always using it when writing a function decorator that (the most common case) wraps the decorated function, and you won’t regret it.
The inspect module is the only right way to do introspection in modern Python. inspect.getargspec in particular gives you information on what arguments a function accepts, and what the default values for them are, if any (the two bits of info I’m ignoring, by assigning them to _, are about *a and **k special args, which this decorator doesn’t support). See https://docs.python.org/2/library/inspect.html?highlight=getargspec#inspect.getargspec for more.
self, by convention, is always the first arg to a method (and this decorator is meant for methods only:-). So, the first for loop deals with positional args (whether explicitly given in the call or defaulting to default values); then, the second for loop deals with named args (that one, I hope, is simpler to grasp:-). setattr of course is the precious built-in function which sets an attribute with a variable name, https://docs.python.org/2/library/functions.html?highlight=setattr#setattr for more.
Incidentally, if you only care to use this in __init__ (as you see in the example below, absorb_attrs per se has no such constraint), then write a class decorator which singles out the class’s __init__ for this treatment, and apply that class decorator to the class itself.
Also, if your class’s __init__ has no work left to do once args are “absorbed” in this way, you must still define the (decorated) __init__, but its body can be limited to a docstring explaining the arguments (I personally prefer to always also have a pass statement in such cases, but that’s a personal style issue).
And now, back to the original answer, with an example…!-)
And then, e.g, something like
class Struggle(object):
@absorb_args
def __init__(self, a, b, c, bab='bo', bip='bop'):
self.d = a + b
@absorb_args
def setit(self, x, y, z, u=23, w=45):
self.t = x + y
def __str__(self):
attrs = sorted(self.__dict__)
r = ['%s: %s' % (a, getattr(self, a)) for a in attrs]
return ', '.join(r)
s = Struggle('fee', 'fie', 'foo', bip='beeeeeep')
s.setit(1, 2, 3, w=99)
print(s)
would print
a: fee, b: fie, bab: bo, bip: beeeeeep, c: foo, d: feefie, t: 3, u: 23, w: 99, x: 1, y: 2, z: 3
as desired.
My only excuse for “reinventing the wheel” this way (rather than scouring the web for a solution) is that the other evening my wife and co-author Anna (only ever woman winner of the Frank Willison Memorial Award for contribution to the Python community, BTW:-) asked me about it (we are, slowly alas!, writing the 3rd edition of “Python in a Nutshell”) — it took me 10 minutes to code this (sorry, no tests yet:-) while in the same 10 minutes she (despite being a very skilled web-searcher:-) could not locate an existing solution on the web. And, this way I need not worry about copyright issues if I want to post it here, include it in the next Nutshell, present about it at OSCON or Pycon, and so forth…:-)
A couple of years later, Python 3.7 introduced the dataclass, which allows you to define a class like this:
from dataclasses import dataclass
@dataclass
class ManyInitVariables:
a: int = 0
b: int = 2
...
z: 'typing.Any' = 9
This will automatically take care of setting the instance attributes.
There’s also make_dataclass:
from dataclasses import make_dataclass, field
attributes = [('a', int, 0), ('b', int, 2), ..., ('z', 'typing.Any', 9)]
ManyInitVariables = make_dataclass(
'ManyInitVariables',
[(attr_name, attr_type, field(default=attr_default))
for attr_name, attr_type, attr_default in attributes])
Arguments that should not be assigned as instance attributes can be marked as "init-only variables" using dataclasses.InitVar. These can then be processed in a __post_init__ method.