Should a class convert types of the parameters at init time? If so, how?
Question:
I’ve defined a class with 5 instance variables
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
# rest of init code
I want to ensure:
rating
is an int
name
is a str
lat
, long
, elev
are floats
When reading my input file, everything works creating a list of objects based on my class. When I start comparing values I got weird results since the instance variables were still strings.
Is the “most Pythonic way” to cast the values as the object is being created using int(string)
and float(string)
when calling the constructor or should this casting be done with logic inside the class?
Answers:
Personally, I would do any string parsing before passing the values to the constructor, unless parsing is one (or the) explicitly stated responsibility of the class. I prefer my program to fail because I didn’t explicitly cast a value than to be too flexible and end up in a Javascript-like 0 == "0"
situation. That said, if you want to accept strings as parameters you can just call int(my_parameter)
or float(my_parameter)
as needed in the constructor and that will make sure this are numbers not matter you pass a number, a string or even a Boolean.
In case you want to know more about type safety in Python, you can take a look at type annotations, which are supported by type checkers like mypy, and the traits package for type safety in class attributes.
Define custom field types
One way is to define your own field types and do the conversion and error handling in them. The fields are going to be based on descriptors. This is something you’re going to find in Django models, Flask-SQLAlchemy, DRF-Fields etc
Having such custom fields will allow you to cast them, validate them and this will work not just in __init__
, but anywhere we try to assign a value to it.
class Field:
type = None
def __init__(self, default=None):
self.value = default
def __get__(self, instance, cls):
if instance is None:
return self
return self.value
def __set__(self, instance, value):
# Here we could either try to cast the value to
# desired type or validate it and throw an error
# depending on the requirement.
try:
self.value = self.type(value)
except Exception:
raise ValueError('Failed to cast {value!r} to {type}'.format(
value=value, type=self.type
))
class IntField(Field):
type = int
class FloatField(Field):
type = float
class StrField(Field):
type = str
class PassPredictData:
rating = IntField()
name = StrField()
lat = FloatField()
long = FloatField()
elev = FloatField()
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
Demo:
>>> p = PassPredictData(1.2, 'foo', 1.1, 1.2, 1.3)
>>> p.lat = '123'
>>> p.lat
123.0
>>> p.lat = 'foo'
...
ValueError: Failed to cast 'foo' to <class 'float'>
>>> p.name = 123
>>> p.name
'123'
Use a static analyzer
Another option is to use static analyzers like Mypy and catch the errors before the program gets executed. The code below is using Python 3.6 syntax, but you can make it work with other versions as well by making some changes.
class PassPredictData:
rating: int
name: str
lat: float
long: float
elev: float
def __init__(self, rating: int, name: str, lat: float, long: float, elev: float) -> None:
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
PassPredictData(1, 2, 3, 4, 5)
PassPredictData(1, 'spam', 3.1, 4.2, 5.3)
PassPredictData(1.2, 'spam', 3.1, 4.2, 5)
When we run Mypy on this we get:
/so.py:15: error: Argument 2 to "PassPredictData" has incompatible type "int"; expected "str"
/so.py:17: error: Argument 1 to "PassPredictData" has incompatible type "float"; expected "int"
If you type import this
in the Python interpreter, you will get "The Zen of Python, by Tim Peters". The first three lines seem to apply to your situation:
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
I would recommend implementing your class like this:
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = int(rating)
self.name = str(name)
self.lat = float(lat)
self.long = float(long)
self.elev = float(elev)
This is the implementation you mention in your question. It is simple and explicit. Beauty is in the eye of the beholder.
Responses to Comments
The implementation is explicit to the writer of the class versus some other solution that hides the type conversion behind some opaque mechanism.
There is a valid argument that it is not obvious from the function signature what the expected parameter types are. However, the question implies that all parameters are passed as strings. In that case, the expected type is str
for all the constructor parameters. Perhaps the question title does not clearly describe the problem. Maybe a better title would be "Enforce Instance Variable Types When Passing Strings as Parameters to Constructor".
Note
I encourage folks to look at the revision history of the question to see the whole story.
EDIT: (edit because the topic of the question changed) I would not recommend convert type of parameters at init time. For example:
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = int(rating)
self.name = str(name)
...
In my opinion, This type of Implicit conversion is dangerous for few reasons.
- Implicitly converts parameter type to another without giving warning is very misleading
- It won’t raise any exceptions if users pass in undesired type. This goes hand in hand with the implicit casting. This could be avoided using explicit type checking.
- Silently convert type violates duck-typing
Instead of convert type of parameters, it is better to check the parameter type at init time. This approach would avoid the above three issues. To accomplish this you may use the strong type checking from typedecorator I like it because it is simple and very readable
For Python2 [edit: leaving this as a reference as OP requested]
from typedecorator import params, returns, setup_typecheck, void, typed
class PassPredictData:
@void
@params(self=object, rating = int, name = str, lat = float, long = float, elev = float)
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
setup_typecheck()
x = PassPredictData(1, "derp" , 6.8 , 9.8, 7.6) #works fine
x1 = PassPredictData(1.8, "derp" , 6.8 , 9.8, 7.6) #TypeError: argument rating = 1.8 doesn't match signature int
x2 = PassPredictData(1, "derp" , "gagaga" , 9.8, 7.6) #TypeError: argument lat = 'gagaga' doesn't match signature float
x3 = PassPredictData(1, 5 , 6.8 , 9.8, 7.6) #TypeError: argument name = 5 doesn't match signature str
For Python3 you can use the annotation syntax:
class PassPredictData1:
@typed
def __init__(self : object, rating : int, name : str, lat : float, long : float, elev : float):
self.rating = rating
setup_typecheck()
x = PassPredictData1(1, 5, 4, 9.8, 7.6)
throws an error:
TypeError: argument name = 5 doesn’t match signature str
Seems like there’s a million ways to do this, but here’s the formula I use:
class PassPredictData(object):
types = {'lat' : float,
'long' : float,
'elev' : float,
'rating': int,
'name' : str,
}
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
[rest of init code]
@classmethod
def from_string(cls, string):
[code to parse your string into a dict]
typed = {k: cls.types[k](v) for k, v in parsed.items()}
return cls(**typed)
A thing that’s nice about this: you can directly use a re.groupdict()
to produce your dict (as an example):
parsed = re.search('(?P<name>w): Latitude: (?P<lat>d+), Longitude: (?P<long>d+), Elevation: (?P<elev>d+) meters. (?P<rating>d)', some_string).groupdict()
In Python 3.5+ you can use type hints and the typing module.
class PassPredictData:
def __init__(self, rating: int, name: str, lat: float, long: float, elev: float):
self.rating = rating
#rest of init code
Note that these are just hints. Python doesn’t actually do anything with them like showing an error if the wrong type is used.
Even without relying on external libraries you can define your own simple typechecking decorator in just a few lines. This uses the inspect
module from Core-Python to get the parameter names, but even without it you could just zip
the args
with a list of types, although this will make using kwargs
difficult.
import inspect
def typecheck(**types):
def __f(f):
def _f(*args, **kwargs):
all_args = {n: a for a, n in zip(args, inspect.getargspec(f).args)}
all_args.update(kwargs)
for n, a in all_args.items():
t = types.get(n)
if t is not None and not isinstance(a, t):
print("WARNING: Expected {} for {}, got {}".format(t, n, a))
return f(*args, **kwargs)
return _f
return __f
class PassPredictData:
@typecheck(rating=int, name=str, elev=float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
self.rating = rating
p = PassPredictData(5.1, "foo", elev=4)
# WARNING: Expected <class 'int'> for rating, got 5.1
# WARNING: Expected <class 'float'> for elev, got 4
Instead of printing a warning, you could of course also raise an exception. Or, using the same approach, you could also just (try to) cast the parameters to the expected type:
def typecast(**types):
def __f(f):
def _f(*args, **kwargs):
all_args = {n: a for a, n in zip(args, inspect.getargspec(f).args)}
all_args.update(kwargs)
for n, a in all_args.items():
t = types.get(n)
if t is not None:
all_args[n] = t(a) # instead of checking, just cast
return f(**all_args) # pass the map with the typecast params
return _f
return __f
class PassPredictData:
@typecast(rating=int, name=str, elev=float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
print([rating, name, lat, long, elev])
p = PassPredictData("5", "foo", elev="3.14")
# Output of print: [5, 'foo', 0.0, 0.0, 3.14]
Or a simpler version, without inspect
, but not working for kwargs
and requiring to provide the type for each parameter, including self
(or None
for no type cast):
def typecast(*types):
def __f(f):
def _f(*args):
return f(*[t(a) if t is not None else a
for a, t in zip(args, types)])
return _f
return __f
class PassPredictData:
@typecast(None, int, str, float, float, float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
print([rating, name, lat, long, elev])
I’ve defined a class with 5 instance variables
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
# rest of init code
I want to ensure:
rating
is an intname
is a strlat
,long
,elev
are floats
When reading my input file, everything works creating a list of objects based on my class. When I start comparing values I got weird results since the instance variables were still strings.
Is the “most Pythonic way” to cast the values as the object is being created using int(string)
and float(string)
when calling the constructor or should this casting be done with logic inside the class?
Personally, I would do any string parsing before passing the values to the constructor, unless parsing is one (or the) explicitly stated responsibility of the class. I prefer my program to fail because I didn’t explicitly cast a value than to be too flexible and end up in a Javascript-like 0 == "0"
situation. That said, if you want to accept strings as parameters you can just call int(my_parameter)
or float(my_parameter)
as needed in the constructor and that will make sure this are numbers not matter you pass a number, a string or even a Boolean.
In case you want to know more about type safety in Python, you can take a look at type annotations, which are supported by type checkers like mypy, and the traits package for type safety in class attributes.
Define custom field types
One way is to define your own field types and do the conversion and error handling in them. The fields are going to be based on descriptors. This is something you’re going to find in Django models, Flask-SQLAlchemy, DRF-Fields etc
Having such custom fields will allow you to cast them, validate them and this will work not just in __init__
, but anywhere we try to assign a value to it.
class Field:
type = None
def __init__(self, default=None):
self.value = default
def __get__(self, instance, cls):
if instance is None:
return self
return self.value
def __set__(self, instance, value):
# Here we could either try to cast the value to
# desired type or validate it and throw an error
# depending on the requirement.
try:
self.value = self.type(value)
except Exception:
raise ValueError('Failed to cast {value!r} to {type}'.format(
value=value, type=self.type
))
class IntField(Field):
type = int
class FloatField(Field):
type = float
class StrField(Field):
type = str
class PassPredictData:
rating = IntField()
name = StrField()
lat = FloatField()
long = FloatField()
elev = FloatField()
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
Demo:
>>> p = PassPredictData(1.2, 'foo', 1.1, 1.2, 1.3)
>>> p.lat = '123'
>>> p.lat
123.0
>>> p.lat = 'foo'
...
ValueError: Failed to cast 'foo' to <class 'float'>
>>> p.name = 123
>>> p.name
'123'
Use a static analyzer
Another option is to use static analyzers like Mypy and catch the errors before the program gets executed. The code below is using Python 3.6 syntax, but you can make it work with other versions as well by making some changes.
class PassPredictData:
rating: int
name: str
lat: float
long: float
elev: float
def __init__(self, rating: int, name: str, lat: float, long: float, elev: float) -> None:
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
PassPredictData(1, 2, 3, 4, 5)
PassPredictData(1, 'spam', 3.1, 4.2, 5.3)
PassPredictData(1.2, 'spam', 3.1, 4.2, 5)
When we run Mypy on this we get:
/so.py:15: error: Argument 2 to "PassPredictData" has incompatible type "int"; expected "str"
/so.py:17: error: Argument 1 to "PassPredictData" has incompatible type "float"; expected "int"
If you type import this
in the Python interpreter, you will get "The Zen of Python, by Tim Peters". The first three lines seem to apply to your situation:
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
I would recommend implementing your class like this:
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = int(rating)
self.name = str(name)
self.lat = float(lat)
self.long = float(long)
self.elev = float(elev)
This is the implementation you mention in your question. It is simple and explicit. Beauty is in the eye of the beholder.
Responses to Comments
The implementation is explicit to the writer of the class versus some other solution that hides the type conversion behind some opaque mechanism.
There is a valid argument that it is not obvious from the function signature what the expected parameter types are. However, the question implies that all parameters are passed as strings. In that case, the expected type is str
for all the constructor parameters. Perhaps the question title does not clearly describe the problem. Maybe a better title would be "Enforce Instance Variable Types When Passing Strings as Parameters to Constructor".
Note
I encourage folks to look at the revision history of the question to see the whole story.
EDIT: (edit because the topic of the question changed) I would not recommend convert type of parameters at init time. For example:
class PassPredictData:
def __init__(self, rating, name, lat, long, elev):
self.rating = int(rating)
self.name = str(name)
...
In my opinion, This type of Implicit conversion is dangerous for few reasons.
- Implicitly converts parameter type to another without giving warning is very misleading
- It won’t raise any exceptions if users pass in undesired type. This goes hand in hand with the implicit casting. This could be avoided using explicit type checking.
- Silently convert type violates duck-typing
Instead of convert type of parameters, it is better to check the parameter type at init time. This approach would avoid the above three issues. To accomplish this you may use the strong type checking from typedecorator I like it because it is simple and very readable
For Python2 [edit: leaving this as a reference as OP requested]
from typedecorator import params, returns, setup_typecheck, void, typed
class PassPredictData:
@void
@params(self=object, rating = int, name = str, lat = float, long = float, elev = float)
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
self.name = name
self.lat = lat
self.long = long
self.elev = elev
setup_typecheck()
x = PassPredictData(1, "derp" , 6.8 , 9.8, 7.6) #works fine
x1 = PassPredictData(1.8, "derp" , 6.8 , 9.8, 7.6) #TypeError: argument rating = 1.8 doesn't match signature int
x2 = PassPredictData(1, "derp" , "gagaga" , 9.8, 7.6) #TypeError: argument lat = 'gagaga' doesn't match signature float
x3 = PassPredictData(1, 5 , 6.8 , 9.8, 7.6) #TypeError: argument name = 5 doesn't match signature str
For Python3 you can use the annotation syntax:
class PassPredictData1:
@typed
def __init__(self : object, rating : int, name : str, lat : float, long : float, elev : float):
self.rating = rating
setup_typecheck()
x = PassPredictData1(1, 5, 4, 9.8, 7.6)
throws an error:
TypeError: argument name = 5 doesn’t match signature str
Seems like there’s a million ways to do this, but here’s the formula I use:
class PassPredictData(object):
types = {'lat' : float,
'long' : float,
'elev' : float,
'rating': int,
'name' : str,
}
def __init__(self, rating, name, lat, long, elev):
self.rating = rating
[rest of init code]
@classmethod
def from_string(cls, string):
[code to parse your string into a dict]
typed = {k: cls.types[k](v) for k, v in parsed.items()}
return cls(**typed)
A thing that’s nice about this: you can directly use a re.groupdict()
to produce your dict (as an example):
parsed = re.search('(?P<name>w): Latitude: (?P<lat>d+), Longitude: (?P<long>d+), Elevation: (?P<elev>d+) meters. (?P<rating>d)', some_string).groupdict()
In Python 3.5+ you can use type hints and the typing module.
class PassPredictData:
def __init__(self, rating: int, name: str, lat: float, long: float, elev: float):
self.rating = rating
#rest of init code
Note that these are just hints. Python doesn’t actually do anything with them like showing an error if the wrong type is used.
Even without relying on external libraries you can define your own simple typechecking decorator in just a few lines. This uses the inspect
module from Core-Python to get the parameter names, but even without it you could just zip
the args
with a list of types, although this will make using kwargs
difficult.
import inspect
def typecheck(**types):
def __f(f):
def _f(*args, **kwargs):
all_args = {n: a for a, n in zip(args, inspect.getargspec(f).args)}
all_args.update(kwargs)
for n, a in all_args.items():
t = types.get(n)
if t is not None and not isinstance(a, t):
print("WARNING: Expected {} for {}, got {}".format(t, n, a))
return f(*args, **kwargs)
return _f
return __f
class PassPredictData:
@typecheck(rating=int, name=str, elev=float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
self.rating = rating
p = PassPredictData(5.1, "foo", elev=4)
# WARNING: Expected <class 'int'> for rating, got 5.1
# WARNING: Expected <class 'float'> for elev, got 4
Instead of printing a warning, you could of course also raise an exception. Or, using the same approach, you could also just (try to) cast the parameters to the expected type:
def typecast(**types):
def __f(f):
def _f(*args, **kwargs):
all_args = {n: a for a, n in zip(args, inspect.getargspec(f).args)}
all_args.update(kwargs)
for n, a in all_args.items():
t = types.get(n)
if t is not None:
all_args[n] = t(a) # instead of checking, just cast
return f(**all_args) # pass the map with the typecast params
return _f
return __f
class PassPredictData:
@typecast(rating=int, name=str, elev=float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
print([rating, name, lat, long, elev])
p = PassPredictData("5", "foo", elev="3.14")
# Output of print: [5, 'foo', 0.0, 0.0, 3.14]
Or a simpler version, without inspect
, but not working for kwargs
and requiring to provide the type for each parameter, including self
(or None
for no type cast):
def typecast(*types):
def __f(f):
def _f(*args):
return f(*[t(a) if t is not None else a
for a, t in zip(args, types)])
return _f
return __f
class PassPredictData:
@typecast(None, int, str, float, float, float)
def __init__(self, rating, name, lat=0.0, long=0.0, elev=0.0):
print([rating, name, lat, long, elev])