When should I subclass EnumMeta instead of Enum?
Question:
In this article Nick Coghlan talks about some of the design decisions that went in to the PEP 435 Enum
type, and how EnumMeta
can be subclassed to provide a different Enum
experience.
However, the advice I give (and I am the primary stdlib Enum
author) about using a metaclass is it should not be done without a really good reason — such as not being able to accomplish what you need with a class decorator, or a dedicated function to hide any ugliness; and in my own work I’ve been able to do whatever I needed simply by using __new__
, __init__
, and/or normal class/instance methods when creating the Enum
class:
And then there is this cautionary tale of being careful when delving into Enum
, with and without metaclass subclassing:
Given all that, when would I need to fiddle with EnumMeta
itself?
Answers:
The best cases I have seen so far for subclassing EnumMeta
comes from these four questions:
-
-
-
-
We’ll examine the dynamic member case further here.
First, a look at the code needed when not subclassing EnumMeta
:
The stdlib way
from enum import Enum
import json
class BaseCountry(Enum):
def __new__(cls, record):
member = object.__new__(cls)
member.country_name = record['name']
member.code = int(record['country-code'])
member.abbr = record['alpha-2']
member._value_ = member.abbr, member.code, member.country_name
if not hasattr(cls, '_choices'):
cls._choices = {}
cls._choices[member.code] = member.country_name
cls._choices[member.abbr] = member.country_name
return member
def __str__(self):
return self.country_name
Country = BaseCountry(
'Country',
[(rec['alpha-2'], rec) for rec in json.load(open('slim-2.json'))],
)
The aenum
way 1 2
from aenum import Enum, MultiValue
import json
class Country(Enum, init='abbr code country_name', settings=MultiValue):
_ignore_ = 'country this' # do not add these names as members
# create members
this = vars()
for country in json.load(open('slim-2.json')):
this[country['alpha-2']] = (
country['alpha-2'],
int(country['country-code']),
country['name'],
)
# have str() print just the country name
def __str__(self):
return self.country_name
The above code is fine for a one-off enumeration — but what if creating Enums from JSON files was common for you? Imagine if you could do this instead:
class Country(JSONEnum):
_init_ = 'abbr code country_name' # remove if not using aenum
_file = 'some_file.json'
_name = 'alpha-2'
_value = {
1: ('alpha-2', None),
2: ('country-code', lambda c: int(c)),
3: ('name', None),
}
As you can see:
_file
is the name of the json file to use
_name
is the path to whatever should be used for the name
_value
is a dictionary mapping paths to values3
_init_
specifies the attribute names for the different value components (if using aenum
)
The JSON data is taken from https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes — here is a short excerpt:
[{"name":"Afghanistan","alpha-2":"AF","country-code":"004"},
{"name":"Ă…land Islands","alpha-2":"AX","country-code":"248"},
{"name":"Albania","alpha-2":"AL","country-code":"008"},
{"name":"Algeria","alpha-2":"DZ","country-code":"012"}]
Here is the JSONEnumMeta
class:
class JSONEnumMeta(EnumMeta):
@classmethod
def __prepare__(metacls, cls, bases, **kwds):
# return a standard dictionary for the initial processing
return {}
def __init__(cls, *args , **kwds):
super(JSONEnumMeta, cls).__init__(*args)
def __new__(metacls, cls, bases, clsdict, **kwds):
import json
members = []
missing = [
name
for name in ('_file', '_name', '_value')
if name not in clsdict
]
if len(missing) in (1, 2):
# all three must be present or absent
raise TypeError('missing required settings: %r' % (missing, ))
if not missing:
# process
name_spec = clsdict.pop('_name')
if not isinstance(name_spec, (tuple, list)):
name_spec = (name_spec, )
value_spec = clsdict.pop('_value')
file = clsdict.pop('_file')
with open(file) as f:
json_data = json.load(f)
for data in json_data:
values = []
name = data[name_spec[0]]
for piece in name_spec[1:]:
name = name[piece]
for order, (value_path, func) in sorted(value_spec.items()):
if not isinstance(value_path, (list, tuple)):
value_path = (value_path, )
value = data[value_path[0]]
for piece in value_path[1:]:
value = value[piece]
if func is not None:
value = func(value)
values.append(value)
values = tuple(values)
members.append(
(name, values)
)
# get the real EnumDict
enum_dict = super(JSONEnumMeta, metacls).__prepare__(cls, bases, **kwds)
# transfer the original dict content, _items first
items = list(clsdict.items())
items.sort(key=lambda p: (0 if p[0][0] == '_' else 1, p))
for name, value in items:
enum_dict[name] = value
# add the members
for name, value in members:
enum_dict[name] = value
return super(JSONEnumMeta, metacls).__new__(metacls, cls, bases, enum_dict, **kwds)
# for use with both Python 2/3
JSONEnum = JSONEnumMeta('JsonEnum', (Enum, ), {})
A few notes:
-
JSONEnumMeta.__prepare__
returns a normal dict
-
EnumMeta.__prepare__
is used to get an instance of _EnumDict
— this is the proper way to get one
-
keys with a leading underscore are passed to the real _EnumDict
first as they may be needed when processing the enum members
-
Enum members are in the same order as they were in the file
1 Disclosure: I am the author of the Python stdlib Enum
, the enum34
backport, and the Advanced Enumeration (aenum
) library.
2 This requires aenum 2.0.5+
.
3 The keys are numeric to keep multiple values in order should your Enum
need more than one.
In this article Nick Coghlan talks about some of the design decisions that went in to the PEP 435 Enum
type, and how EnumMeta
can be subclassed to provide a different Enum
experience.
However, the advice I give (and I am the primary stdlib Enum
author) about using a metaclass is it should not be done without a really good reason — such as not being able to accomplish what you need with a class decorator, or a dedicated function to hide any ugliness; and in my own work I’ve been able to do whatever I needed simply by using __new__
, __init__
, and/or normal class/instance methods when creating the Enum
class:
And then there is this cautionary tale of being careful when delving into Enum
, with and without metaclass subclassing:
Given all that, when would I need to fiddle with EnumMeta
itself?
The best cases I have seen so far for subclassing EnumMeta
comes from these four questions:
We’ll examine the dynamic member case further here.
First, a look at the code needed when not subclassing EnumMeta
:
The stdlib way
from enum import Enum
import json
class BaseCountry(Enum):
def __new__(cls, record):
member = object.__new__(cls)
member.country_name = record['name']
member.code = int(record['country-code'])
member.abbr = record['alpha-2']
member._value_ = member.abbr, member.code, member.country_name
if not hasattr(cls, '_choices'):
cls._choices = {}
cls._choices[member.code] = member.country_name
cls._choices[member.abbr] = member.country_name
return member
def __str__(self):
return self.country_name
Country = BaseCountry(
'Country',
[(rec['alpha-2'], rec) for rec in json.load(open('slim-2.json'))],
)
The aenum
way 1 2
from aenum import Enum, MultiValue
import json
class Country(Enum, init='abbr code country_name', settings=MultiValue):
_ignore_ = 'country this' # do not add these names as members
# create members
this = vars()
for country in json.load(open('slim-2.json')):
this[country['alpha-2']] = (
country['alpha-2'],
int(country['country-code']),
country['name'],
)
# have str() print just the country name
def __str__(self):
return self.country_name
The above code is fine for a one-off enumeration — but what if creating Enums from JSON files was common for you? Imagine if you could do this instead:
class Country(JSONEnum):
_init_ = 'abbr code country_name' # remove if not using aenum
_file = 'some_file.json'
_name = 'alpha-2'
_value = {
1: ('alpha-2', None),
2: ('country-code', lambda c: int(c)),
3: ('name', None),
}
As you can see:
_file
is the name of the json file to use_name
is the path to whatever should be used for the name_value
is a dictionary mapping paths to values3_init_
specifies the attribute names for the different value components (if usingaenum
)
The JSON data is taken from https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes — here is a short excerpt:
[{"name":"Afghanistan","alpha-2":"AF","country-code":"004"},
{"name":"Ă…land Islands","alpha-2":"AX","country-code":"248"},
{"name":"Albania","alpha-2":"AL","country-code":"008"},
{"name":"Algeria","alpha-2":"DZ","country-code":"012"}]
Here is the JSONEnumMeta
class:
class JSONEnumMeta(EnumMeta):
@classmethod
def __prepare__(metacls, cls, bases, **kwds):
# return a standard dictionary for the initial processing
return {}
def __init__(cls, *args , **kwds):
super(JSONEnumMeta, cls).__init__(*args)
def __new__(metacls, cls, bases, clsdict, **kwds):
import json
members = []
missing = [
name
for name in ('_file', '_name', '_value')
if name not in clsdict
]
if len(missing) in (1, 2):
# all three must be present or absent
raise TypeError('missing required settings: %r' % (missing, ))
if not missing:
# process
name_spec = clsdict.pop('_name')
if not isinstance(name_spec, (tuple, list)):
name_spec = (name_spec, )
value_spec = clsdict.pop('_value')
file = clsdict.pop('_file')
with open(file) as f:
json_data = json.load(f)
for data in json_data:
values = []
name = data[name_spec[0]]
for piece in name_spec[1:]:
name = name[piece]
for order, (value_path, func) in sorted(value_spec.items()):
if not isinstance(value_path, (list, tuple)):
value_path = (value_path, )
value = data[value_path[0]]
for piece in value_path[1:]:
value = value[piece]
if func is not None:
value = func(value)
values.append(value)
values = tuple(values)
members.append(
(name, values)
)
# get the real EnumDict
enum_dict = super(JSONEnumMeta, metacls).__prepare__(cls, bases, **kwds)
# transfer the original dict content, _items first
items = list(clsdict.items())
items.sort(key=lambda p: (0 if p[0][0] == '_' else 1, p))
for name, value in items:
enum_dict[name] = value
# add the members
for name, value in members:
enum_dict[name] = value
return super(JSONEnumMeta, metacls).__new__(metacls, cls, bases, enum_dict, **kwds)
# for use with both Python 2/3
JSONEnum = JSONEnumMeta('JsonEnum', (Enum, ), {})
A few notes:
-
JSONEnumMeta.__prepare__
returns a normaldict
-
EnumMeta.__prepare__
is used to get an instance of_EnumDict
— this is the proper way to get one -
keys with a leading underscore are passed to the real
_EnumDict
first as they may be needed when processing the enum members -
Enum members are in the same order as they were in the file
1 Disclosure: I am the author of the Python stdlib Enum
, the enum34
backport, and the Advanced Enumeration (aenum
) library.
2 This requires aenum 2.0.5+
.
3 The keys are numeric to keep multiple values in order should your Enum
need more than one.