What are Python metaclasses useful for?

Add extra flexibility to your programming:

But according to this Metaclass programming in Python you might not need them ( yet )

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).

— Python Guru Tim Peters

They are rarely needed, but come in useful in places where you want to add behavior to the basic behavior of an object — compare Aspect Oriented Programming, or the instrumentation done in persistence frameworks like Hibernate.

For example, you may want a class which persists or logs each new object.

If you are looking for examples of using the metaclass mechanism, you can read the source code of django.forms.

The declarative style of form definition is implemented through metaclass.

Probably there is nothing that can be done exclusively with metaclasses, but to some people (mine included) it’s a interesting tool you can use. Just be careful not to abuse, as it can be tricky.

For example, I’ve used metaprogramming in a recent project. It was a OpenOffice calc sheet, which, using some pyUNO macros, generates some files with information. There was one sheet that presents to the user the information to fill, and the others can be used to describe the kind of elements and their properties. The user can then select the number of elements and the type of each, and generate the files.
The macro will create a class via metaprogramming following the configuration on each sheet. Then, the user can instanciate each class and generate objects.

It could be done without metaprogramming, but to me seemed natural to use the metaprogramming capabilities to do it.

Take a look at Django sources – for example metaclasses are used there to generate models.

http://code.djangoproject.com/wiki/DynamicModels

Internally, Django uses metaclasses to
create models based on a class you
provide in your source code. Without
getting into too many details, that
means that rather than your classes
being the actual models, Django
receives a description of your class,
which it uses to create a model in its
place.

I use metaclasses with some frequency, and they’re an extremely powerful tool to have in the toolbox. Sometimes your solution to a problem can be more elegant, less code, with them than without.

The thing I find myself using metaclasses for most often, is post-processing the class attributes during class creation. For example, setting a name attribute on objects where appropriate (like how the Django ORM might work):

class AutonamingType(type):
    def __init__(cls, name, bases, attrs):
        for k,v in attrs.iteritems():
            if getattr(v, '__autoname__', False):
                v.name = k

class Autonamer(object):
    __metaclass__ = AutonamingType

If you have this as a tool, and you’re using a class which must know its name before it can do_something():

class Foo(object):
    __autoname__ = True
    def __init__(self, name=None):
        self.name = name
    def do_something(self):
        if self.name is None:
            raise ValueError('name is None')
        # now, do something

It can make the difference in the rest of your code between this:

class Bar(object):
    myfoo1 = Foo('myfoo1')
    myfoo2 = Foo('myfoo2')
    myfoo3 = Foo('myfoo3')

and this:

class Baaz(Autonamer):
    myfoo1 = Foo()
    myfoo2 = Foo()
    myfoo3 = Foo()

Thus reducing duplication (and the chances that the variable name and the assigned name might get out-of-sync).

The first commentator states that the use of metaclasses here was ‘a gemstone’ that helped him track down unexpected errors that had been occurring for (many) ‘days’.

Two things to keep in mind:

• You probably don’t need them. They sound cool, but most people don’t actually need them.

• Python supports dynamic classes via type(), multiple inheritance and method chaining via super().foo(). You can even add a function to a class and it will get bound to it at run time.

So, most of the time metaclasses are somewhat overkill.

However, one thing I found they can be very useful for is to assemble behavior from the attributes – not just the methods – of multiple classes.

For example, a validation framework that puts together multiple requirements:

• An http response should have

  • a status code – value to be specified in a subclass
  • content type – value to be specified in a subclass

• Furthermore, a html response checking class on which success is expected should:

  • expect 200 in that status code. html in the content type
  • expect a title – value to be specified… elsewhere

• The test instance would then only need specify the exact title.

A metaclass system can assemble all these requirements together so that your unittest only has to specify the title and the class "knows" it has to scrape the page for a <title>, check the content type and check the response code.

Here’s a much more modest attempt at an example of using metaclasses for this. We are printing letters which have various header templates that get assembled together. The footer could use the same mechanism, but instead uses mixins a la Python’s super() considered super! . These 2 approaches – assembling class-level attributes and chaining methods other than __init__ via super work well together.

def get_annos(v):
    return getattr(v, "__annotations__", {})

class MailMeta(type):

    def get_annos(cls: type, ancestors_in: list[type]):
        res = {}
        for acls in ancestors_in:
            res.update(**get_annos(acls))
        return res

    def _calc_annotations(cls, li_bases):
        """ concatenate the annotations for the class and ancestors """
        annos = cls.get_annos(li_bases + [cls])
        cls.annos = annos

    def _concat_template(cls, varname, bases : list[type]):
        """this glues together some templates
        assembling complex attributes from multiple classes is 
        something metaclasses are uniquely good at.
        how to do this can be tricky though!  "Unordered" attributes
        like sets or dictionaries can be easier but use `copy` 
        liberally.      
        """

        li = []
        cls_var = f"cls_{varname}"
        for acls in bases:
            v = getattr(acls, cls_var, "")
            if not v:
                continue
            if v in li:
                continue

            li.append(v)

        all_ = "".join(li)
        setattr(cls, varname, all_)

    def __init__(cls, name, bases, attrs, **kwargs):
        """assemble together some of the attributes from the mro"""

        li_bases2current = [
            v for v in list(reversed(cls.mro())) if not v is object and not v is cls
        ]

        cls._calc_annotations(li_bases2current)
        cls._concat_template("header", [cls] + list(reversed(li_bases2current)) )

        super().__init__(name, bases, attrs)
        

class Mail(metaclass=MailMeta):
    """ specify some base behavior"""
    name : str  # mail always need to state who it is for
    salutation : str 

    cls_header : str = ""
    template : str
    footer : str = ""

    def _set_annotations(self, **kwargs):
        """"""
        undefined = object()

        for attrname in self.annos.keys():
            v = kwargs.get(attrname,undefined)
            if v is not undefined:
                setattr(self, attrname, v)
            else:
                #maybe its set on the classes
                v = getattr(self, attrname,undefined)
                if v is undefined:
                    raise ValueError(f"{attrname} is undefined")

    def __init__(self, **kwargs):
        """ track which variables we were expecting"""
        self._set_annotations(**kwargs)
        self.fulltemplate = "n".join([self.header,self.template,self.get_footer()])

    def get_footer(self):
        return f"n{self.footer}"

    def print(self):
        print(f"n{'-' * 40}n{self.__class__.__name__}:n")

        di = dict(**vars(self.__class__))
        di.update(**vars(self))
        print(self.fulltemplate % (di))        


class Email:
    cls_header = "Dear %(salutation)s %(name)s,"
    cls_footer = "Sincerely,"

class Spam(Email, Mail):
    amount : int
    template = "You have won a prize of $%(amount)s"

class Signature:
    t_footer = "%(from_)s"
    from_ : str

    def get_footer(self):
        return super().get_footer() + self.t_footer % vars(self)

class TrustedSpam(Signature, Spam):
    "Trusted cuz signed"

    template = "You have won $%(amount)s.  This is a once in a lifetime opportunity."

    def get_footer(self):
        return super().get_footer() + "nPlease trust me"

class SnailMailTrustedSpamForDoctors(TrustedSpam):
    "Doctors like to be called Dr."

    salutation = "Dr."
    cls_header = "101 Honest Fellows Streetnn"


try:
    mail = Mail(saluation="Ms.", template="")
except (ValueError,) as exc: 
    print(f"ooops.  forgot `name`.              All good: {exc=} ✅")

try:
    mail = Spam(salutation="Ms.", name="Johnson")
except (Exception,) as exc: 
    print(f"ooops.  forgot `amount` on a Spam.  All good: {exc=} ✅")


mail = Spam(salutation="Ms.", amount=2000, name="Johnson")
mail.print()

mail = TrustedSpam(salutation="Ms.", amount=2000, name="Johnson", from_="Joe Trustworthy")
mail.print()

mail = SnailMailTrustedSpamForDoctors(amount=10000, name="Gullible", from_="Joe Trustworthy")
mail.print()

And the output:

ooops.  forgot `name`.              All good: exc=ValueError('name is undefined') ✅
ooops.  forgot `amount` on a Spam.  All good: exc=ValueError('amount is undefined') ✅

----------------------------------------
Spam:

Dear Ms. Johnson,
You have won a prize of $2000



----------------------------------------
TrustedSpam:

Dear Ms. Johnson,
You have won $2000.  This is a once in a lifetime opportunity.

Joe Trustworthy
Please trust me

----------------------------------------
SnailMailTrustedSpamForDoctors:

101 Honest Fellows Street

Dear Dr. Gullible,
You have won $10000.  This is a once in a lifetime opportunity.

Joe Trustworthy
Please trust me