Forward declaration of classes?

Question:

I have some classes looking like this:

class Base:
  subs = [Sub3,Sub1]
  # Note that this is NOT a list of all subclasses!
  # Order is also important

class Sub1(Base): pass
class Sub2(Base): pass
class Sub3(Base): pass
...

Now, this fails because Sub1 and Sub3 are not defined when Base.subs is. But obviously I can’t put the subclasses before Base either. Is there a way to forward-declare classes in Python? I want to work with isinstance so the types in subs actually have to be the same as the later declared subclasses, it’s not enough that they have the same name and other properties.

One workaround is to do: Base.subs = [Sub3,Sub1] after the subclasses have been defined, but I don’t like having to split my class in that way.

Edit: Added information about order

Asked By: pafcu

||

Answers:

Edit: Because of the added requirement of order I completely reworked my answer. I also make use of a class decorator, which was used here first by @Ignacio Vazquez-Abrams.

Edit2: code now tested and some stupidity slightly corrected


class Base(object):
    subs = []

    @classmethod
    def addsub(cls, before=None): 
        def inner(subclass):
            if before and before in cls.subs:
                cls.subs.insert(cls.subs.index(before), subclass)
            else:
                cls.subs.append(subclass)
            return subclass
        return inner

@Base.addsub()
class Sub1(Base):
    pass

class Sub2(Base):
    pass

@Base.addsub(before=Sub1)
class Sub3(Base):
    pass

Answered By: knitti

Write a decorator that adds it to the registry in Base.

class Base(object):
  subs = []

  @classmethod
  def addsub(cls, scls):
    cls.subs.append(scls)

 ...

@Base.addsub
class Sub1(Base):
  pass

class Sub2(Base):
  pass

@Base.addsub
class Sub3(Base):
  pass
class Foo:
  pass

class Bar:
  pass

Foo.m = Bar()
Bar.m = Foo()
Answered By: ceth

I would just define the subclasses as strings and have the inevitable decorator replace the strings with the classes that they name. I would also define the decorator on a metaclass because I think that that’s more in line with the objective: we’re modifying class behavior and just like you modify object behavior by modifying its class, you modify class behavior by modifying its metaclass.

class BaseMeta(type):

    def register(cls, subcls):
        try:
            i = cls.subs.index(subcls.__name__)
        except ValueError:
            pass
        else:
            cls.subs[i] = subcls
        finally:
            return cls


class Base(object):
    __metaclass__ = BaseMeta
    subs = ['Sub3', 'Sub1']

@Base.register
class Sub1(Base): pass

@Base.register
class Sub2(Base): pass

@Base.register
class Sub3(Base): pass

print Base.subs

This outputs:

[<class '__main__.Sub3'>, <class '__main__.Sub1'>]
Answered By: aaronasterling

Here’s essentially a hybrid version of @Ignacio Vazquez-Abrams’ and @aaronasterling’s answers which preserves the order of the subclasses in the list. Initially the desired subclass names (i.e. strings) are manually placed in the subs list in the desired order, then as each subclass is defined, a class decorator causes the corresponding string to be replaced with the actual subclass:

class Base(object):  # New-style class (i.e. explicitly derived from object).

    @classmethod
    def register_subclass(cls, subclass):
        """ Class decorator for registering subclasses. """

        # Replace any occurrences of the class name in the class' subs list.
        # with the class itself.
        # Assumes the classes in the list are all subclasses of this one.
        # Works because class decorators are called *after* the decorated class'
        # initial creation.
        while subclass.__name__ in cls.subs:
            cls.subs[cls.subs.index(subclass.__name__)] = subclass

        return cls  # Return modified class.

    subs = ['Sub3', 'Sub1']  # Ordered list of subclass names.


@Base.register_subclass
class Sub1(Base): pass

@Base.register_subclass
class Sub2(Base): pass

@Base.register_subclass
class Sub3(Base): pass

print('Base.subs: {}'.format(Base.subs))
# Base.subs: [<class '__main__.Sub3'>, <class '__main__.Sub1'>]

Update: Metaclasses

Exactly the same thing can also be done using a metaclass—which has the advantage that it eliminates the need to explicitly decorate each subclass as shown in my original answer above (which you accepted), however it makes it all happen automagically.

Note that even though the metaclass’ __init__() is called for the creation of every subclass, it only updates the subs list if the subclass’ name appears in it—so the initial Base class’ definition of the contents of the subs list still controls what gets replaced in it (maintaining its order).

class BaseMeta(type):

    def __init__(cls, name, bases, classdict):
        if classdict.get('__metaclass__') is not BaseMeta:  # Metaclass instance?
            # Replace any occurrences of a subclass' name in the class being
            # created the class' sub list with the subclass itself.
            # Names of classes which aren't direct subclasses will be ignored.
            while name in cls.subs:
                cls.subs[cls.subs.index(name)] = cls

        # Chain to __init__() of the class instance being created after changes.
        # Note class instance being defined must be new-style class.
        super(BaseMeta, cls).__init__(name, bases, classdict)


# Python 2 metaclass syntax.
class Base(object):  # New-style class (derived from built-in object class).
    __metaclass__ = BaseMeta
    subs = ['Sub3', 'Sub1']  # Ordered list of subclass names.

# Python 3 metaclass syntax.
#class Base(metaclass=BaseMeta):
#    subs = ['Sub3', 'Sub1']  # Ordered list of subclass names.


# Note: No need to manually register the (direct) subclasses.
class Sub1(Base): pass
class Sub2(Base): pass
class Sub3(Base): pass

print('Base.subs: {}'.format(Base.subs))

Output:

Base.subs: [<class '__main__.Sub3'>, <class '__main__.Sub1'>]

It important to note that there’s at least one subtle difference between these two answers—namely that the first will work with any class name that is registered via @Base.register_subclass(), whether or not its actually a subclass of Base (although that might be possible to change/fix.)

I’m pointing this out for a couple of reasons: First because in your comments you said that subs was a "bunch of classes in a list, some of which might be its subclasses", and more importantly, because that’s not the case with the code in my update, which only works for Base subclasses since they effectively get "registered" automatically via the metaclass—but will leave anything else in the list alone. This could be considered a bug or a feature. ;¬)

Update: Python 3.6+

In Python 3.6 a new object method was added named __init_subclass__() which provides an even simpler way implement things that also doesn’t require decorating all the subclasses or defining a metaclass:

#!/usr/bin/env python3.6

class Base:
    subs = ['Sub3', 'Sub1']  # Ordered list of subclass names.

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        while cls.__name__ in cls.subs:
            cls.subs[cls.subs.index(cls.__name__)] = cls


# Note: No need to manually register the subclasses.
class Sub1(Base): pass
class Sub2(Base): pass
class Sub3(Base): pass

print('Base.subs: {}'.format(Base.subs))
Answered By: martineau

I’m pretty sure this should work for you. Just assign the depended class attribute afterwards.
This is also a lot less complicated.

class Base:pass

class Sub1(Base): pass
class Sub2(Base): pass
class Sub3(Base): pass

Base.subs = [Sub3,Sub1]
print(Sub1.subs)
#[<class __main__.Sub3 at 0x0282B2D0>, <class __main__.Sub1 at 0x01C79810>]
Answered By: user1761366

There is no way to directly declare forward-references in Python, but there are several workarounds, a few of which are reasonable:

1) Add the subclasses manually after they are defined.

    - Pros: easy to do; Base.subs is updated in one place
    - Cons: easy to forget (plus you don't want to do it this way)

Example:

class Base(object):
    pass

class Sub1(Base):
    pass

class Sub2(Base):
    pass

class Sub3(Base):
    pass

Base.subs = [sub3, sub1]

2) Create Base.subs with str values, and use a class decorator to substitute the actual subclasses (this can be a class method on Base or a function — I’m showing the function version, although I would probably use the method version).

- Pros: easy to do
- Cons: somewhat easy to forget; 

Example:

def register_with_Base(cls):
    name = cls.__name__
    index = Base.subs.index(name)
    Base.subs[index] = cls
    return cls

class Base(object):
    subs = ['Sub3', 'Sub1']

@register_with_Base
class Sub1(Base):
    pass

class Sub2(Base):
    pass

@register_with_Base
class Sub3(Base):
    pass

3) Create Base.subs with str values, and have the method that uses Base.subs do the substitution.

- Pros: no extra work in decorators, no forgetting to update `subs` later
- Cons: small amount of extra work when accessing `subs`

Example:

class Base(object):
    subs = ['Sub3', 'Sub1']
    def select_sub(self, criteria):
        for sub in self.subs:
            sub = globals()[sub]
            if #sub matches criteria#:
                break
        else:
            # use a default, raise an exception, whatever
        # use sub, which is the class defined below

class Sub1(Base):
    pass

class Sub2(Base):
    pass

class Sub3(Base):
    pass

I would use option 3 myself, as it keeps the functionality and the data all in one place. The only thing you have to do is keep subs up to date (and write the appropriate subclasses, of course).

Answered By: Ethan Furman

There is a workaround: put referenced dummy Sub1, Sub3 classes at the top, this works as “forward declaration”. They will be replaced by the actual implementations with the same names when executed.

forward-declaration.py:

class Sub1(): 
    print("Sub1 dummy class called")
    pass
class Sub3():
    print("Sub3 dummy class called")
    pass

class Base:
  subs = [Sub3, Sub1]
  print("Base class called")

class Sub1(Base): 
    print("Sub1 class called")
    def __init__(self):
        print("Sub1:__init__ called")
    pass
class Sub2(Base):
    def __init__(self):
        print("Sub2:__init__ called")
    pass
class Sub3(Base):
    def __init__(self):
        print("Sub3:__init__ called")
    pass

sub_1 = Sub1()
sub_2 = Sub2()
sub_3 = Sub3()

print(Base.subs)

python forward-declaration.py

Sub1 dummy class called
Sub3 dummy class called
Base class called
Sub1 class called
Sub1:__init__ called
Sub2:__init__ called
Sub3:__init__ called
[<class '__main__.Sub3'>, <class '__main__.Sub1'>]

Note: The above method fails on mypy or pylint static check but works properly

Answered By: Jonathan L

Not a direct answer to your question about forward declarations, but a direct solution for your problem of knowing all sub-classes in the base class itself: just use the (probably not very well known) __subclasses__ function, which should be defined on all classes in Python3. It directly gives you exactly what you want:

In [1]: class Base: pass

In [2]: class Sub1(Base): pass

In [3]: class Sub2(Base): pass

In [4]: class Sub3(Base): pass
    
In [5]: Base.__subclasses__()
Out[5]: [__main__.Sub1, __main__.Sub2, __main__.Sub3]

The documentation doesn’t specify if the order in which the sub-classes are defined is maintained, but it does appear to do so.

See also this answer to a question that asks about your problem.

Answered By: Bas Swinckels

If local inner classes inside an outer class can fullfill the given demands, a simple solution can help (so the inner classes are declared "in forward"; at least before they are instantiated inside the outer class):

class Base:
    class Sub1:
        # ...
    class Sub2:
        # ...
    class Sub3:
        # ...

    __Object1 = Sub1()
    __Object2 = Sub2()
    __Object3 = Sub3()
    # ...
Answered By: Sven650123
Categories: questions Tags:
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.