Does python preserve order of annotations with other methods?

Question

Consider the following class:

@dataclass
class Point:
   id: int

   x: int
   y: int
   @property
   def distance_from_zero(self): return (self.x**2 + self.y**2)**0.5

   color: tuple #RGB or something...

I know that I can get the annotations from the __annotations__ variable, or the fields function from dataclasses.fields and in order.
I also know that the normal methods of any object can be read with dir or using the __dict__ method.

But what I’m after is something that can give me both in the right order, in the above case, it’ll be something like:

>>>get_all_fields(Point)
['id', 'x', 'y', 'distance_from_zero', 'color']

The only thing i can think of is using something like the inspect module to read the actual code and somehow find the order. But that sounds really nasty.

Asked By: aliqandil

||

Source

Answer 1

Well, this is the best workaround I’ve found so far, the idea is that when a class is being created, the __annotations__ object will start to fill one by one, so an option is to keep track of properties during the creation of the class. It’s not perfect as it forces you to use an alternative decorator instead of property, and it also can’t do the same with function methods (but I don’t care about that right now). In my implementation, you also have to decorate the whole class as to attach a classmethod that actually outputs the orders.

import inspect
def ordered_property( f ):
    if isinstance(f, type):
        @classmethod
        def list_columns( cls ):
            if not list_columns.initiated:
                for annotation in cls.__annotations__:
                    if annotation not in cls.__columns__:
                        cls.__columns__.append(annotation)
                list_columns.initiated = True
            return cls.__columns__

        list_columns.initiated = False
        f.list_columns = list_columns
        return f
    else:
        #Two stacks from the start, is the class object that's being constructed.
        class_locals = inspect.stack()[1].frame.f_locals
        class_locals.setdefault('__columns__', [])

        for annotation in class_locals['__annotations__']:
            if annotation not in class_locals['__columns__']:
                class_locals['__columns__'].append(annotation)

        class_locals['__columns__'].append(f.__name__)
        return property(f)

And the example from the question have to be changed to this:

@dataclass
@ordered_property
class Point:
   id: int

   x: int
   y: int
   @ordered_property
   def distance_from_zero(self): return (self.x**2 + self.y**2)**0.5

   color: tuple #RGB or something...

And finally the output will be something like this:

>>>Point.list_columns()
['id', 'x', 'y', 'distance_from_zero', 'color']

(I won’t mark this as the answer, as it has a bit of a footprint, and doesn’t consider callable methods from the class)

Answered By: aliqandil

Answer 2

It is possible to build a clean solution with introspection, since python is nice enough to make the module in charge of parsing python code (ast) available to us.

The api takes some getting used to, but here is a fix for your problem with inspect.getsource and a little custom ast-node walker:

import ast
import inspect


class AttributeVisitor(ast.NodeVisitor):
    def visit_ClassDef(self, node):
        self.attributes = []
        for statement in node.body:
            if isinstance(statement, ast.AnnAssign):
                self.attributes.append(statement.target.id)
            elif isinstance(statement, ast.FunctionDef):
                # only consider properties
                if statement.decorator_list:
                    if "property" in [d.id for d in statement.decorator_list]:
                        self.attributes.append(statement.name)
            else:
                print(f"Skipping {statement=}")

# parse the source code of "Point", so we don't have to write a parser ourselves
tree = ast.parse(inspect.getsource(Point), '<string>')

# create a visitor and run it over the tree line by line
visitor = AttributeVisitor()
visitor.visit(tree)

# print result, should be ['id', 'x', 'y', 'distance_from_zero', 'color']
print(visitor.attributes)

Using this solution means you don’t have to alter your Point class in any way to get what you want.

Answered By: Arne

Answer 3

There is also a solution involving metaclasses and the __prepare__ hook. The trick is to use it to provide a custom dictionary that will be used throughout the class creation, keeping track of new additions in a fields entry, and which propagates this accumulation of fields also when __annotations__ is added to it.

class _FieldAccumulatorDict(dict):

    def __init__(self, fields=None):
        if fields is None:
            # class dict, add a new `fields` list to the dictionary
            fields = []
            super().__setitem__("fields", fields)
        self.__fields = fields

    def __setitem__(self, key, value):
        if not key.startswith("__"):
            self.__fields.append(key)
        elif key == "__annotations__":
            # propagate accumulation when the `__annotation__` field is set internally
            value = _FieldAccumulatorDict(self.__fields)
        super().__setitem__(key, value)


class FieldAccumulatorMetaclass(type):

    def __prepare__(metacls, *args, **kwargs):
        return _FieldAccumulatorDict()


@dataclass
class Point(metaclass=FieldAccumulatorMetaclass):
    id: int

    x: int
    y: int

    @property
    def distance_from_zero(self): return (self.x ** 2 + self.y ** 2) ** 0.5

    color: tuple  # RGB or something...


print(Point.fields)  # will print ['id', 'x', 'y', 'distance_from_zero', 'color']

Consider also that metaclasses are inherited, so the metaclass can be assigned just to the root class and will then be used by all classes in the hierarchy.

Answered By: Paolo Tranquilli

Does python preserve order of annotations with other methods?

Question:

Answers: