Class inheritance in Python 3.7 dataclasses

You’re seeing this error because an argument without a default value is being added after an argument with a default value. The insertion order of inherited fields into the dataclass is the reverse of Method Resolution Order, which means that the Parent fields come first, even if they are over written later by their children.

An example from PEP-557 – Data Classes:

@dataclass
class Base:
    x: Any = 15.0
    y: int = 0

@dataclass
class C(Base):
    z: int = 10
    x: int = 15

The final list of fields is, in order,x, y, z. The final type of x is int, as specified in class C.

Unfortunately, I don’t think there’s any way around this. My understanding is that if the parent class has a default argument, then no child class can have non-default arguments.

based on Martijn Pieters solution I did the following:

1) Create a mixing implementing the post_init

from dataclasses import dataclass

no_default = object()


@dataclass
class NoDefaultAttributesPostInitMixin:

    def __post_init__(self):
        for key, value in self.__dict__.items():
            if value is no_default:
                raise TypeError(
                    f"__init__ missing 1 required argument: '{key}'"
                )

2) Then in the classes with the inheritance problem:

from src.utils import no_default, NoDefaultAttributesChild

@dataclass
class MyDataclass(DataclassWithDefaults, NoDefaultAttributesPostInitMixin):
    attr1: str = no_default

EDIT:

After a time I also find problems with this solution with mypy, the following code fix the issue.

from dataclasses import dataclass
from typing import TypeVar, Generic, Union

T = TypeVar("T")


class NoDefault(Generic[T]):
    ...


NoDefaultVar = Union[NoDefault[T], T]
no_default: NoDefault = NoDefault()


@dataclass
class NoDefaultAttributesPostInitMixin:
    def __post_init__(self):
        for key, value in self.__dict__.items():
            if value is NoDefault:
                raise TypeError(f"__init__ missing 1 required argument: '{key}'")


@dataclass
class Parent(NoDefaultAttributesPostInitMixin):
    a: str = ""

@dataclass
class Child(Foo):
    b: NoDefaultVar[str] = no_default

The approach below deals with this problem while using pure python dataclasses and without much boilerplate code.

The ugly: dataclasses.InitVar[bool] serves as a pseudo-field just to help us do initialization and will be lost once the instance is created. While _ugly: bool = field(init=False) is an instance member which will not be initialized by __init__ method but can be alternatively initialized using __post_init__ method (you can find more here.).

from dataclasses import dataclass, field, InitVar

@dataclass
class Parent:
    name: str
    age: int
    ugly: InitVar[bool]
    _ugly: bool = field(init=False)

    def __post_init__(self, ugly: bool):
        self._ugly = ugly

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f'The Name is {self.name} and {self.name} is {self.age} year old')

@dataclass
class Child(Parent):
    school: str

jack = Parent('jack snr', 32, ugly=True)
jack_son = Child('jack jnr', 12, school='havard', ugly=True)

jack.print_id()
jack_son.print_id()

Note that this makes field ugly mandatory to make it optional you can define a class method on the Parent that includes ugly as an optional parameter:

from dataclasses import dataclass, field, InitVar

@dataclass
class Parent:
    name: str
    age: int
    ugly: InitVar[bool]
    _ugly: bool = field(init=False)

    def __post_init__(self, ugly: bool):
        self._ugly = ugly
    
    @classmethod
    def create(cls, ugly=True, **kwargs):
        return cls(ugly=ugly, **kwargs)

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f'The Name is {self.name} and {self.name} is {self.age} year old')

@dataclass
class Child(Parent):
    school: str

jack = Parent.create(name='jack snr', age=32, ugly=False)
jack_son = Child.create(name='jack jnr', age=12, school='harvard')

jack.print_id()
jack_son.print_id()

Now you can use the create(...) class method as a factory method for creating Parent/Child classes with a default value for ugly. Note you must use named parameters for this approach to work.

A possible work-around is to use monkey-patching to append the parent fields

import dataclasses as dc

def add_args(parent): 
    def decorator(orig):
        "Append parent's fields AFTER orig's fields"

        # Aggregate fields
        ff  = [(f.name, f.type, f) for f in dc.fields(dc.dataclass(orig))]
        ff += [(f.name, f.type, f) for f in dc.fields(dc.dataclass(parent))]

        new = dc.make_dataclass(orig.__name__, ff)
        new.__doc__ = orig.__doc__

        return new
    return decorator

class Animal:
    age: int = 0 

@add_args(Animal)
class Dog:
    name: str
    noise: str = "Woof!"

@add_args(Animal)
class Bird:
    name: str
    can_fly: bool = True

Dog("Dusty", 2)               # --> Dog(name='Dusty', noise=2, age=0)
b = Bird("Donald", False, 40) # --> Bird(name='Donald', can_fly=False, age=40)

It’s also possible to prepend non-default fields,
by checking if f.default is dc.MISSING,
but this is probably too dirty.

While monkey-patching lacks some features of inheritance,
it can still be used to add methods to all pseudo-child classes.

For more fine-grained control, set the default values
using dc.field(compare=False, repr=True, ...)

You can use attributes with defaults in parent classes if you exclude them from the init function. If you need the possibility to override the default at init, extend the code with the answer of Praveen Kulkarni.

from dataclasses import dataclass, field

@dataclass
class Parent:
    name: str
    age: int
    ugly: bool = field(default=False, init=False)

@dataclass
class Child(Parent):
    school: str

jack = Parent('jack snr', 32)
jack_son = Child('jack jnr', 12, school = 'havard')
jack_son.ugly = True

Or even

@dataclass
class Child(Parent):
    school: str
    ugly = True
    # This does not work
    # ugly: bool = True

jack_son = Child('jack jnr', 12, school = 'havard')
assert jack_son.ugly

You can use a modified version of dataclasses, which will generate a keyword only __init__ method:

import dataclasses


def _init_fn(fields, frozen, has_post_init, self_name):
    # fields contains both real fields and InitVar pseudo-fields.
    globals = {'MISSING': dataclasses.MISSING,
               '_HAS_DEFAULT_FACTORY': dataclasses._HAS_DEFAULT_FACTORY}

    body_lines = []
    for f in fields:
        line = dataclasses._field_init(f, frozen, globals, self_name)
        # line is None means that this field doesn't require
        # initialization (it's a pseudo-field).  Just skip it.
        if line:
            body_lines.append(line)

    # Does this class have a post-init function?
    if has_post_init:
        params_str = ','.join(f.name for f in fields
                              if f._field_type is dataclasses._FIELD_INITVAR)
        body_lines.append(f'{self_name}.{dataclasses._POST_INIT_NAME}({params_str})')

    # If no body lines, use 'pass'.
    if not body_lines:
        body_lines = ['pass']

    locals = {f'_type_{f.name}': f.type for f in fields}
    return dataclasses._create_fn('__init__',
                      [self_name, '*'] + [dataclasses._init_param(f) for f in fields if f.init],
                      body_lines,
                      locals=locals,
                      globals=globals,
                      return_type=None)


def add_init(cls, frozen):
    fields = getattr(cls, dataclasses._FIELDS)

    # Does this class have a post-init function?
    has_post_init = hasattr(cls, dataclasses._POST_INIT_NAME)

    # Include InitVars and regular fields (so, not ClassVars).
    flds = [f for f in fields.values()
            if f._field_type in (dataclasses._FIELD, dataclasses._FIELD_INITVAR)]
    dataclasses._set_new_attribute(cls, '__init__',
                       _init_fn(flds,
                                frozen,
                                has_post_init,
                                # The name to use for the "self"
                                # param in __init__.  Use "self"
                                # if possible.
                                '__dataclass_self__' if 'self' in fields
                                else 'self',
                                ))

    return cls


# a dataclass with a constructor that only takes keyword arguments
def dataclass_keyword_only(_cls=None, *, repr=True, eq=True, order=False,
              unsafe_hash=False, frozen=False):
    def wrap(cls):
        cls = dataclasses.dataclass(
            cls, init=False, repr=repr, eq=eq, order=order, unsafe_hash=unsafe_hash, frozen=frozen)
        return add_init(cls, frozen)

    # See if we're being called as @dataclass or @dataclass().
    if _cls is None:
        # We're called with parens.
        return wrap

    # We're called as @dataclass without parens.
    return wrap(_cls)

(also posted as a gist, tested with Python 3.6 backport)

This will require to define the child class as

@dataclass_keyword_only
class Child(Parent):
    school: str
    ugly: bool = True

And would generate __init__(self, *, name:str, age:int, ugly_bool=True, school:str) (which is valid python). The only caveat here is not allowing to initialize objects with positional arguments, but otherwise it’s a completely regular dataclass with no ugly hacks.

I came back to this question after discovering that dataclasses may be getting a decorator parameter that allows fields to be reordered. This is certainly a promising development, though progress on this feature seems to have stalled somewhat.

Right now, you can get this behaviour, plus some other niceties, by using dataclassy, my reimplementation of dataclasses that overcomes frustrations like this. Using from dataclassy in place of from dataclasses in the original example means it runs without errors.

Using inspect to print the signature of Child makes what is going on clear; the result is (name: str, age: int, school: str, ugly: bool = True). Fields are always reordered so that fields with default values come after fields without them in the parameters to the initializer. Both lists (fields without defaults, and those with them) are still ordered in definition order.

Coming face to face with this issue was one of the factors that prompted me to write a replacement for dataclasses. The workarounds detailed here, while helpful, require code to be contorted to such an extent that they completely negate the readability advantage dataclasses’ naive approach (whereby field ordering is trivially predictable) offers.

A quick and dirty solution:

from typing import Optional

@dataclass
class Child(Parent):
    school: Optional[str] = None
    ugly: bool = True

    def __post_init__(self):
        assert self.school is not None

Then go back and refactor once (hopefully) the language is extended.

Complementing the Martijn Pieters solution that uses attrs: it is possible to create the inheritance without the default attributes replication, with:

import attr

@attr.s(auto_attribs=True)
class Parent:
    name: str
    age: int
    ugly: bool = attr.ib(default=False, kw_only=True)


@attr.s(auto_attribs=True)
class Child(Parent):
    school: str
    ugly: bool = True

More about the kw_only parameter can be found here

How about defining the ugly field like this, instead of the default way?

ugly: bool = field(metadata=dict(required=False, missing=False))

When you use Python inheritance to create dataclasses, you cannot guarantee that all fields with default values will appear after all fields without default values.

An easy solution is to avoid using multiple inheritance to construct a "merged" dataclass. Instead, we can build a merged dataclass just by filtering and sorting on the fields of your parent dataclasses.

Try out this merge_dataclasses() function:

import dataclasses
import functools
from typing import Iterable, Type


def merge_dataclasses(
    cls_name: str,
    *,
    merge_from: Iterable[Type],
    **kwargs,
):
    """
    Construct a dataclass by merging the fields
    from an arbitrary number of dataclasses.

    Args:
        cls_name: The name of the constructed dataclass.

        merge_from: An iterable of dataclasses
            whose fields should be merged.

        **kwargs: Keyword arguments are passed to
            :py:func:`dataclasses.make_dataclass`.

    Returns:
        Returns a new dataclass
    """
    # Merge the fields from the dataclasses,
    # with field names from later dataclasses overwriting
    # any conflicting predecessor field names.
    each_base_fields = [d.__dataclass_fields__ for d in merge_from]
    merged_fields = functools.reduce(
        lambda x, y: {**x, **y}, each_base_fields
    )

    # We have to reorder all of the fields from all of the dataclasses
    # so that *all* of the fields without defaults appear
    # in the merged dataclass *before* all of the fields with defaults.
    fields_without_defaults = [
        (f.name, f.type, f)
        for f in merged_fields.values()
        if isinstance(f.default, dataclasses._MISSING_TYPE)
    ]
    fields_with_defaults = [
        (f.name, f.type, f)
        for f in merged_fields.values()
        if not isinstance(f.default, dataclasses._MISSING_TYPE)
    ]
    fields = [*fields_without_defaults, *fields_with_defaults]

    return dataclasses.make_dataclass(
        cls_name=cls_name,
        fields=fields,
        **kwargs,
    )

And then you can merge dataclasses as follows. Note that we can merge A and B and the default fields b and d are moved to the end of the merged dataclass.

@dataclasses.dataclass
class A:
    a: int
    b: int = 0


@dataclasses.dataclass
class B:
    c: int
    d: int = 0


C = merge_dataclasses(
    "C",
    merge_from=[A, B],
)

# Note that 
print(C(a=1, d=1).__dict__)
# {'a': 1, 'd': 1, 'b': 0, 'c': 0}

Of course, the pitfall of this solution is that C doesn’t actually inherit from A and B, which means that you cannot use isinstance() or other type assertions to verify C’s parentage.

Note that with Python 3.10, it is now possible to do it natively with dataclasses.

Dataclasses 3.10 added the kw_only attribute (similar to attrs).
It allows you to specify which fields are keyword_only, thus will be set at the end of the init, not causing an inheritance problem.

Taking directly from Eric Smith’s blog post on the subject:

There are two reasons people [were asking for] this feature:

• When a dataclass has many fields, specifying them by position can become unreadable. It also requires that for backward compatibility, all new fields are added to the end of the dataclass. This isn’t always desirable.
• When a dataclass inherits from another dataclass, and the base class has fields with default values, then all of the fields in the derived class must also have defaults.

What follows is the simplest way to do it with this new argument, but there are multiple ways you can use it to use inheritance with default values in the parent class:

from dataclasses import dataclass

@dataclass(kw_only=True)
class Parent:
    name: str
    age: int
    ugly: bool = False

@dataclass(kw_only=True)
class Child(Parent):
    school: str

ch = Child(name="Kevin", age=17, school="42")
print(ch.ugly)

Take a look at the blogpost linked above for a more thorough explanation of kw_only.

Cheers !

PS: As it is fairly new, note that your IDE might still raise a possible error, but it works at runtime

An experimental but interesting solution would be to use metaclasses. The solution below enables the usage of Python dataclasses with simple inheritance without using the dataclass decorator at all. Moreover, it makes it possible to inherit the fields of the parent base classes without complaining about the order of positional arguments(non-default fields).

from collections import OrderedDict
import typing as ty
import dataclasses
from itertools import takewhile

class DataClassTerm:
    def __new__(cls, *args, **kwargs):
        return super().__new__(cls)

class DataClassMeta(type):
    def __new__(cls, clsname, bases, clsdict):
        fields = {}

        # Get list of base classes including the class to be produced(initialized without its original base classes as those have already become dataclasses)
        bases_and_self = [dataclasses.dataclass(super().__new__(cls, clsname, (DataClassTerm,), clsdict))] + list(bases)

        # Whatever is a subclass of DataClassTerm will become a DataClassTerm. 
        # Following block will iterate and create individual dataclasses and collect their fields
        for base in bases_and_self[::-1]: # Ensure that last fields in last base is prioritized
            if issubclass(base, DataClassTerm):
                to_dc_bases = list(takewhile(lambda c: c is not DataClassTerm, base.__mro__))
                for dc_base in to_dc_bases[::-1]: # Ensure that last fields in last base in MRO is prioritized(same as in dataclasses)
                    if dataclasses.is_dataclass(dc_base):
                        valid_dc = dc_base
                    else:
                        valid_dc = dataclasses.dataclass(dc_base)
                    for field in dataclasses.fields(valid_dc):
                        fields[field.name] = (field.name, field.type, field)
        
        # Following block will reorder the fields so that fields without default values are first in order
        reordered_fields = OrderedDict()
        for n, t, f  in fields.values():
            if f.default is dataclasses.MISSING and f.default_factory is dataclasses.MISSING:
                reordered_fields[n] = (n, t, f)
        for n, t, f  in fields.values():
            if n not in reordered_fields.keys():
                reordered_fields[n] = (n, t, f)
        
        # Create a new dataclass using `dataclasses.make_dataclass`, which ultimately calls type.__new__, which is the same as super().__new__ in our case
        fields = list(reordered_fields.values())
        full_dc = dataclasses.make_dataclass(cls_name=clsname, fields=fields, init=True, bases=(DataClassTerm,))
        
        # Discard the created dataclass class and create new one using super but preserve the dataclass specific namespace.
        return super().__new__(cls, clsname, bases, {**full_dc.__dict__,**clsdict})
    
class DataClassCustom(DataClassTerm, metaclass=DataClassMeta):
    def __new__(cls, *args, **kwargs):
        if len(args)>0:
            raise RuntimeError("Do not use positional arguments for initialization.")
        return super().__new__(cls, *args, **kwargs)

Now let’s create a sample dataclass with a parent dataclass and a sample mixing class:

class DataClassCustomA(DataClassCustom):
    field_A_1: int = dataclasses.field()
    field_A_2: ty.AnyStr = dataclasses.field(default=None)

class SomeOtherClass:
    def methodA(self):
        print('print from SomeOtherClass().methodA')

class DataClassCustomB(DataClassCustomA,SomeOtherClass):
    field_B_1: int = dataclasses.field()
    field_B_2: ty.Dict = dataclasses.field(default_factory=dict)

The result is

result_b = DataClassCustomB(field_A_1=1, field_B_1=2)

result_b
# DataClassCustomB(field_A_1=1, field_B_1=2, field_A_2=None, field_B_2={})

result_b.methodA()
# print from SomeOtherClass().methodA

An attempt to do the same with @dataclass decorator on each parent class would have raised an exception in the following child class, like TypeError(f'non-default argument <field-name) follows default argument'). The above solution prevents this from happening because the fields are first reordered. However, since the order of fields is modified the prevention of *args usage in DataClassCustom.__new__ is mandatory as the original order is no longer valid.

Although in Python >=3.10 the kw_only feature was introduced that essentially makes inheritance in dataclasses much more reliable, the above example still can be used as a way to make dataclasses inheritable that do not require the usage of @dataclass decorator.

If you are using Python 3.10+, then you can utilize keyword-only arguments for the dataclass as discussed in this answer and in the python docs.

If you’re using < Python 3.10, then you can utilize dataclasses.field with a default_factory that throws. Since the attribute will be declared with field(), it gets treated as if it has a default; but if a user attempts to create an instance without providing the value for that field, it will use the factory, which will error.

This technique isn’t equivalent to keyword only, because you could still provide all the arguments positionally. However, this does solve the problem, and is simpler than messing around with various dataclass dunder methods.

from dataclasses import dataclass, field
from datetime import datetime
from typing import Optional, TypeVar

T = TypeVar("T")


def required() -> T:
    f: T

    def factory() -> T:
        # mypy treats a Field as a T, even though it has attributes like .name, .default, etc
        field_name = f.name  # type: ignore[attr-defined]
        raise ValueError(f"field '{field_name}' required")

    f = field(default_factory=factory)
    return f


@dataclass
class Event:
    id: str
    created_at: datetime
    updated_at: Optional[datetime] = None


@dataclass
class NamedEvent(Event):
    name: str = required()


event = NamedEvent(name="Some Event", id="ab13c1a", created_at=datetime.now())
print("created event:", event)


event2 = NamedEvent("ab13c1a", datetime.now(), name="Some Other Event")
print("created event:", event2)

event3 = NamedEvent("ab13c1a", datetime.now())

Output:

created event: NamedEvent(id='ab13c1a', created_at=datetime.datetime(2022, 7, 23, 19, 22, 17, 944550), updated_at=None, name='Some Event')
created event: NamedEvent(id='ab13c1a', created_at=datetime.datetime(2022, 7, 23, 19, 22, 17, 944588), updated_at=None, name='Some Other Event')
Traceback (most recent call last):
  File ".../gist.py", line 39, in <module>
    event3 = NamedEvent("ab13c1a", datetime.now())
  File "<string>", line 6, in __init__
  File ".../gist.py", line 14, in factory
    raise ValueError(f"field '{field_name}' required")
ValueError: field 'name' required

You can also find this code on this github gist.