我目前正在尝试Python 3.7中引入的新数据类结构。我目前被困在试图做一些继承的父类。看起来参数的顺序被我当前的方法搞砸了,比如子类中的bool形参在其他形参之前传递。这将导致一个类型错误。

from dataclasses import dataclass

@dataclass
class Parent:
    name: str
    age: int
    ugly: bool = False

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f'The Name is {self.name} and {self.name} is {self.age} year old')

@dataclass
class Child(Parent):
    school: str
    ugly: bool = True


jack = Parent('jack snr', 32, ugly=True)
jack_son = Child('jack jnr', 12, school = 'havard', ugly=True)

jack.print_id()
jack_son.print_id()

当我运行这段代码时,我得到这个TypeError:

TypeError: non-default argument 'school' follows default argument

我怎么解决这个问题?


当前回答

一个实验性但有趣的解决方案是使用元类。下面的解决方案允许使用带有简单继承的Python数据类,而完全不使用数据类装饰器。此外,它可以继承父基类的字段,而不必抱怨位置参数的顺序(非默认字段)。

from collections import OrderedDict
import typing as ty
import dataclasses
from itertools import takewhile

class DataClassTerm:
    def __new__(cls, *args, **kwargs):
        return super().__new__(cls)

class DataClassMeta(type):
    def __new__(cls, clsname, bases, clsdict):
        fields = {}

        # Get list of base classes including the class to be produced(initialized without its original base classes as those have already become dataclasses)
        bases_and_self = [dataclasses.dataclass(super().__new__(cls, clsname, (DataClassTerm,), clsdict))] + list(bases)

        # Whatever is a subclass of DataClassTerm will become a DataClassTerm. 
        # Following block will iterate and create individual dataclasses and collect their fields
        for base in bases_and_self[::-1]: # Ensure that last fields in last base is prioritized
            if issubclass(base, DataClassTerm):
                to_dc_bases = list(takewhile(lambda c: c is not DataClassTerm, base.__mro__))
                for dc_base in to_dc_bases[::-1]: # Ensure that last fields in last base in MRO is prioritized(same as in dataclasses)
                    if dataclasses.is_dataclass(dc_base):
                        valid_dc = dc_base
                    else:
                        valid_dc = dataclasses.dataclass(dc_base)
                    for field in dataclasses.fields(valid_dc):
                        fields[field.name] = (field.name, field.type, field)
        
        # Following block will reorder the fields so that fields without default values are first in order
        reordered_fields = OrderedDict()
        for n, t, f  in fields.values():
            if f.default is dataclasses.MISSING and f.default_factory is dataclasses.MISSING:
                reordered_fields[n] = (n, t, f)
        for n, t, f  in fields.values():
            if n not in reordered_fields.keys():
                reordered_fields[n] = (n, t, f)
        
        # Create a new dataclass using `dataclasses.make_dataclass`, which ultimately calls type.__new__, which is the same as super().__new__ in our case
        fields = list(reordered_fields.values())
        full_dc = dataclasses.make_dataclass(cls_name=clsname, fields=fields, init=True, bases=(DataClassTerm,))
        
        # Discard the created dataclass class and create new one using super but preserve the dataclass specific namespace.
        return super().__new__(cls, clsname, bases, {**full_dc.__dict__,**clsdict})
    
class DataClassCustom(DataClassTerm, metaclass=DataClassMeta):
    def __new__(cls, *args, **kwargs):
        if len(args)>0:
            raise RuntimeError("Do not use positional arguments for initialization.")
        return super().__new__(cls, *args, **kwargs)

现在让我们创建一个带有父数据类和混合类的样本数据类:

class DataClassCustomA(DataClassCustom):
    field_A_1: int = dataclasses.field()
    field_A_2: ty.AnyStr = dataclasses.field(default=None)

class SomeOtherClass:
    def methodA(self):
        print('print from SomeOtherClass().methodA')

class DataClassCustomB(DataClassCustomA,SomeOtherClass):
    field_B_1: int = dataclasses.field()
    field_B_2: ty.Dict = dataclasses.field(default_factory=dict)

结果是

result_b = DataClassCustomB(field_A_1=1, field_B_1=2)

result_b
# DataClassCustomB(field_A_1=1, field_B_1=2, field_A_2=None, field_B_2={})

result_b.methodA()
# print from SomeOtherClass().methodA

尝试在每个父类上使用@dataclass装饰器做同样的事情会在接下来的子类中引发一个异常,如TypeError(f'non-default argument <field-name)跟随默认参数')。上面的解决方案防止了这种情况的发生,因为字段首先被重新排序。然而,由于字段的顺序被修改了,在DataClassCustom中防止*args的使用。__new__是强制的,因为原来的顺序不再有效。

虽然在Python >=3.10中引入了kw_only特性,本质上使数据类中的继承更加可靠,但上面的示例仍然可以用作一种使数据类可继承的方法,而不需要使用@dataclass装饰器。

其他回答

一种可行的解决方法是使用monkey-patch来附加父字段

import dataclasses as dc

def add_args(parent): 
    def decorator(orig):
        "Append parent's fields AFTER orig's fields"

        # Aggregate fields
        ff  = [(f.name, f.type, f) for f in dc.fields(dc.dataclass(orig))]
        ff += [(f.name, f.type, f) for f in dc.fields(dc.dataclass(parent))]

        new = dc.make_dataclass(orig.__name__, ff)
        new.__doc__ = orig.__doc__

        return new
    return decorator

class Animal:
    age: int = 0 

@add_args(Animal)
class Dog:
    name: str
    noise: str = "Woof!"

@add_args(Animal)
class Bird:
    name: str
    can_fly: bool = True

Dog("Dusty", 2)               # --> Dog(name='Dusty', noise=2, age=0)
b = Bird("Donald", False, 40) # --> Bird(name='Donald', can_fly=False, age=40)

也可以预先添加非默认字段, 通过检查f.default是否为dc。失踪, 但这可能太脏了。

虽然猴子补丁缺乏遗传的一些特征, 它仍然可以用于向所有伪子类添加方法。

对于更细粒度的控制,请设置默认值 使用直流。字段(compare=False, repr=True,…)

下面的方法在使用纯python数据类和没有太多样板代码的情况下处理这个问题。

丑陋的:数据类。InitVar[bool]只是作为一个伪字段来帮助我们进行初始化,一旦创建实例就会丢失。而_ugly: bool = field(init=False)是一个实例成员,它不会通过__init__方法初始化,但也可以使用__post_init__方法初始化(你可以在这里找到更多)。

from dataclasses import dataclass, field, InitVar

@dataclass
class Parent:
    name: str
    age: int
    ugly: InitVar[bool]
    _ugly: bool = field(init=False)

    def __post_init__(self, ugly: bool):
        self._ugly = ugly

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f'The Name is {self.name} and {self.name} is {self.age} year old')

@dataclass
class Child(Parent):
    school: str

jack = Parent('jack snr', 32, ugly=True)
jack_son = Child('jack jnr', 12, school='havard', ugly=True)

jack.print_id()
jack_son.print_id()

注意,这使得字段ugly成为强制性的,使其成为可选的。你可以在父类上定义一个类方法,其中包含ugly作为可选参数:

from dataclasses import dataclass, field, InitVar

@dataclass
class Parent:
    name: str
    age: int
    ugly: InitVar[bool]
    _ugly: bool = field(init=False)

    def __post_init__(self, ugly: bool):
        self._ugly = ugly
    
    @classmethod
    def create(cls, ugly=True, **kwargs):
        return cls(ugly=ugly, **kwargs)

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f'The Name is {self.name} and {self.name} is {self.age} year old')

@dataclass
class Child(Parent):
    school: str

jack = Parent.create(name='jack snr', age=32, ugly=False)
jack_son = Child.create(name='jack jnr', age=12, school='harvard')

jack.print_id()
jack_son.print_id()

现在您可以使用create(…)类方法作为创建父/子类的工厂方法,并使用默认值ugly。注意,这种方法必须使用命名参数才能工作。

你可以使用数据类的修改版本,它将生成一个只包含关键字的__init__方法:

import dataclasses


def _init_fn(fields, frozen, has_post_init, self_name):
    # fields contains both real fields and InitVar pseudo-fields.
    globals = {'MISSING': dataclasses.MISSING,
               '_HAS_DEFAULT_FACTORY': dataclasses._HAS_DEFAULT_FACTORY}

    body_lines = []
    for f in fields:
        line = dataclasses._field_init(f, frozen, globals, self_name)
        # line is None means that this field doesn't require
        # initialization (it's a pseudo-field).  Just skip it.
        if line:
            body_lines.append(line)

    # Does this class have a post-init function?
    if has_post_init:
        params_str = ','.join(f.name for f in fields
                              if f._field_type is dataclasses._FIELD_INITVAR)
        body_lines.append(f'{self_name}.{dataclasses._POST_INIT_NAME}({params_str})')

    # If no body lines, use 'pass'.
    if not body_lines:
        body_lines = ['pass']

    locals = {f'_type_{f.name}': f.type for f in fields}
    return dataclasses._create_fn('__init__',
                      [self_name, '*'] + [dataclasses._init_param(f) for f in fields if f.init],
                      body_lines,
                      locals=locals,
                      globals=globals,
                      return_type=None)


def add_init(cls, frozen):
    fields = getattr(cls, dataclasses._FIELDS)

    # Does this class have a post-init function?
    has_post_init = hasattr(cls, dataclasses._POST_INIT_NAME)

    # Include InitVars and regular fields (so, not ClassVars).
    flds = [f for f in fields.values()
            if f._field_type in (dataclasses._FIELD, dataclasses._FIELD_INITVAR)]
    dataclasses._set_new_attribute(cls, '__init__',
                       _init_fn(flds,
                                frozen,
                                has_post_init,
                                # The name to use for the "self"
                                # param in __init__.  Use "self"
                                # if possible.
                                '__dataclass_self__' if 'self' in fields
                                else 'self',
                                ))

    return cls


# a dataclass with a constructor that only takes keyword arguments
def dataclass_keyword_only(_cls=None, *, repr=True, eq=True, order=False,
              unsafe_hash=False, frozen=False):
    def wrap(cls):
        cls = dataclasses.dataclass(
            cls, init=False, repr=repr, eq=eq, order=order, unsafe_hash=unsafe_hash, frozen=frozen)
        return add_init(cls, frozen)

    # See if we're being called as @dataclass or @dataclass().
    if _cls is None:
        # We're called with parens.
        return wrap

    # We're called as @dataclass without parens.
    return wrap(_cls)

(也作为要点发布,用Python 3.6 backport测试)

这需要将子类定义为

@dataclass_keyword_only
class Child(Parent):
    school: str
    ugly: bool = True

并且会生成__init__(self, *, name:str, age:int, ugly:bool=True, school:str)(这是有效的python)。这里唯一的警告是不允许使用位置参数初始化对象,但除此之外,它是一个完全常规的数据类,没有丑陋的hack。

数据类组合属性的方式阻止您在基类中使用带有默认值的属性,然后在子类中使用没有默认值的属性(位置属性)。

That's because the attributes are combined by starting from the bottom of the MRO, and building up an ordered list of the attributes in first-seen order; overrides are kept in their original location. So Parent starts out with ['name', 'age', 'ugly'], where ugly has a default, and then Child adds ['school'] to the end of that list (with ugly already in the list). This means you end up with ['name', 'age', 'ugly', 'school'] and because school doesn't have a default, this results in an invalid argument listing for __init__.

这被记录在PEP-557数据类中,在继承下:

When the Data Class is being created by the @dataclass decorator, it looks through all of the class's base classes in reverse MRO (that is, starting at object) and, for each Data Class that it finds, adds the fields from that base class to an ordered mapping of fields. After all of the base class fields are added, it adds its own fields to the ordered mapping. All of the generated methods will use this combined, calculated ordered mapping of fields. Because the fields are in insertion order, derived classes override base classes.

规格项下:

如果一个没有默认值的字段紧跟在一个有默认值的字段之后,将引发TypeError。无论是在单个类中发生这种情况,还是作为类继承的结果,都是如此。

您确实有一些选择来避免这个问题。

第一个选项是使用单独的基类,将具有默认值的字段强制放到MRO顺序的后面位置。无论如何,避免直接在要用作基类的类上设置字段,例如Parent。

下面的类层次结构可以工作:

# base classes with fields; fields without defaults separate from fields with.
@dataclass
class _ParentBase:
    name: str
    age: int
    
@dataclass
class _ParentDefaultsBase:
    ugly: bool = False

@dataclass
class _ChildBase(_ParentBase):
    school: str

@dataclass
class _ChildDefaultsBase(_ParentDefaultsBase):
    ugly: bool = True

# public classes, deriving from base-with, base-without field classes
# subclasses of public classes should put the public base class up front.

@dataclass
class Parent(_ParentDefaultsBase, _ParentBase):
    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f"The Name is {self.name} and {self.name} is {self.age} year old")

@dataclass
class Child(_ChildDefaultsBase, Parent, _ChildBase):
    pass

通过将字段提取到具有无默认字段和具有默认字段的独立基类中,并仔细选择继承顺序,您可以生成一个MRO,将所有无默认字段放在具有默认字段之前。Child的反向MRO(忽略对象)是:

_ParentBase
_ChildBase
_ParentDefaultsBase
Parent
_ChildDefaultsBase

注意,虽然Parent没有设置任何新字段,但它确实从_ParentDefaultsBase继承了字段,并且不应该在字段列表顺序中以“最后”结束;上面的顺序把_ChildDefaultsBase放在最后,所以它的字段“win”。数据类规则也得到了满足;带默认字段的类(_ParentBase和_ChildBase)位于带默认字段的类(_ParentDefaultsBase和_ChildDefaultsBase)前面。

结果是父类和子类的字段都是旧的,而Child仍然是Parent的子类:

>>> from inspect import signature
>>> signature(Parent)
<Signature (name: str, age: int, ugly: bool = False) -> None>
>>> signature(Child)
<Signature (name: str, age: int, school: str, ugly: bool = True) -> None>
>>> issubclass(Child, Parent)
True

所以你可以创建这两个类的实例:

>>> jack = Parent('jack snr', 32, ugly=True)
>>> jack_son = Child('jack jnr', 12, school='havard', ugly=True)
>>> jack
Parent(name='jack snr', age=32, ugly=True)
>>> jack_son
Child(name='jack jnr', age=12, school='havard', ugly=True)

另一种选择是只使用默认字段;你仍然可以通过在__post_init__中引发一个错误来不提供学校值:

_no_default = object()

@dataclass
class Child(Parent):
    school: str = _no_default
    ugly: bool = True

    def __post_init__(self):
        if self.school is _no_default:
            raise TypeError("__init__ missing 1 required argument: 'school'")

但这确实改变了场的顺序;学校结束后丑陋:

<Signature (name: str, age: int, ugly: bool = True, school: str = <object object at 0x1101d1210>) -> None>

类型提示检查器会提示_no_default不是字符串。

您还可以使用attrs项目,该项目激发了数据类的灵感。它使用了不同的继承合并策略;它将子类中被覆盖的字段拉到字段列表的末尾,因此父类中的['name', 'age', 'ugly']在子类中变成了['name', 'age', 'school', 'ugly'];通过使用默认值重写字段,attrs允许重写而不需要执行MRO舞蹈。

attrs支持定义没有类型提示的字段,但是让我们通过设置auto_attribs=True坚持支持的类型提示模式:

import attr

@attr.s(auto_attribs=True)
class Parent:
    name: str
    age: int
    ugly: bool = False

    def print_name(self):
        print(self.name)

    def print_age(self):
        print(self.age)

    def print_id(self):
        print(f"The Name is {self.name} and {self.name} is {self.age} year old")

@attr.s(auto_attribs=True)
class Child(Parent):
    school: str
    ugly: bool = True

请注意,在Python 3.10中,现在可以使用数据类原生地进行此操作。

Dataclasses 3.10添加了kw_only属性(类似于attrs)。 它允许您指定哪些字段是keyword_only,因此将在init结束时设置,而不会导致继承问题。

直接从埃里克·史密斯关于这个主题的博客文章中摘录:

人们要求这个功能的原因有两个: 当一个数据类有很多字段时,通过位置指定它们可能变得不可读。为了向后兼容,它还要求将所有新字段添加到数据类的末尾。这并不总是可取的。 当一个数据类从另一个数据类继承,并且基类的字段具有默认值时,派生类中的所有字段也必须具有默认值。

下面是使用这个new参数的最简单的方法,但是有多种方法可以使用它来继承父类中的默认值:

from dataclasses import dataclass

@dataclass(kw_only=True)
class Parent:
    name: str
    age: int
    ugly: bool = False

@dataclass(kw_only=True)
class Child(Parent):
    school: str

ch = Child(name="Kevin", age=17, school="42")
print(ch.ugly)

看一下上面链接的博客文章,可以更彻底地解释kw_only。

干杯!

PS:由于它是相当新的,请注意您的IDE仍然可能会引发一个错误,但它在运行时工作