How to override the copy/deepcopy operations for a Python object?

I might be a bit off on the specifics, but here goes;

From the copy docs;

• A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.
• A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

In other words: copy() will copy only the top element and leave the rest as pointers into the original structure. deepcopy() will recursively copy over everything.

That is, deepcopy() is what you need.

If you need to do something really specific, you can override __copy__() or __deepcopy__(), as described in the manual. Personally, I’d probably implement a plain function (e.g. config.copy_config() or such) to make it plain that it isn’t Python standard behaviour.

Putting together Alex Martelli’s answer and Rob Young’s comment you get the following code:

from copy import copy, deepcopy

class A(object):
    def __init__(self):
        print 'init'
        self.v = 10
        self.z = [2,3,4]

    def __copy__(self):
        cls = self.__class__
        result = cls.__new__(cls)
        result.__dict__.update(self.__dict__)
        return result

    def __deepcopy__(self, memo):
        cls = self.__class__
        result = cls.__new__(cls)
        memo[id(self)] = result
        for k, v in self.__dict__.items():
            setattr(result, k, deepcopy(v, memo))
        return result

a = A()
a.v = 11
b1, b2 = copy(a), deepcopy(a)
a.v = 12
a.z.append(5)
print b1.v, b1.z
print b2.v, b2.z

prints

init
11 [2, 3, 4, 5]
11 [2, 3, 4]

here __deepcopy__ fills in the memo dict to avoid excess copying in case the object itself is referenced from its member.

Its not clear from your problem why you need to override these methods, since you don’t want to do any customization to the copying methods.

Anyhow, if you do want to customize the deep copy (e.g. by sharing some attributes and copying others), here is a solution:

from copy import deepcopy


def deepcopy_with_sharing(obj, shared_attribute_names, memo=None):
    '''
    Deepcopy an object, except for a given list of attributes, which should
    be shared between the original object and its copy.

    obj is some object
    shared_attribute_names: A list of strings identifying the attributes that
        should be shared between the original and its copy.
    memo is the dictionary passed into __deepcopy__.  Ignore this argument if
        not calling from within __deepcopy__.
    '''
    assert isinstance(shared_attribute_names, (list, tuple))
    shared_attributes = {k: getattr(obj, k) for k in shared_attribute_names}

    if hasattr(obj, '__deepcopy__'):
        # Do hack to prevent infinite recursion in call to deepcopy
        deepcopy_method = obj.__deepcopy__
        obj.__deepcopy__ = None

    for attr in shared_attribute_names:
        del obj.__dict__[attr]

    clone = deepcopy(obj)

    for attr, val in shared_attributes.iteritems():
        setattr(obj, attr, val)
        setattr(clone, attr, val)

    if hasattr(obj, '__deepcopy__'):
        # Undo hack
        obj.__deepcopy__ = deepcopy_method
        del clone.__deepcopy__

    return clone



class A(object):

    def __init__(self):
        self.copy_me = []
        self.share_me = []

    def __deepcopy__(self, memo):
        return deepcopy_with_sharing(self, shared_attribute_names = ['share_me'], memo=memo)

a = A()
b = deepcopy(a)
assert a.copy_me is not b.copy_me
assert a.share_me is b.share_me

c = deepcopy(b)
assert c.copy_me is not b.copy_me
assert c.share_me is b.share_me

Following Peter’s excellent answer, to implement a custom deepcopy, with minimal alteration to the default implementation (e.g. just modifying a field like I needed) :

class Foo(object):
    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        cp.__deepcopy__ = deepcopy_method

        # custom treatments
        # for instance: cp.id = None

        return cp

Edit: a limitation of this approach, as Igor Kozyrenko points out, is that the copies’ __deepcopy__ will still be bound to the original object, so a copy of a copy will actually be a copy of the original. There’s perhaps a way to re-bind the __deepcopy__ to cp, instead of just assigning it with cp.__deepcopy__ = deepcopy_method

Building on Antony Hatchkins’ clean answer, here’s my version where the class in question derives from another custom class (s.t. we need to call super):

class Foo(FooBase):
    def __init__(self, param1, param2):
        self._base_params = [param1, param2]
        super(Foo, result).__init__(*self._base_params)

    def __copy__(self):
        cls = self.__class__
        result = cls.__new__(cls)
        result.__dict__.update(self.__dict__)
        super(Foo, result).__init__(*self._base_params)
        return result

    def __deepcopy__(self, memo):
        cls = self.__class__
        result = cls.__new__(cls)
        memo[id(self)] = result
        for k, v in self.__dict__.items():
            setattr(result, k, copy.deepcopy(v, memo))
        super(Foo, result).__init__(*self._base_params)
        return result

The copy module uses eventually the __getstate__()/__setstate__() pickling protocol, so these are also valid targets to override.

The default implementation just returns and sets the __dict__ of the class, so you don’t have to call super() and worry about Eino Gourdin’s clever trick, above.

Peter‘s and Eino Gourdin‘s answers are clever and useful, but they have a very subtle bug!

Python methods are bound to their object. When you do cp.__deepcopy__ = deepcopy_method, you are actually giving the object cp a reference to __deepcopy__ on the original object. Any calls to cp.__deepcopy__ will return a copy of the original!
If you deepcopy your object and then deepcopy that copy, the output is a NOT a copy of the copy!

Here’s a minimal example of the behavior, along with my fixed implementation where you copy the __deepcopy__ implementation and then bind it to the new object:

from copy import deepcopy
import types


class Good:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        # Copy the function object
        func = types.FunctionType(
            deepcopy_method.__code__,
            deepcopy_method.__globals__,
            deepcopy_method.__name__,
            deepcopy_method.__defaults__,
            deepcopy_method.__closure__,
        )
        # Bind to cp and set
        bound_method = func.__get__(cp, cp.__class__)
        cp.__deepcopy__ = bound_method

        return cp


class Bad:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        cp.__deepcopy__ = deepcopy_method
        return cp


x = Bad()
copy = deepcopy(x)
copy.i = 1
copy_of_copy = deepcopy(copy)
print(copy_of_copy.i)  # 0

x = Good()
copy = deepcopy(x)
copy.i = 1
copy_of_copy = deepcopy(copy)
print(copy_of_copy.i)  # 1

I came here for performance reasons. Using the default copy.deepcopy() function was slowing down my code by up to 30 times.
Using the answer by @Anthony Hatchkins as a starting point, I realized that copy.deepcopy() is really slow for e.g. lists. I replaced the setattr loop with simple [:] slicing to copy whole lists. For anyone concerned with performance it is worthwhile doing timeit.timeit() comparisons and replacing the calls to copy.deepcopy() by faster alternatives.

setup = 'import copy; l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]'
timeit.timeit(setup = setup, stmt='m=l[:]')
timeit.timeit(setup = setup, stmt='m=l.copy()')
timeit.timeit(setup = setup, stmt='m=copy.deepcopy(l)')

will give these results:

0.11505379999289289
0.09126630000537261
6.423627900003339

Similar with Zach Price‘s thoughts, there is a simpler way to achieve that goal, i.e. unbind the original __deepcopy__ method then bind it to cp

from copy import deepcopy
import types


class Good:
    def __init__(self):
        self.i = 0

    def __deepcopy__(self, memo):
        deepcopy_method = self.__deepcopy__
        self.__deepcopy__ = None
        cp = deepcopy(self, memo)
        self.__deepcopy__ = deepcopy_method
        
        # Bind to cp by types.MethodType
        cp.__deepcopy__ = types.MethodType(deepcopy_method.__func__, cp)

        return cp