How to deep copy a list?
Question:
After E0_copy = list(E0), I guess E0_copy is a deep copy of E0 since id(E0) is not equal to id(E0_copy). Then I modify E0_copy in the loop, but why is E0 not the same after?
E0 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
for k in range(3):
E0_copy = list(E0)
E0_copy[k][k] = 0
#print(E0_copy)
print E0 # -> [[0, 2, 3], [4, 0, 6], [7, 8, 0]]
Answers:
E0_copy is not a deep copy. You don’t make a deep copy using list(). (Both list(...) and testList[:] are shallow copies, as well as testList.copy().)
You use copy.deepcopy(...) for deep copying a list.
copy.deepcopy(x[, memo])
Return a deep copy of x.
See the following snippet –
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b # b changes too -> Not a deepcopy.
[[1, 10, 3], [4, 5, 6]]
Now see the deepcopy operation
>>> import copy
>>> b = copy.deepcopy(a)
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
>>> a[0][1] = 9
>>> a
[[1, 9, 3], [4, 5, 6]]
>>> b # b doesn't change -> Deep Copy
[[1, 10, 3], [4, 5, 6]]
To explain, list(...) does not recursively make copies of the inner objects. It only makes a copy of the outermost list, while still referencing the same inner lists, hence, when you mutate the inner lists, the change is reflected in both the original list and the shallow copy. You can see that shallow copying references the inner lists by checking that id(a[0]) == id(b[0]) where b = list(a).
If your list elements are immutable objects then you can use this, otherwise you have to use deepcopy from copy module.
you can also use shortest way for deep copy a list like this.
a = [0,1,2,3,4,5,6,7,8,9,10]
b = a[:] #deep copying the list a and assigning it to b
print id(a)
20983280
print id(b)
12967208
a[2] = 20
print a
[0, 1, 20, 3, 4, 5, 6, 7, 8, 9,10]
print b
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10]
just a recursive deep copy function.
def deepcopy(A):
rt = []
for elem in A:
if isinstance(elem,list):
rt.append(deepcopy(elem))
else:
rt.append(elem)
return rt
Edit: As Cfreak mentioned, this is already implemented in copy module.
Regarding the list as a tree, the deep_copy in python can be most compactly written as
def deep_copy(x):
if not isinstance(x, list):
return x
else:
return [deep_copy(elem) for elem in x]
It’s basically recursively traversing the list in a depth-first way.
In Python, there is a module called copy with two useful functions:
import copy
copy.copy()
copy.deepcopy()
copy() is a shallow copy function. If the given argument is a compound data structure, for instance a list, then Python will create another object of the same type (in this case, a new list) but for everything inside the old list, only their reference is copied. Think of it like:
newList = [elem for elem in oldlist]
Intuitively, we could assume that deepcopy() would follow the same paradigm, and the only difference is that for each elem we will recursively call deepcopy, (just like mbguy’s answer)
but this is wrong!
deepcopy() actually preserves the graphical structure of the original compound data:
a = [1,2]
b = [a,a] # there's only 1 object a
c = deepcopy(b)
# check the result
c[0] is a # False, a new object a_1 is created
c[0] is c[1] # True, c is [a_1, a_1] not [a_1, a_2]
This is the tricky part: during the process of deepcopy(), a hashtable (dictionary in Python) is used to map each old object ref onto each new object ref, which prevents unnecessary duplicates and thus preserves the structure of the copied compound data.
If the contents of the list are primitive data types, you can use a comprehension
new_list = [i for i in old_list]
You can nest it for multidimensional lists like:
new_grid = [[i for i in row] for row in grid]
Here’s an example of how to deep copy a 2D list:
b = [x[:] for x in a]
@Sukrit Kalra
No.1: list(), [:], copy.copy() are all shallow copy. If an object is compound, they are all not suitable. You need to use copy.deepcopy().
No.2: b = a directly, a and b have the same reference, changing a is even as changing b.
set a to b
if assgin a to b directly, a and b share one reference.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[1, [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
shadow copy
by list()
list() and [:] are the same. Except for the first layer changes, all other layers’ changes will be transferred.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
by [:]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
list() and [:] change the other layers, except for the 1st layer
# =========== [:] ===========
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2] = 4
>>> a
[[1, 2, 4], [4, 5, 6]]
>>> b
[[1, 2, 4], [4, 5, 6]]
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2][0] = 999
>>> a
[[1, 2, [999, 6]], [4, 5, 6]]
>>> b
[[1, 2, [999, 6]], [4, 5, 6]]
# =========== list() ===========
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2] = 4
>>> a
[[1, 2, 4], [4, 5, 6]]
>>> b
[[1, 2, 4], [4, 5, 6]]
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2][0] = 999
>>> a
[[1, 2, [999, 6]], [4, 5, 6]]
>>> b
[[1, 2, [999, 6]], [4, 5, 6]]
by copy()
You will find that copy() function is the same as list() and [:]. They are all shallow copy.
For much more information about shallow copy and deep copy, maybe you can reference here.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.copy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
by deepcopy()
>>> import copy
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.deepcopy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.deepcopy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
If you are not allowed to directly import modules you can define your own deepcopy function as –
def copyList(L):
if type(L[0]) != list:
return [i for i in L]
else:
return [copyList(L[i]) for i in range(len(L))]
It’s working can be seen easily as –
>>> x = [[1,2,3],[3,4]]
>>> z = copyList(x)
>>> x
[[1, 2, 3], [3, 4]]
>>> z
[[1, 2, 3], [3, 4]]
>>> id(x)
2095053718720
>>> id(z)
2095053718528
>>> id(x[0])
2095058990144
>>> id(z[0])
2095058992192
>>>
If you are assigning to the same list using deepcopy,
use a temporary variable instead. For some reason, copy.deepcopy() does not work on object lists when trying to self update same variable using indexing
S = S[idx] ->x
S = copy.deepcopy(S[idx]) -> x
vvvv although this worked
Stemp = np.zeros(N,dtype=object)
for ii in range(N):
Stemp[ii]=copy.deepcopy(S[idx[ii]])
S= copy.deepcopy(Stemp)
After E0_copy = list(E0), I guess E0_copy is a deep copy of E0 since id(E0) is not equal to id(E0_copy). Then I modify E0_copy in the loop, but why is E0 not the same after?
E0 = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
for k in range(3):
E0_copy = list(E0)
E0_copy[k][k] = 0
#print(E0_copy)
print E0 # -> [[0, 2, 3], [4, 0, 6], [7, 8, 0]]
E0_copy is not a deep copy. You don’t make a deep copy using list(). (Both list(...) and testList[:] are shallow copies, as well as testList.copy().)
You use copy.deepcopy(...) for deep copying a list.
copy.deepcopy(x[, memo])Return a deep copy of x.
See the following snippet –
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b # b changes too -> Not a deepcopy.
[[1, 10, 3], [4, 5, 6]]
Now see the deepcopy operation
>>> import copy
>>> b = copy.deepcopy(a)
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
>>> a[0][1] = 9
>>> a
[[1, 9, 3], [4, 5, 6]]
>>> b # b doesn't change -> Deep Copy
[[1, 10, 3], [4, 5, 6]]
To explain, list(...) does not recursively make copies of the inner objects. It only makes a copy of the outermost list, while still referencing the same inner lists, hence, when you mutate the inner lists, the change is reflected in both the original list and the shallow copy. You can see that shallow copying references the inner lists by checking that id(a[0]) == id(b[0]) where b = list(a).
If your list elements are immutable objects then you can use this, otherwise you have to use deepcopy from copy module.
you can also use shortest way for deep copy a list like this.
a = [0,1,2,3,4,5,6,7,8,9,10]
b = a[:] #deep copying the list a and assigning it to b
print id(a)
20983280
print id(b)
12967208
a[2] = 20
print a
[0, 1, 20, 3, 4, 5, 6, 7, 8, 9,10]
print b
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10]
just a recursive deep copy function.
def deepcopy(A):
rt = []
for elem in A:
if isinstance(elem,list):
rt.append(deepcopy(elem))
else:
rt.append(elem)
return rt
Edit: As Cfreak mentioned, this is already implemented in copy module.
Regarding the list as a tree, the deep_copy in python can be most compactly written as
def deep_copy(x):
if not isinstance(x, list):
return x
else:
return [deep_copy(elem) for elem in x]
It’s basically recursively traversing the list in a depth-first way.
In Python, there is a module called copy with two useful functions:
import copy
copy.copy()
copy.deepcopy()
copy() is a shallow copy function. If the given argument is a compound data structure, for instance a list, then Python will create another object of the same type (in this case, a new list) but for everything inside the old list, only their reference is copied. Think of it like:
newList = [elem for elem in oldlist]
Intuitively, we could assume that deepcopy() would follow the same paradigm, and the only difference is that for each elem we will recursively call deepcopy, (just like mbguy’s answer)
but this is wrong!
deepcopy() actually preserves the graphical structure of the original compound data:
a = [1,2]
b = [a,a] # there's only 1 object a
c = deepcopy(b)
# check the result
c[0] is a # False, a new object a_1 is created
c[0] is c[1] # True, c is [a_1, a_1] not [a_1, a_2]
This is the tricky part: during the process of deepcopy(), a hashtable (dictionary in Python) is used to map each old object ref onto each new object ref, which prevents unnecessary duplicates and thus preserves the structure of the copied compound data.
If the contents of the list are primitive data types, you can use a comprehension
new_list = [i for i in old_list]
You can nest it for multidimensional lists like:
new_grid = [[i for i in row] for row in grid]
Here’s an example of how to deep copy a 2D list:
b = [x[:] for x in a]
@Sukrit Kalra
No.1: list(), [:], copy.copy() are all shallow copy. If an object is compound, they are all not suitable. You need to use copy.deepcopy().
No.2: b = a directly, a and b have the same reference, changing a is even as changing b.
set a to b
if assgin a to b directly, a and b share one reference.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[1, [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
shadow copy
by list()
list() and [:] are the same. Except for the first layer changes, all other layers’ changes will be transferred.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
by [:]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
list() and [:] change the other layers, except for the 1st layer
# =========== [:] ===========
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2] = 4
>>> a
[[1, 2, 4], [4, 5, 6]]
>>> b
[[1, 2, 4], [4, 5, 6]]
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = a[:]
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2][0] = 999
>>> a
[[1, 2, [999, 6]], [4, 5, 6]]
>>> b
[[1, 2, [999, 6]], [4, 5, 6]]
# =========== list() ===========
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2] = 4
>>> a
[[1, 2, 4], [4, 5, 6]]
>>> b
[[1, 2, 4], [4, 5, 6]]
>>> a = [[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b = list(a)
>>> a
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> b
[[1, 2, [3.5, 6]], [4, 5, 6]]
>>> a[0][2][0] = 999
>>> a
[[1, 2, [999, 6]], [4, 5, 6]]
>>> b
[[1, 2, [999, 6]], [4, 5, 6]]
by copy()
You will find that copy() function is the same as list() and [:]. They are all shallow copy.
For much more information about shallow copy and deep copy, maybe you can reference here.
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.copy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 10, 3], [4, 5, 6]]
by deepcopy()
>>> import copy
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.deepcopy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0] = 1
>>> a
[1, [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> b = copy.deepcopy(a)
>>> a
[[1, 2, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
>>> a[0][1] = 10
>>> a
[[1, 10, 3], [4, 5, 6]]
>>> b
[[1, 2, 3], [4, 5, 6]]
If you are not allowed to directly import modules you can define your own deepcopy function as –
def copyList(L):
if type(L[0]) != list:
return [i for i in L]
else:
return [copyList(L[i]) for i in range(len(L))]
It’s working can be seen easily as –
>>> x = [[1,2,3],[3,4]]
>>> z = copyList(x)
>>> x
[[1, 2, 3], [3, 4]]
>>> z
[[1, 2, 3], [3, 4]]
>>> id(x)
2095053718720
>>> id(z)
2095053718528
>>> id(x[0])
2095058990144
>>> id(z[0])
2095058992192
>>>
If you are assigning to the same list using deepcopy,
use a temporary variable instead. For some reason, copy.deepcopy() does not work on object lists when trying to self update same variable using indexing
S = S[idx] ->x
S = copy.deepcopy(S[idx]) -> x
vvvv although this worked
Stemp = np.zeros(N,dtype=object)
for ii in range(N):
Stemp[ii]=copy.deepcopy(S[idx[ii]])
S= copy.deepcopy(Stemp)