Why does creating a list from a list make it larger?
Question:
I’m seeing some inconsistencies when using sys.getsizeof on what should be identical lists. (Python 2.7.5)
>>> lst = [0,1,2,3,4,5,6,7,8,9]
>>> sys.getsizeof(lst)
76
>>> lst2 = list(lst)
>>> sys.getsizeof(lst2)
104
>>> lst3 = list(lst2)
>>> sys.getsizeof(lst3)
104
>>> sys.getsizeof(lst[:])
76
>>> sys.getsizeof(lst2[:])
76
Does anybody have a simple explanation?
Answers:
When you create a list literal, the size reported is the minimum size needed to hold the data. You can see this because the size jumps up if you append a single element. However, when you use list to copy it, it allocates some extra space – it takes a few appends before it reallocates (in your case, I suspect the 8th append will do it – it needs 4 more bytes per element). There is probably a reason why these allocation behaviors are different, but I’m not sure what that might be.
With a list literal, the VM creates the list with a set length. When passing a sequence to the list() constructor the elements are added one by one (via list.extend()) and as such the list is resized when appropriate. Since the resize operation overallocates in order to amortize the cost, the final list will usually be larger than the source list.
I’m seeing some inconsistencies when using sys.getsizeof on what should be identical lists. (Python 2.7.5)
>>> lst = [0,1,2,3,4,5,6,7,8,9]
>>> sys.getsizeof(lst)
76
>>> lst2 = list(lst)
>>> sys.getsizeof(lst2)
104
>>> lst3 = list(lst2)
>>> sys.getsizeof(lst3)
104
>>> sys.getsizeof(lst[:])
76
>>> sys.getsizeof(lst2[:])
76
Does anybody have a simple explanation?
When you create a list literal, the size reported is the minimum size needed to hold the data. You can see this because the size jumps up if you append a single element. However, when you use list to copy it, it allocates some extra space – it takes a few appends before it reallocates (in your case, I suspect the 8th append will do it – it needs 4 more bytes per element). There is probably a reason why these allocation behaviors are different, but I’m not sure what that might be.
With a list literal, the VM creates the list with a set length. When passing a sequence to the list() constructor the elements are added one by one (via list.extend()) and as such the list is resized when appropriate. Since the resize operation overallocates in order to amortize the cost, the final list will usually be larger than the source list.