Python: how to get size of all objects in current namespace?

Question

I have some code that I am running from my own package and the program is using a lot more memory (60GB) than it should be. How can I print the size of all objects (in bytes) in the current namespace in order to attempt to work out where this memory is being used?

I attempted something like

from pympler import asizeof

for objname in dir():
    print(asizeof.asizeof(thing)/1024) # print size in kb

But it doesn’t work as it just prints the size of the string containing the name of the object in the namespace. Is there a way to get an object reference to everything in the namespace in order to use this method or is there a better method for working out what is using the memory?

Asked By: SomeRandomPhysicist

||

Source

Answer 1

dir() returns only the names present in the local scope. Use the locals() function to get the local scope as a dictionary:

for obj in locals().values():
        print(asizeof.asizeof(obj) / 1024)

Note that outside of functions, locals() is the same mapping as globals().

If asizeof() is in the dictionary, you want to filter it out:

for name, obj in locals().items():
    if name != 'asizeof':
        print(asizeof.asizeof(obj) / 1024)

dir() without arguments is functionally equivalent to sorted(locals()) (a sorted list of the keys of the local namespace).

Answered By: Martijn Pieters

Answer 2

You can use gc.get_objects() to just fetch all objects tracked by the collector, not just those in a specific namespace. I’d start by using it to count the number of instances of each type as that might give you some clues in itself.

from collections import Counter
import gc

c = Counter(type(o) for o in gc.get_objects())
print(c.most_common(20))

Then you might drill down to find the size of any likely suspects.

Answered By: Duncan

Answer 3

If you prefer to use a standard library and also want them sorted by size:

import sys
objects=[]
for name,obj in locals().items():
   objects.append([name,sys.getsizeof(obj)])
sorted(objects,key=lambda x: x[1],reverse=True)

Answered By: otocan

Python: how to get size of all objects in current namespace?

Question:

Answers: