Should I worry about circular references in Python?

Question

Suppose I have code that maintains a parent/children structure. In such a structure I get circular references, where a child points to a parent and a parent points to a child. Should I worry about them? I’m using Python 2.5.

I am concerned that they will not be garbage collected and the application will eventually consume all memory.

Asked By: bodacydo

||

Source

Answer 1

Python will detect the cycle and release the memory when there are no outside references.

Answered By: Ned Batchelder

Answer 2

Experimentally: you’re fine:

import itertools

for i in itertools.count():
    a = {}
    b = {"a":a}
    a["b"] = b

It consistently stays at using 3.6 MB of RAM.

Answered By: cobbal

Answer 3

Circular references are a normal thing to do, so I don’t see a reason to be worried about them. Many tree algorithms require that each node have links to its children and its parent. They’re also required to implement something like a doubly linked list.

Answered By: Colin

Answer 4

I don’t think you should worry. Try the following program and will you see that it won’t consume all memory:

while True:
    a=range(100)
    b=range(100)
    a.append(b)
    b.append(a)
    a.append(a)
    b.append(b)

Answered By: douglaz

Answer 5

“Worry” is misplaced, but if your program turns out to be slow, consume more memory than expected, or have strange inexplicable pauses, the cause is indeed likely to be in those garbage reference loops — they need to be garbage collected by a different procedure than “normal” (acyclic) reference graphs, and that collection is occasional and may be slow if you have a lot of objects tied up in such loops (the cyclical-garbage collection is also inhibited if an object in the loop has a __del__ special method).

So, reference loops will not affect your program’s correctness, but may affect its performance and/or footprint.

If and when you want to remove unwanted loops of references, you can often use the weakref module in Python’s standard library.

If and when you want to exert more direct control (or perform debugging, see what exactly is happening) regarding cyclical garbage collection, use the gc module in Python’s standard library.

Answered By: Alex Martelli

Answer 6

There seems to be a issue with references to methods in lists in a variable. Here are two examples. The first one does not call __del__. The second one with weakref is ok for __del__. However, in this later case the problem is that you cannot weakly reference methods: http://docs.python.org/2/library/weakref.html

import sys, weakref

class One():
    def __init__(self):
        self.counters = [ self.count ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(One)
one = One()
sys.getrefcount(One)
del one
sys.getrefcount(One)


class Two():
    def __init__(self):
        self.counters = [ weakref.ref(self.count) ]
    def __del__(self):
        print("__del__ called")
    def count(self):
        print(sys.getrefcount(self))


sys.getrefcount(Two)
two = Two()
sys.getrefcount(Two)
del two
sys.getrefcount(Two)

Answered By: Finn Årup Nielsen

Should I worry about circular references in Python?

Question:

Answers: