Programming Python

Python Gotchas No. 2: Garbage Collection Oddities

Python is a garbage collected language. The garbage collector will collect orphaned objects. These are objects that have no references.
If an object has a __del__ method, it will be called when that object is collected. Note however, that there are no guarantees as to when this will take place. This means that while you should release all owned resources in the __del__ method, you should not depend on it to release these resources when the object “goes out of scope” as in C++.
Specifically note that del x does not call x’s __del__ method, just removes this specific reference to x.
To be explicit about resource management, call the appropriate function yourself in the flow of your program, or better yet, use try-finally or the with statement. (For more information about those, see my presentation about advanced python subjects).

There is another interesting caveat regarding Python’s gc. Consider the following code:

class A(object):
a = A()
b = A()
a.x = b
b.x = a
del a
del b

Did a and b lose their references? Well, they didn’t. They’re still pointing at each other, creating a reference cycle. Happily, Python’s garbage collector can still handle those. However, it won’t be able to handle reference cycles if at least one object in the cycle has a __del__ method.
To understand why, consider the above example, only this time assume A has a __del__ method that calls self.x.release().

Now, which __del__ method should be called first? if it is called, is the other one still valid? Python refuses the temptation to guess, and leaves the cycle be, creating a memory (or resource) leak.
The solution?
1. Avoid data structures with reference cycles. For instance, there are very few instances where you’d need a doubly linked list :)
2. If you do need reference cycles, consider using the weakref module, which allows you to create references that “don’t count” in the eyes of the gc.

Further reading material:
1. The __del__ method.
2. The gc (garbage collector) module.

One reply on “Python Gotchas No. 2: Garbage Collection Oddities”

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.