GeneratorExit: another reason to upgrade to Python 2.6

Ever heard of GeneratorExit? Unless you write generators, and do some unusual stuff with them, you probably haven’t encountered it.

GeneratorExit is a special exception that gets raised from within a generator when it is close()-ed.
For example, consider the following code:

def func():
	f = file('temp.txt','w')
	for i in range(10):
		f.write('%d\n' % i)
		yield i
	f.close()

When iterating over func(), when will f be closed? Obviously, when the generator is exhausted. What if f.write raised an exception? It’s better to make sure then (in py2.5, do from __future__ import with-statement):

def func():
    with file('temp.txt','w') as f:
        for i in range(10):
            f.write('%d\n' % i)
            yield i

(If you don’t know what with does: basically it wraps the code in a try-finally clause, and makes sure that f is closed in the finally).

This solution is clean enough, but there’s something interesting going on. What if we use func in the following manner?

for i in func():
    print i
    if i == 5:
        break

Well, the generator func() returned is no longer used, so eventually it will be cleaned by the gc. However, the code inside func is still executing – it is “blocking” on the yield statement!

To solve this issue cleanly, the yield statement will raise an exception when the generator returned is close()-ed.

What does this have to do with upgrading to Python 2.6?
It turns out that GeneratorExit is supposed to inherit from BaseException, just like KeyboardInterrupt and StopIteration, so that except Exception won’t catch it. However, in Python 2.5 it inherits from Exception, which means that the following sample code will behave strangely:

def func():
    with file('temp.txt','w') as f:
        for i in range(10):
            try:
                f.write('%d\n' % i)
                yield i
            except Exception:
                pass
 
In [13]: for i in func():
   ....:     print i
   ....:     if i == 5:
   ....:         break
   ....:
   ....:
0
1
2
3
4
5
Exception exceptions.RuntimeError: 'generator ignored GeneratorExit' in <generator object at 0x021BEAD0> ignored

The workaround is simple: before except Exception, catch GeneratorExit and handle it correctly. Or, upgrade to Python 2.6 instead :)