Must Have Python Packages

Python 2.6 and 3.0 are upon us.
I helped a friend set up the new Python environment for research and development, and we thought about constructing a PythonPack: a compilation of packages every home should have lying around.

I decided I’d write down my list of generic Python packages: packages you’d want to have regardless of what you were working on. So I didn’t include any of the numerous web-development platforms, or some other very specialized packages.

What you won’t see in this list however, are IDE’s and editors. I don’t feel like joining in on the religious wars just yet :)

So without further ado, here’s my list:

• IPython – I just can’t stress enough how useful this tool is. If you ever worked with Python’s interactive prompt, this is the same thing, just on steroids. It has completion, history, shell capabilities, you name it. If you didn’t yet work interactively, you won’t do badly to start with IPython.
• ActiveState Python, or the win32 python extensions – if you’re developing on win32, it will allow you to get at the native win32 API, which I usually found as a blessing
• PyGame – If you want to start game development on Python, that’s the tool for you. Even if you don’t, I found out I’ve used PyGame many times just for quick and dirty visualizations. Besides, it makes for a nice toy :)
• numpy – Ever needed to do any computation that’s a bit more complicated than 1/(1-math.sin(x))**2? numpy is the way to go, with plenty of tools for vectors, matrices, and what not.
• After mentioning numpy, it’s important to mention scipy and pylab (matplotlib). Just a bit of visualization strength, in the vein of matlab.
• coverage.py – Python code coverage by Ned Batchelder. An essential tool for effective testing. While you’re there I’ll also recommend checking out his blog.
• ply – for most of your text parsing needs. Of course, this is just my preference. I heard about (but didn’t use) PyParsing and other modules
• construct – If you intend to do some binary parsing, construct is an excellent tool for that.
• PyOpenGL – I use it mostly for 3D visualizations, see for example my recursive spring.
• PIL – Python Imaging Library is a very useful tool for working with images. I used it in my twindragon fractal implementation
• nltk – Natural Language Toolkit – I realize most people won’t be using this one, but it’s generic enough, and I’ve used it enough times so far to include it in here. For example, I used it when I wanted to find rhymes.
• WinPDB – A Python debugger especially useful for embedded Python apps (Python running from within another application).
• nose – the framework I use for testing. Easy to start using it if you’re already using unittest.
• psyco – if it so happens that you start to care about speed, psyco is by far the easiest way to go about doing it, short of writing your code in a different language.

Other useful but less often used packages include:

• pefile – for parsing win32 pe files.
• PyDBG (from within PaiMei) – an interace to the win32 debugging api (there’s an upcoming post about that one :)
• scapy – I didn’t use it lately, but if you want to sniff packets programmatically, that’s a good way to go about doing that
• Peach – Easy Python tool for fuzzing. Didn’t get the chance to use it myself though.
• diStorm – the fastest disassembler for x86 and amd64 around, with a Python interface. Of course, it helps if I’m also working with gil on the next version :)
• sqlalchemy and sqlobject – your handy ORMs. I use them mostly for web development. For other work I don’t often need a full blown db.
• wx and PyGTK– I really don’t do that much GUI work, so I don’t have really much to say about those two. I still have them lying around though, just in case.
• RPyC – Didn’t get a chance to use it myself, but if you want to get into distributed Python programming, I’ll look into RPyC first.
• NetworkX – if you happen to need some graph algorithm or visualization, I’d go for networkx. Last time I used graphs I mostly wrote my own code, and discovered networkx only afterwards. Next time I’ll be sure to use it.
• SymPy – Symbolic Python, in the vein of Maple and Mathematica. It allows you to do symbolic operations for example, integrating f(x)=sin(x) and still getting a function and not a number. I used it a little bit for my homework.

Now it’s your turn: what packages did I miss?