Comments on: Python module usage statistics

By: lorg

lorg — Thu, 22 Nov 2007 07:18:03 +0000

I agree. That’s why before I started acquiring these statistics, I sat down and wrote what I thought the results should be.
You *need* the stdlib, there’s no escaping that. os and sys weren’t surprises. types was (at least for me).
Also, I don’t believe sys.argv alone explains the high usage of the sys module. You’d think that sys.argv would be used once per project, in the main file, but if this was the only usage, the numbers should have been much lower (please check the updated numbers in the next post though).

It might be interesting to do statistics on module usage as well – just count the number of different strings after the dot, as in “module_name.usage”.

By: bulkanix

bulkanix — Thu, 22 Nov 2007 05:58:34 +0000

It’s not surprising to see that sys and os are the most used modules. For example you need sys to get access to argv (command line arguments) and os (os.path) for path handling and most modules should use os for OS independent path handling :)

By: lorg

lorg — Wed, 21 Nov 2007 06:57:17 +0000

Doug: Nice idea, although I believe the results will be a little slanted, as parts of the stdlib are implemented in C. As for the code, I’ll release it in a few days.
Phillip: I didn’t exclude setup.py files on purpose. I figured that there is probably one setup.py tools per project, so there should be one distutils import per project, while other imports may appear more than once. This way, I still got to know how common is distutil usage relative to other modules. The original post by Doug mentioned distutils as a ‘problematic’ module, so those statistics were relevant.

By: Phillip J. Eby

Phillip J. Eby — Wed, 21 Nov 2007 00:42:05 +0000

Did you exclude setup.py files from your analysis? If not, that’s probably why setuptools, distutils.core, and ez_setup are over-represented in your results.

By: Doug Hellmann

Doug Hellmann — Tue, 20 Nov 2007 20:18:39 +0000

That’s an interesting analysis, Imri. I like the idea of using PyPI for the research. Have you posted the code?

It would be interesting to run the same tool against the standard library to see which modules are reused internally, too.