Most useful Python modules from the standard library?
Question:
I am teaching a graduate level Python class at the University of Paris, and the students need to be introduced to the standard library. I want to discuss with them about some of the most important standard modules.
What modules do you think are absolute musts? Even though responses probably vary depending on your field (web programming, science, etc.), I feel that some modules are commonly needed: math, sys, re, os, os.path, logging,… and maybe: collections, struct,…
What modules would you suggest I present, in a 1 or 2 hour slot?
Answers:
I’d offer itertools and functools. These modules operate over abstractions that are found everywhere in programming, so I think they are useful to study. Among more practical things, xml modules (xml.dom, xml.sax) can be very useful.
Have a look at PyMOTW (Python Module Of The Week). Although it is not strictly stdlib, it’s a great resource of obvious and not so obvious gems of the python stdlib. What’s more, it also serves as excellent documentation of the introduced modules.
Aside from those you mentioned, I found subprocess and sqlite3 modules particularly useful. But I would certainly advice to students to take a look at the list of standard library modules themselves. Also, from modules outside of standard library, I would mention numpy (or numarray) and pyparsing.
I’d place some weight on the decimal module. If they are beginners at programming, they certainly won’t be aware of the implications of floating point accuracy. The decimal module is extremely valuable if working with currency or other units that must retain exact decimal precision through several mathematic operations.
Of course, you’d probably want to touch on situations when you don’t need to be that accurate as well.
I would add urllib2 to the list.
In only a one-two hour slot, I would introduce easy_install and the PyPI repository: even if they are not in the standard lib, they enable you to install many other external modules, and it is the first place where to look when you can’t find in the standard lib.
Apart from that, I would introduce numpy, re, doctest/unittest, and maybe pickle.
Don’t forget about datetime, weakref, pickle, StringIO, heapq, may be threading.
And numpy also worths mentioning, although it is not from the standard library.
operator, next to what’s already mentioned.
Modules to cover in a 1-2 hour slot entirely depend on your audience’s interest or focus. What other classes are they taking? What are they prepared to make use of immediately?
Be sure to mention math, decimal and datetime and time and re.
For IT-types who will be doing file-oriented work: glob, fnmatch, os, os.path, tempfile, and shutil.
Database folks must hear about sqlite and json.
Simulation audience may want to hear about random.
Web developers must hear about urllib2 from a client point of view. Also Beautiful Soup and an XML parser of your choice.
Web developers must hear about logging and wsgiref from a server point of view.
It depends a little on what they will be doing and what level they are.
Some modules I wish someone pointed out to me when I started are:
- StringIO – to stop them from reimplementing it, which they will if they don’t discover it.
- logging – to put them on the right path when it comes to debug printouts
- pickle – to stop them from trying to use XML everywhere.
- xml.etree.ElementTree – To save them from the DOM model when they actually need to work with XML.
- pprint – to make nested structures in python less intimidating.
I’d go for a few modules which make the most sense to a typical computer user/programmer performing typical computer tasks. That way, there’s the largest chance that they might actually use python on their own time.
In my opinion, the operations most people will likely perform are file operations, for example going over every file in a directory and performing some action on it.
Therefore, I’d say the modules: os and os.path are probably the most important, and also mention glob, fnmatch and shutil. Also, subprocess might be very useful too, since it tends to get used in the above mentioned context.
Lastly, I’d go with optparse, since that will get them into very quickly making usable, programmer-friendly programs, which hopefully will also encourage them to actually write programs that other people want to use.
os and os.path: because those are the core modules which anyone will require to write platform independent code in python and students can switch from shell script to python script after learning os and os.path.
I think everyone here got all the important ones, except for sys. If you look at actual Python code, sys is probably one of the most commonly used modules (usually because of sys.version).
Also, it’s not really a module, but I would mention __future__.
And nobody should use Python without doing import this.
For science student, a rarely heard but powerful module ‘networkx’ will be valuable. But they need to install it first. This module is well documented: http://networkx.lanl.gov/index.html
I just remember a very practical module: copy.
I use the deepcopy() from it quite often.
It is hard to live without timeit
>>> # Python shell usage
... import timeit
>>> tt = timeit.Timer("foo = 'time this'", "print 'setup with this arg'")
>>> tt.timeit(number=1000)
setup with this arg
0.00021100044250488281
>>>
[mpenning@Bucksnort ~]$ # Bash shell usage
[mpenning@Bucksnort ~]$ # 5 runs with 1000 samples each.
[mpenning@Bucksnort ~]$ python -m timeit -n 1000 -r 5 -s "print 'setup w/ this arg'"
"foo = 'time this'"
setup w/ this arg
setup w/ this arg
setup w/ this arg
setup w/ this arg
setup w/ this arg
1000 loops, best of 5: 0.173 usec per loop
[mpenning@Bucksnort ~]$
I am teaching a graduate level Python class at the University of Paris, and the students need to be introduced to the standard library. I want to discuss with them about some of the most important standard modules.
What modules do you think are absolute musts? Even though responses probably vary depending on your field (web programming, science, etc.), I feel that some modules are commonly needed: math, sys, re, os, os.path, logging,… and maybe: collections, struct,…
What modules would you suggest I present, in a 1 or 2 hour slot?
I’d offer itertools and functools. These modules operate over abstractions that are found everywhere in programming, so I think they are useful to study. Among more practical things, xml modules (xml.dom, xml.sax) can be very useful.
Have a look at PyMOTW (Python Module Of The Week). Although it is not strictly stdlib, it’s a great resource of obvious and not so obvious gems of the python stdlib. What’s more, it also serves as excellent documentation of the introduced modules.
Aside from those you mentioned, I found subprocess and sqlite3 modules particularly useful. But I would certainly advice to students to take a look at the list of standard library modules themselves. Also, from modules outside of standard library, I would mention numpy (or numarray) and pyparsing.
I’d place some weight on the decimal module. If they are beginners at programming, they certainly won’t be aware of the implications of floating point accuracy. The decimal module is extremely valuable if working with currency or other units that must retain exact decimal precision through several mathematic operations.
Of course, you’d probably want to touch on situations when you don’t need to be that accurate as well.
I would add urllib2 to the list.
In only a one-two hour slot, I would introduce easy_install and the PyPI repository: even if they are not in the standard lib, they enable you to install many other external modules, and it is the first place where to look when you can’t find in the standard lib.
Apart from that, I would introduce numpy, re, doctest/unittest, and maybe pickle.
Don’t forget about datetime, weakref, pickle, StringIO, heapq, may be threading.
And numpy also worths mentioning, although it is not from the standard library.
operator, next to what’s already mentioned.
Modules to cover in a 1-2 hour slot entirely depend on your audience’s interest or focus. What other classes are they taking? What are they prepared to make use of immediately?
Be sure to mention math, decimal and datetime and time and re.
For IT-types who will be doing file-oriented work: glob, fnmatch, os, os.path, tempfile, and shutil.
Database folks must hear about sqlite and json.
Simulation audience may want to hear about random.
Web developers must hear about urllib2 from a client point of view. Also Beautiful Soup and an XML parser of your choice.
Web developers must hear about logging and wsgiref from a server point of view.
It depends a little on what they will be doing and what level they are.
Some modules I wish someone pointed out to me when I started are:
- StringIO – to stop them from reimplementing it, which they will if they don’t discover it.
- logging – to put them on the right path when it comes to debug printouts
- pickle – to stop them from trying to use XML everywhere.
- xml.etree.ElementTree – To save them from the DOM model when they actually need to work with XML.
- pprint – to make nested structures in python less intimidating.
I’d go for a few modules which make the most sense to a typical computer user/programmer performing typical computer tasks. That way, there’s the largest chance that they might actually use python on their own time.
In my opinion, the operations most people will likely perform are file operations, for example going over every file in a directory and performing some action on it.
Therefore, I’d say the modules: os and os.path are probably the most important, and also mention glob, fnmatch and shutil. Also, subprocess might be very useful too, since it tends to get used in the above mentioned context.
Lastly, I’d go with optparse, since that will get them into very quickly making usable, programmer-friendly programs, which hopefully will also encourage them to actually write programs that other people want to use.
os and os.path: because those are the core modules which anyone will require to write platform independent code in python and students can switch from shell script to python script after learning os and os.path.
I think everyone here got all the important ones, except for sys. If you look at actual Python code, sys is probably one of the most commonly used modules (usually because of sys.version).
Also, it’s not really a module, but I would mention __future__.
And nobody should use Python without doing import this.
For science student, a rarely heard but powerful module ‘networkx’ will be valuable. But they need to install it first. This module is well documented: http://networkx.lanl.gov/index.html
I just remember a very practical module: copy.
I use the deepcopy() from it quite often.
It is hard to live without timeit
>>> # Python shell usage
... import timeit
>>> tt = timeit.Timer("foo = 'time this'", "print 'setup with this arg'")
>>> tt.timeit(number=1000)
setup with this arg
0.00021100044250488281
>>>
[mpenning@Bucksnort ~]$ # Bash shell usage
[mpenning@Bucksnort ~]$ # 5 runs with 1000 samples each.
[mpenning@Bucksnort ~]$ python -m timeit -n 1000 -r 5 -s "print 'setup w/ this arg'"
"foo = 'time this'"
setup w/ this arg
setup w/ this arg
setup w/ this arg
setup w/ this arg
setup w/ this arg
1000 loops, best of 5: 0.173 usec per loop
[mpenning@Bucksnort ~]$