Tuesday, March 17, 2009

Python cheat sheets

I'm out of hibernation, time I posted something.

As much as I use Python these days, there's a few things I find myself looking up regularly. At one point I just made a small crib sheet and stuck it to my monitor. I have examples on it for list comprehensions, filter, and map.

List Comprehensions
These are useful for creating modified lists from existing data without a lot of fuss. They aren't all that hard to remember, but the syntax was a bit alien to me for awhile. They're basically an expression followed by a for clause.

The below example takes an existing list, my_list and builds a new list with only the elements that are greater than 2. In this case the result is assigned right back to my_list.

my_list = [x for x in my_list if x > 2]
Filter
Using filter is a powerful way to remove undesired elements from a list. You pass a function as the first argument, which generally returns True/False based on some criteria. The second argument is the sequence to be filtered (or any iterable object). Only the elements that return True when passed to that function will remain in the newly returned list.

Filtering is often done with a lambda as the first function argument. A lambda is a one-off function that's defined and used in the same place. Since it's only used once, it doesn't need a name. It's so common to see filter and lambda together, the fact they were seperate didn't occur to me when I was learning the language.

In this example, we have a list of filenames, my_files, and we want to remove any that aren't Python scripts, ending in '.py'.
my_files = filter(lambda f: f.endswith('.py'), my_files)
That is the shorter equivalent of:
def is_py_filename(filename):
   return filename.endswith('.py')

my_files = filter(is_py_filename, my_files)
With filter, passing None as the first argument instead of a function automatically removes any elements that don't evaluate to True. That includes integers or floats that are zero, as well as occurrences of False or None.

my_files = filter(None, my_files)
Map
Mapped functions let you apply a function to every element in a sequence.
def add_ten(x):
   return x+10

result = map(add_ten, [1,2,3,4,5])
The value of result would be [11, 12, 13, 14, 15]. Of course you could also use a lambda here, too:
result = map(lambda x: x+10, [1,2,3,4,5])
So what's on your cheat sheet?

5 comments:

Alexander Artemenko said...

I've read somewhere, that 'map', 'filter' have more performance benefits, when used with builtins, not with lambda. In all cases, where you want to use lambda, it is better to replace 'map' or 'filter' with list comprehentions.

By the way, you miss the list comprehension's syntax which results with an iterator instead of list. It is useful to use such iterators in more comprex expressions.

rayo said...

You can write both without a lambda construct:

import operator
import functools

my_files = filter(operator.methodcaller('endswith', '.py'), my_files)

result = map(functools.partial(operator.add,10), [1,2,3,4,5])

For operator.methodcaller you will need python 2.6.

James Stevenson said...

More often then not, you can use a list comprehension in place of filter:

my_files = filter(lambda f: f.endswith('.py'), my_files)

...becomes...

my_files = [file for file in my_files if file.endswith('.py')]

Marius Gedminas said...

My cheatsheet contains an example of optparse usage.

Everything else I either remember, can guess, or use rarely enough so that going to a browser and looking up the documentation isn't painful.

The "can guess" part is interesting: it's an awesome feeling when you try out something that ought to work (at least, it would, if you were the one designing the API), and you discover that it does in fact work the way you'd expect it to. It means that the language/library designers' minds work in the same way mine does, at least once I've calibrated mine by reading/writing Python code and following Python idioms. I call that "Pythonic".

(Back to optparse: I dearly wish 'pydoc optparse' showed an example. I should file a bug/send a patch, I guess...)

matt harrison said...

I've written various python cheatsheets, one for python and excel (xlrd), another for doctest, one that I haven't published for heappy/guppy, an intro to python one, a couple for scripting (a shorter one page example of a script, and a 4 page newer best practices one).

I tend to keep my brain uncluttered, but like to have dead tree notes. (I memorize the position of where the notes are on the page instead of memorizing the notes).

Here's a link containing links to all handouts

cheers