Thursday, July 10, 2008

Newbie Nugget: Introduction to the with statement

I’ve signed up to present the first BayPIGgies “Newbie Nugget” at tonight’s meeting, on the with statement. I know from past experience that the best way to learn something is to explain it—and one has to learn it very well to explain it to a large group of smart people.

What follows are the notes I compiled as I taught myself about the with statement. I’ll turn these into slides for tonight. Since I only have 10 minutes for the presentation, I’ll keep these notes brief. (Think of this as an introduction to the with statement.) You’ll find links to further reading below.

We’re all used to (or should be used to) writing code that looks like this:

f = open(filename)

The with statement (introduced in Python 2.5) simplifies that to:

from __future__ import with_statement
with open(filename) as f:

(“with” and “as” are keywords in Python 2.6, and “from __future__ import…” is not needed.)

The specified syntax for the with statement is:

with EXPR as VAR:

The “as VAR” part is optional.

The EXPR must result in a context manager. Briefly, a context manager is an object with a __context__() method, which returns a context object. A context object has an __enter__() method, which is invoked as soon as a context object is returned. Its return value is assigned to VAR (if there is one). When the with statement terminates, the context object’s __exit__() method is called. (I won’t take the time to go into the arguments to __exit__().)

The contextlib module provides “utilities for common tasks involving the with statement.”

The contextlib.closing function returns a context manager that closes its argument. For example:

from __future__ import with_statement
from contextlib import closing
import urllib

with closing(urllib.urlopen('')) as page:
for line in page:
print line

And the contextlib.contextmanager function “is a decorator that can be used to define a factory function for with statement context managers, without needing to create a class or separate enter() and exit() methods.” For example:

from __future__ import with_statement
from contextlib import contextmanager

def opened(filename):
f = open(filename)
yield f

with opened(filename) as f:

There is plenty more detail in PEP 343, including plenty of fun examples. My favorites are:

  • (2) The opened() context manager used above
  • (3) transaction() context manager for committing or rolling back a database transaction
  • (5) stdout_redirected() context manager for temporarily redirecting stdout
  • (7) A context manager for blocking signals. (This one is actually left as an exercise to the reader.)

To learn about the with statement (before reading PEP 343 and the contextlib module documentation), I first turned to section 10.4 (Context Management) of Core Python Programming (2nd Edition) by Wesley J. Chun. It provided an excellent introduction.


Nuestra Viaje a Guatemala said...

I went to the meeting tonight but got there too late for your presentation. Is this on video? If yes, where is the video located?

nikhil said...

is this somewhat like Ruby's blocks?

And could you list some of the advantages of this other than the reduced typing. Is the with block automatically handling any errors or restricting the scope of the open()