PyCon 2008 Notes

I just completed writing up my notes on PyCon 2008 for my employer (who paid my way), and thought I should also share them here. I apologize for using one long blog post; I decided the ability to refer to a single post outweighed the advantages of several smaller ones. (And you'll have to ignore the "TODOs" I've sprinkled in the text.)

I attended PyCon from March 13 through 20, 2008.

Tutorials

Thursday April 13 was a day of tutorials. I attended three (of a maximum of three).

Secrets of the Framework Creators Tutorial

The first tutorial I attended was "Secrets of the Framework Creators", presented by Feihong Hsu and Kumar McMillan. This was an excellent tutorial that opened my my to some much more sophisticated ways of using Python. (See the three previous posts I've written on this tutorial so far.)

You'll find the materials they prepared for the tutorial in the Google Group they created. They covered four topics:

• Frame Hacks
• Decorators
• Metaclasses
• Magic Methods

(Of these four, frame hacks and metaclasses were completely new to me. I already had some experience with decorators and magic methods.)

It would be a waste of my time to summarize any of the material presented, since the tutorial materials do such an excellent job introducing these subjects. If you want a quick overview, you can probably read through them in an hour or less. And if you really want to learn to use these techniques, you can spend a few hours on the exercises they've provided.

Here are the few brief notes I took during the tutorial:

• TODO - Learn about properties and descriptors
• TODO - Look into Trellis
• TODO - Look at functools
• TODO - Look at the inspect module
• The "automatic debugger methods" recipe in the metaclasses section is way cool

Generator Tricks for Systems Programmers

The second tutorial I attended was "Generator Tricks for Systems Programmers", presented by David Beazley. David has also made his tutorial materials (including excellent slides and plenty of code samples) publicly available: http://www.dabeaz.com/generators/.

At the start of the tutorial I had written iterators, and had a vague understanding of generators. After the tutorial, I now feel like my understanding of generators is much deeper. I very highly recommend you take the time to read through the slides (probably one to two hours) and look through his code samples.

TODO - add a summary here

My notes:

• I didn't know that "Conditional Expressions" (aka "the Ternary Operator") was added to Python in 2.5. See an example at the bottom of slide 64.
• There's a nifty trick in using the max function under the first bullet of slide 71. TODO - explain.
• I wrote on slide 135 that the find_404() function could be modified to take a receiver arg and call it instead of the print (and the print could be wrapped in a consumer function). TODO - explain
• TODO - I also gave myself a TODO on slide 135 to compare this to the previous use(s) of broadcast
• TODO - Someone mentioned the "etree" (I think they meant ElementTree), which either uses generators for parsing XML or can be used with generators.

Django Code Lab

The last tutorial I attended was the Django Code Lab. The presenters, Adrian Holovaty (one of the creators and core developers of Django), Jacob Kaplan-Moss (a core developer of Django) and James Bennett (release manager for Django) asked people to send in questions and code samples, and then reviewed them in the lab. (I did submit some code developed at the last minute--my main question was how best to integrate models that don't use the database with others that do--but mine was not chosen.)

The slides are posted at http://toys.jacobian.org/presentations/2008/pycon/codelab/. They may not be too useful out of context. But I refer to them in my notes (below):

The first up was Pim Van Heuven:

• he had a huge urls.py file - see the slides
• they also presented "Forms with extra parameters"

Next was Justin Lilly:

• Jacob ranted about TDD
• then he presented some useful information about testing Django
  • look at the model examples in the Django documentation, they're all working unit tests
  • see slides for example of using django.test.TestCase

Next was Richard House:

• he had a long list of BooleanFields in his model
  • they pointed out that one should use NullBooleanField rather than BooleanField(null=True)
  • the slides present a pattern for using "two pairs of models"
  • Adrian pointed out this is called EAV (Entity-Attribute-Value) and noted this has problems with searching and queries

Next, Peter Herndon:

• he suffered from slow queries (see the slides)
• someone recommended a Malcolm Tredinnick blog post (I think this one); though his blog is definitely recommended reading for anyone using Django
• see the optimization notes for your database - Jacob says the PostgreSQL optimization notes are especially good
• James recommends especially django-sphinx for searching (David Cramer used it for a very busy site with a single database server)
• Jacob mentioned Google Co-op several times

Next, J. Clifford (Cliff) Dyer:

• Handling previous/next links
• I didn't fully understand the discussion, and there were no code examples provided unfortunately (and no slides)

Next, Dave Lowe:

• the subject was when not to use the admin
  • the advice was that the admin should only be used by someone trusted with access to the entire database (even after the newforms-admin branch is merged back into the trunk)
• Adrian said that the key to mastering Django as a tool is to learn (through experience) when to use different parts of Django and when not to

Next, Bob Haugen:

• prepping for deployment
• make sure that django.views.static.serve is blocked out inside settings.DEBUG
• see the HOME trick with os.path.join() for TEMPLATES_DIR
• see the urlresolvers.reverse slide
• worth looking at mod_wsgi first (before mod_python)
• Jacob admitted (reluctantly) that MySQL is easier to setup than PostgreSQL
• see the slide 97, which summarizes "Develop" vs. "Deploy"
  • Jacob forgot to include creating 500.html & 404.html pages

Next, Wiley Kestner:

• see the slides for details on the Django dispatcher

Then they ran out of time, but the slides also contain some details on REST APIs,

Slides for other tutorials

I got the URL for the slides from the Introduction to SQLAlchemy tutorial from IRC: http://utahpython.org/jellis/. The slides are in "sa-intro.pdf". (But it looks like there's some other interesting stuff there too.)

Conference Sessions

Friday, April 14 through Sunday, April 16 were the Conference Days. Each day contained keynote talks, scheduled talks, lightning talks, and "open space". (See the previous link for details.)

I'll go into detail on the scheduled talks (from all three days) first, and then dump all my notes from the lightning talks.

The A/V team as begun posting recordings from PyCon 2008 to their YouTube channel: see this blog post. As of 2008-03-28 they've got 12 up, some from regular sessions and some from lightning talks. None are from tutorials--I don't know if they intend to post tutorial videos.

Guido van Rossum - "Python 3000 and You"

The first technical talk of the conference was fittingly Guido's talk on Python 3000. Guido has posted the slides and some advice when porting to Python 3.0 on his blog.

I didn't take any notes, but if you're interested in Python 3000, PEP 3000 is work reading. (In addition to the slides.)

Brett Cannon - How Import Works

This was useful information. You'll find the slides in http://us.pycon.org/2008/conference/schedule/event/12/.

See also http://www.python.org/dev/peps/pep-0328/ and http://www.python.org/dev/peps/pep-0302/.

Dr. Tim Couper - Python references and practical solutions to reference-related problems

This was an excellent talk. The video isn't up yet, but I hope it is soon. (Unfortunately no slides are attached to http://us.pycon.org/2008/conference/schedule/event/16/.)

Here are my (sketchy) notes:

• see the getrefcount() function
• weakref.ref()
  • can get the actual reference from a week reference, with "()"
• getweakrefcount()
• getweakref()
• garbage collector will clean up objects not deleted because of a circular reference
• can turn off GC if you're sure you'll have no circular references
• GC will not clean up objects with a __del__ magic method
  • but you can get a list of these objects
• then I got distracted and got lost when he talked about pickling and weak references as a solution for this?
  • something interesting about a pattern using __getstate__ and __setstate__ methods
  • also talked about finalizers

Noah Gift - Using Optparse, Subprocess, and Doctest To Make Agile Unix Utilities

The presentation and code samples are at http://code.noahgift.com/pycon2008/.

TODO - I need to take a look at the subprocess module.

Kevin Dangoor - Rich UI Webapps with TurboGears 2 and Dojo

This was a very slick, fast-moving presentation with excellent examples on how to use Dojo. There was also an example of (bleeding edge, I think) use of Comet to push data from the server to the client.

I'll want to see this one again when it's available on video. I haven't been able to find the slides, but Kevin promises to post a screencast version in this blog post.

Adrian Holovaty - The State of Django

I haven't been able to find the slides, and the video isn't up yet. But I recommend it (if you're interested in Django).

Here are my notes:

• What's new in the last year:
  • 0.96 - released March 23, 2007!
  • Unicode branch - is it still in a separate branch? (I think it was merged to trunk)
  • auto-escaping in templates
  • GeoDjango - still a branch
    • "hope to get it integrated into the trunk soon"
  • Sprints in Sept. & Nov.
  • 2432 checkins
• Community stuff:
  • http://www.djangosites.org/
  • http://www.djangosnippets.org/
  • http://djangogigs.com/
  • http://djangopeople.net/
• What's coming
  • mostly mature/stable
  • queryset-refactor to be merged to trunk
    • includes support for model subclassing
  • select_related - can now specify args
  • newforms-admin
    • admin options defined in separate class; register
    • has_change_position() method in ModelAdmin classes
    • can regulate which objects show up in the admin
    • can have multiple admin sites on the same website
  • want to add more sophisticated INSTALLED_APPS (using classes)
    • can define database prefixes & labels (?)
  • model validation
  • 1.0!

Marty Alchin - Django: Under the Hood

Plenty of useful information that I plan to review again (more slowly). His slides are at http://www.slideshare.net/Gulopine/django-under-the-hood/.

Some notes:

• the quick review of how metaclasses are used was useful as used a metaclass in the code I wrote during the Django sprint
• the signals discussion may be useful to users of Django too
• Utilities
  • functional utilties
    • several, including turn middleware into per-view decorators
  • text utilities
    • various
• "remember, Django is Python, just read the source"
  • but beware of query.py and related.py

See this Adam Gomaa blog post for links to the source files to look at for model metaclasses, the "signalling stuff" and "some neat functional utilities".

Chris McDonough & Mike Naberezny - Supervisor as a Platform

This looks like a very useful tool. I plan on using it very soon. The slides are in http://supervisord.org/wp-content/uploads/2008/03/ and are very good--I don't have anything to add. (Except I read a rumour somewhere that Guido wrote the original version of this--details are in http://supervisord.org/contributors/.)

Mike Bayer - SQLAlchemy 0.4 and Beyond

I missed this because it was at the same time as the Supervisor session. But I heard this was very good. You'll find the slides at http://techspot.zzzeek.org/?p=22.

Rodney Drenth - Decorated State Machines

I missed this one too (it was at the same time as the above two sessions). But I see the slides are at http://us.pycon.org/2008/conference/schedule/event/43/. TODO - I haven't looked at them yet.

Maciej Fijalkowski - The State of PyPy

This was interesting. I learned a bit about PyPy (a subject I know very little about). I can't find the slides, and the video isn't up yet.

My notes are very brief:

• Currently:
  • quite compliant 2.4/2.5
  • still missing some standard libs
  • new: ctypes!

I vaguely recall an interesting demo.

James Bennett - Developing Reusable Django Applications

This was a very good talk that would be worth reviewing by anyone using Django. The slides are in http://us.pycon.org/2008/conference/schedule/event/50/.

I'll repeat some of the slides that I noted:

• Four big ideas:
  • do one thing, and do it well (the UNIX philosophy)
  • don't be afraid of multiple apps
  • write for flexibility
  • build to distribute
• He wrote django-registration
  • people asked for profiles (in django-registration)--he said no
  • but then he wrote django-profiles (as a separate app)
• The Django mindset
  • application == some bit of functionality
  • site == several applications
  • tend to spin off new applications liberally
• What reusable apps look like
  • single module directly on Python path
  • related modules under a package
  • no project cruft whatsoever
• Good examples:
  • django-tagging (Jonathan Buchanan, http://code.google.com/p/django-tagging/)
  • django-atompub (James Tauber, http://code.google.com/p/django-atompub/)
• More information:
  • django-hotclub (http://groups.google.com/group/django-hotclub/)
  • Jannis Leidel’s django-packages (http://code.google.com/p/django-reusableapps/)

Alex Martelli - Don't call us, we'll call you: callback patterns and idioms in Python

This was at the same time as the Developing Reusable Django Applications talk, so I missed it. But the slides are in http://www.aleax.it/pyc08_cback.pdf. TODO - I haven't looked at them yet, but I've read good things about this talk.

Brandon Rhodes - Using Grok to Walk Like a Duck

You'll find the slides and example source code at http://rhodesmill.org/brandon/adapters/. (And this one is on YouTube. The title is misleading--this was really primarily about the Adapter pattern and the Zope Interface class. I left wishing he provided some examples of using Grok.

I got a little lost when he gets into zope.interface.Interface. TODO - The best way to understand this would be a play around with it.

Steven Wilcox - The Power of Django Admin (Even For Non-Django Projects)

The "slides" (actually a paper) are at http://devpicayune.com/pycon2008/django_admin.html. This doesn't look familiar, so I must have been busy with something else at the time--though I can't recall what. Perhaps that presentation wasn't as compelling as the material deserved. TODO - This does appear to be worth reading.

Brian Dorsey, Maciej Fijalkowski - py.test: Towards Interactive, Distributed and Rapid Testing

This was a very good presentation that I would recommend watching when it's available on YouTube.

See http://codespeak.net/py/dist/test.html

Here are my notes:

• tests for py.test are almost identical to nose
• tab completion in bash after -k flag slick
• I missed the test reporting page thing, but heard it was slick - and I believe that's one thing that distinguishes py.test from nose
• 1.0 soon
  • more plug-in architecture
• yes, there is a tool for converting unittest tests to py.test tests
  • you can convert tests, or just run them without (permanently) converting them
• what does py.test do that nose doesn't?
  • the introspection magic
• py.test.raises(Exception, stuff, arg1, arg2)
• TODO check out py.execnet (http://codespeak.net/py/dist/execnet.html - "A new view on distributed execution")

Jim Baker - More Iterators in Action

My notes say the slides are at http://zyasoft.com/, but that page strangely describes Zyasoft consulting services and software development but provides no links or contact information. The slides are attached to http://us.pycon.org/2008/conference/schedule/event/75/ though.

Here are my notes:

• LINQ - Language INtegrated Query - http://en.wikipedia.org/wiki/Language_Integrated_Query
• "The solution to integration in Python is naming (more or less)." [I may have got that wrong.]
• collections.namedtuple - new in 2.6
• see also cookbook recipe NamedTuple? - http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303439
• TODO look up tee() - http://docs.python.org/lib/itertools-functions.html
• operator.itemgetter() - http://docs.python.org/lib/module-operator.html
• defaultdict - http://docs.python.org/lib/defaultdict-objects.html
• Recursive Generators
  • are they practical? sometimes, but they're fun
  • see cookbook recipe 190465 - http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/190465

Raymond Hettinger - Core Python Containers - Under the Hood

An excellent presentation that I will most definitely watch again when it's available on YouTube. I haven't been able to find the slides.

I didn't take very good notes:

• python/trunk/Include/listobject.h
• python/trunk/Objects/listobject.c
• collections.deque() is optimized for appends and pops at both ends, but it is slower for indexed access

Collin Winter - 2to3: Translating Python 2 to Python 3

I haven't been able to find the slides.

My notes are very brief:

• collinw@gmail.com
• workflow:
  • maintain in 2.x
  • fix 2.6 -3 warnings
  • run 2to3
  • test suite!
  • release 3.x version

Lightning Talks

There were three Lightning Talk sessions, after the scheduled talks on Friday, Saturday, and Sunday. Here are my notes. (I marked my favorites with *.)

Friday (April 14)

• *How to write a daemon in Python
• Trestle - a nose plugin for writing testable documentation for web apps (or shell commands)
• *Noonhat - "Changing the world through lunch"
• *Seattle Saturday House
• Thrift lightweight RPC mechanism
• *tinypy

Saturday (April 15)

• Bazaar - "if you can run Python 2.4, then you can run Bazaar"

Sunday (April 16)

• Wesley Chun - http://www.nearbynow.com/
  • Text a "mall code" to 632729 (NEARBY) to search products, brands and sales
• Zed Shaw
• *SNPedia - "SNP" is pronounced "SNIP"
• *python4ply - a Python parser for the Python language
• Jython - 2 use cases:
  • script Java libraries from Python
  • "maintain huge collapsing J2EE app"
• bug.gd - "...submit an error message you've seen and we'll try to find other people who've seen the same error..."
  • see also error_help() for Python hackers
• *geopy - a geocoding toolbox for Python
• from LJWorld.com:
  • Crimes reported in KU residences
  • Foreclosures: Sheriff's Sales
• Elixir - a declarative layer on top of the SQLAlchemy library

UPDATE (2008-04-03)

As I read through blog posts on PyCon, I'll add links here.

To start, I recommend -JJ's posts on:

• SQLAlchemy 0.4 and Beyond
• Managing Complexity (and Testing)
• Consuming HTML
• Core Python Containers--Under the Hood
  

Did I miss any? (You just may want to scan all his PyCon 2008 posts.)

UPDATE (2008-04-11)

Yesterday evening I presented my "PyCon 2008 Notes" to BayPIGgies, in slide form. The slides don't have any of the links above, and I stripped out much of my notes, but if you really want them you'll find them at http://www.spitzer.us/daryl/baypiggies/pycon_2008_notes/ (exported to HTML from Keynote) or http://www.spitzer.us/daryl/baypiggies/pycon_2008_notes/pycon_2008_notes.pdf.