Showing posts with label baypiggies. Show all posts
Showing posts with label baypiggies. Show all posts

Thursday, January 8, 2009

Book Review: Python Web Development with Django


Back in April 2008 I eagerly volunteered to review a “rough cut” of Python Web Development with Django by Jeff Forcier, Paul Bissex, and Wesley Chun when I read of the opportunity on the BayPIGgies mailing list. I first completed a first version of this review in April (based on a version of the rough cut updated March 11, 2008). But I was asked to hold my review until it was updated again—I updated the review using the next two updates, but didn’t complete it. (I’ll post the whole sordid story is on yacitus.com.) I’m ashamed to say the book was published before I finished my review. (But I did post several comments on the rough cut—under the name “yacitus”—so I’m happy that I contributed to the final outcome in a small way.) I’m pleased to finally finish my review, about 10 months after I first read the “rough cut”. It’s fairly long, so you may want to skip to my recommendation in the third-to-last paragraph (before the footnotes).

You’ll find that my coverage of some chapters is extremely detailed, while I only quickly mention others. The deeper coverage was written back when I was reviewing the rough cut, and I just don’t have time right now to go that deep on the material added to the book since then. I plan to (someday) use this book to help me convert www.spitzer.us to a “real” blog, and I will update this review as I read other chapters more carefully then.

While I’ve been using Python for about 4 years (full-time for about half that), I’m quite new to Django. I’ve read The Definitive Guide to Django (which I’ll refer to as TheDjangoBook below) and I won’t be able to resist comparing it to Python Web Development with Django (which I’ll refer to as PyWDwD below).

The first thing I want to do before I start reading (or before I buy) a book is to get a sense of the intended audience. If I’m in a bookstore, I’ll start by reading the back. When browsing online (which for me almost always means Amazon.com), I’ll look at the “Editorial Reviews”. It doesn’t really contain a description of the intended audience (and neither does the back of the book). But back in April there was a clue in the overview on the book’s public Safari page: “This book is designed to help you learn and use Django (and Python, if necessary)…” (though that text is no longer there). So I assumed that the book would contain an optional introduction to Python, but would mostly concentrate on Django.

The second thing I look at is the table of contents. (Which you’ll find the the Amazon.com Editorial Reviews section.) The book is divided into five parts: “Getting Started”, “Django in Depth”, “Django Applications by Example”, “Advanced Django Techniques and Features”, and the appendices.

I’m not going to spend too much time reviewing Chapter 1: “Practical Python for Django”. I have limited experience with Django (which is why I was so eager to read even this incomplete version of this book), but I am quite comfortable with Python. But I did skim through the chapter and tried to keep an eye out for any Python features that are used in Django that they don’t cover. I didn’t find any. I was surprised to find concise introductions to some of the more complex Python features like generators and decorators. After reading the chapter over, I would guess that it would be useful for people with a fair bit of programming experience who are new to Python, but I think it would be confusing for anyone completely new to programming.

The beginning of the chapter states that they intend for it to be more than just a high-level introduction. They explain Python’s “object model, memory management, and philosophies as well as giving a good number of samples and sidebars which directly relate to Django development”. So I also made note of any content that I found valuable. (1)

The next chapter is “Django for the Impatient”. As an introduction to Django, they walk the reader through creating a simple blog project. I’m familiar enough with Django that I skimmed this very quickly, but I expect it would be quite useful to a Django newbie. It was worth the skim: defining ordering in a model (using the ordering attribute of the Meta inner class) was new to me.

In the next chapter (#3), titled “Starting Out”, they take a step back and provide a “tool-agnostic” view of the Web, provide high-level explanations of Django models, views and templates and explain the general philosophy of the creators of Django. They warn the reader not to skip ahead (saying “…even intermediate and experienced Web developers can benefit from taking a step back and reviewing the fundamentals…”), but I found I could have completely skipped the “Dynamic website basics” section. I also could have skipped the “Understanding Models, Views and Templates” section and the explanation of Django’s spin on MVC (2), but I do feel that both would be useful to a Django newbie. The “Core philosophies of Django” section was interesting, but I didn’t learn anything new. (It may have reinforced what I already know, however.)

Part II “Django in Depth” starts with Chapter 4: “Defining and Using Models”. There are (of course) two sections. The “Defining Models” section starts with an explanation of why to use an ORM. Then they describe the different types of fields, and explain primary keys (and they describe the “unique=True” argument, which is new to me). Then they go into detail on foreign keys (many-to-one relationships), many-to-many relationships (both “simple” and “complex”) and a brief explanation of composition with one-to-one relationships (but no examples). That’s followed a brief description of constraining relationships with a “limit_choices_to” argument. That’s new to me, and I don’t see a use case for it, so I’d like to see a real example (rather than their contrived example of a Book model class that will only relate to authors whose name ends in Smith). Next is a detailed explanation of (the new Django feature) model inheritance, where they explain the two different approaches: abstract base classes and multi-table inheritance. They then describe the Meta inner class with a reasonable amount of detail and conclude with admin registration. They avoid a detailed explanation of the admin options, which is in keeping with this book’s intention to be more of a tutorial than a reference.

The “Using Models” section starts with an explanation of syncdb. They’re careful to explain right away that in spite of its name, syncdb will only create database tables to match models—it doesn’t do any sort of synchronization. I don’t recall TheDjangoBook explicitly stating this so clearly, but I also didn’t learn this the hard way. I think it became clear as I read through the examples. They provide a table of the manage.py functions, which I don’t recall seeing in TheDjangoBook, and if I read about the “sql*” functions and “loaddata” and “dumpdata” (in The DjangoBook) then I’ve forgotten.

The meat of the section is in the section on query syntax. I also don’t recall TheDjangoBook explaining so clearly what a Manager is. I’ve heard the term, but until I read it in PyWDwD I didn’t realize a Manager object is returned from a model class’s objects attribute, and its methods (all, filter, exclude and get) always return QuerySets. The explanation in the (next) “QuerySet as a building block” section is also new (to me) and quite lucid. They explain how QuerySet “…is lazy: it will only execute a database query when it absolutely has to…” and explain how this allows them to be composed into complex queries. There’s a lot of detail here, including tweaking the SQL with the QuerySet method extra. Then the chapter concludes with an explanation of how to use SQL features that Django doesn’t provide.

Chapter 5 is entitled “URLs, HTTP Mechanisms and Views”. The URLs section contains a very detailed explanation of URLconfs. Then follows the “Modeling HTTP: Requests, Responses and Middleware” section, which starts a description of request objects, within which their explanation of GET and POST is helpful. And then the section gets even meatier. Their explanation of cookies and sessions is (IIRC) not covered in TheDjangoBook. The section concludes with a brief description of response objects and middleware.

Finally the “Views / Logic” section explains that views are just Python functions, that must take an HttpRequest object and return an HttpResponse object (both of which were explained previously in the “Client/Server - HTTP” section). And then they jump right in to explaining generic views. (3) Unfortunately they don’t (yet?) provide any examples, so I imagine their descriptions of the most common generic views will go over the head of Django newbies. They state that the most common use of “semi-generic” views (calling a generic view from a custom view) is “…to work around an inherent limitation in the URLconf itself: you can’t perform logic with the captured URL parameters until the regular expression has been parsed.” But they only provide one short example with no explanation of what it does. Finally they very briefly explain custom views, and describe rendertoresponse() as replacing “…the two- or three-step process of creating a Context object, rendering a Template with it, and then returning an HttpResponse containing the result.”, but I don’t believe they have yet explained what a context object is and only very briefly described templates.

The final chapter in Part II (#6) is “Templates and Form Processing”, containing (of course) two sections. The “Templates” section is more of an overview of Django templates than an in-depth description. But I did find it clear and easy to follow. I didn’t learn anything new, but I already have a reasonable amount of experience using Django templates. There is plenty of detail in the “Forms” section, and more examples than many of the previous chapters. I enjoyed reading this chapter—it’s a big improvement over the limited coverage of forms in TheDjangoBook.

Part III, “Django Applications by Example” contains four chapters, each dedicated to a different example application. Chapter 7—“Photo Gallery”—presents an example application using Django’s image upload field and a custom ImageField subclass that automatically generates thumbnails. Chapter 8—“Content Management System”—defines “CMS” and describes the “Un-CMS” Flatpages App and presents a simple custom CMS. Chapter 9—“Liveblog”—walks through the creation of a blog application, including Ajax integration. The introduction to the chapter states it “goes over everything you need to know to integrate Ajax with a Django Web application without going too deep into the specifics of complex client-server interaction or animation.” (I’m particularly looking forward to reading this chapter carefully.)

Chapter 10—“Pastebin”—is (I think) last in Part III because it is a lesson in using generic views. They write: “…the essence of this example is seeing how much work we can hand off to the framework. Some might call this approach lazy, but every line of code you don’t write is one you don’t have to debug.” They show and explain the model, and the templates are self-explanatory. The explanation for the URLs is detailed. And then we’re ready to try it out—they walk us through the “add one” form, the newly created paste, the list of submitted items and the admin screen. Then they show us how to limit the number of recent pastes displayed, how to add syntax highlighting using the SyntaxHighlighter JavaScript library, and how to write a cron job that periodically deletes old items.

Part IV—“Advanced Django Techniques and Features” contains two chapters. Chapter 11—“Advanced Django Programming”—has plenty of meat. There are sections on customizing the admin, generating RSS or Atom feeds, generating downloadable files (including examples of a vCard, CSV, and a chart using PyCha), enhancing the Django ORM with custom managers, and plenty of detail on extending the template system (by creating custom template tags, inclusion tags and custom filters) and using other template engines (Mako in this case). Their explanation of inclusion tags is detailed and walks through a useful example of how to create a template tag to display a calendar grid.

Chapter 12—“Advanced Django Development”—also has plenty of satisfying detail. There’s a section on writing utility scripts using Django, with a couple examples: one that can be run using cron to delete old records, and a script to import email from an mbox file to a database using a Django model. They provide plenty of good advice and background on caching. I’m sure it’s just a sign of how little I know about Apache, but I had never heard of “ab”, the Apache Bench tool. I applaud the authors for taking the time to describe its use at the beginning of their caching session, and using it to show measured performance improvements. A section on testing covers doctest & unittest, testing models, testing your entire web app, and testing the Django codebase itself. And there’s a couple paragraphs on customizing the Django sources, where they (rightly) discourage the reader from doing so unless it’s worth contributing the changes back to the Django project.

There are 6 appendices. Appendix A is titled “Command Line Basics”. It is a very simple introduction to using a command-line environment on Linux or Unix. They cover common commands, options and arguments, pipes and redirection, environment variables, and the PATH. I think this would be quite useful to someone with no command-line experience (perhaps with a strictly Windows background).

Appendix B covers “Installing and Running Django”. Including installing Python (and brief mentions of Easy Install and IPython), installing Django itself, choosing and configuring a web server (they cover Apache & mod_python, WSGI, and FastCGI with flup), and choosing and configuring an SQL database (they covert SQLite, PostgreSQL, and MySQL, with a quick mention of Oracle and Microsoft SQL Server and IBM DB2).

Appendix C is titled “Tools for Practical Django Development”. The first section is on version control. They discuss the fundamentals of branches and merging, describe subversion and the Mercurial and Git DVCSs, and then walk through using version control on a Django project (using Mercurial). There’s a brief section on project management software, with a description of Trac. And a brief section on text editors with descriptions of Emacs, Vim, TextMate and Eclipse.

Appendix D is a quick 3 pages on “Finding, Evaluating, and Using Django Applications”, with brief sections on where to look for applications, how to evaluate them, how to use them, and sharing your own applications.

Appendix E covers “Django on the Google App Engine”. They focus on porting an existing Django app to App Engine, and creating a new Django app written specifically for App Engine. While not exhaustive, the detail on porting looks useful. (I hope to come back to this in more detail later.) But the intention of the appendix is to give an overview; they conclude with a list of online resources for more detail.

The final appendix F (and conclusion of the book) is two pages on “Getting Involved in the Django Project”. They describe ways to contribute that don’t require any programming, ways to contribute involving code that “still don’t require Herculean effort”, and offer ideas for contributions that would “have a significant impact on the Django community”. The conclude by pointing the reader to the two Django mailing lists, IRC and some Django community web sites. It feels like an appropriate ending to the book.

In conclusion, I recommend Python Web Development with Django to anyone considering (or just starting) using Django to build a web site. I recommend it over TheDjangoBook, not just because PyWDwD is more up-to-date, but also because I think it does a better job at explaining all the new concepts required when using a web framework. My main concern when reading the rough cut version was that there were not enough examples. But that’s been almost entirely rectified in the published book. (And the book should empower the reader to go out an find more open source examples as needed.)

The only other thing I noticed that was missing was examples of writing Django applications (Django terminology for modules built to be reusable, if possible). But a book can’t be everything to everyone, and that’s a niche that James Bennett’s book Practical Django Projects looks like it will fill nicely. (I hope to review it eventually.)

One other Django book I hope to find time to read is Marty Alchin’s Pro Django. It looks like it will pick up where these other two leave off.


Footnotes:

(1) I made note of the following tidbits along the way:
  • I didn’t know about the enumerate() built-in function. I’m sure there have been times where I could have used that to simplify my code. They have an explanation of a handy way to use enumerate in Django models that you’ll also find used in this Paul Bissex blog post.
  • I also wasn’t aware of the sorted() built-in function, which would also simplify my code at times in place of the list sort() method (which doesn’t have a return value).
  • Their explanation of “tuple-related gotchas” will be quite helpful to Python newbies.
  • I wasn’t aware of “from . import X” and “from .. import Y”. I can think of one specific place in my code where “from ..” will help.
  • Their entire “Common Gotchas” section is excellent reading.
(2) See http://docs.djangoproject.com/en/dev/faq/general/#django-appears-to-be-a-mvc-framework-but-you-call-the-controller-the-view-and-the-view-the-template-how-come-you-don-t-use-the-standard-names

(3) This is the opposite approach to TheDjangoBook, which (IIRC) gives many examples of views and saves generic views for a separate chapter. I don’t think one approach is demonstrably better than the other, but I found it easier to understand generic views this time. But that could be because I read PyWDwD after already reading about generic views in TheDjangoBook.

Saturday, October 18, 2008

Newbie Nugget: Unit Testing with Mock

I presented my second “Newbie Nugget” at the October BayPIGgies meeting. Since I’ve been working with Mock quite a bit lately, I chose to present on that. (I also thought I could present on decorators, but I decided Mock would be easier since I’d have to do the least to prepare. I remember feeling confident on my knowledge of Mock itself, but I suspected there was still a lot I had to learn on unit testing. My suspicions proved correct.)

I didn’t make as much time to prepare as I would have liked. I ended up with 18 slides, and didn’t have time to trim them down. So I had to breeze through them very quickly. (In theory I only had 5 minutes. I don’t know how long I actually took—probably more like 10—it turned out there wasn’t any pressure this time to keep it brief.) I also would have liked to have time to come up with some real-world examples, but in my rush to finish the slides I ended up taking the code examples from Michael Foord’s documentation, except for the last two examples which I pulled out of my own code. I’ve uploaded the Keynote slides exported to PDF and to HTML. To save some of you some time, the references I had in my final slide are:

Note that I prepared this presentation based upon Mock version 0.3.1. Michael has since released version 0.4.0. TODO: I’d like to describe the details on what’s new in 0.4.0—but in the meantime, see the notes in the documentation or Michael’s blog post.

After the main presentation on PyGameSF, there was some extra time left and Jim Stockford asked if I’d be willing to take questions. I’m glad he asked, and I’m glad I did so. (How could I not?) I probably will not recall all the questions. Feel free to ask them again here, or on the BayPIGgies mailing list.

The first question I remember was whether I’ve found any bugs in Mock. I replied no, and elaborated that Mock was written using test-driven development and the tests themselves use Mock (and are included with Mock). So if you do find a bug, you can easily fix it yourself. (But looking at the changelog, it appears there have been few bugs fixes. Most changes have been new features. BTW, I’ve submitted a couple patches with new features myself, but I believe Michael decided he wasn’t ready for them or decided to implement them in a different way. TODO: I have a couple ideas for other new features to add.) And Mock’s unit tests are good examples of how to use Mock.

Then Alex Martelli commented (I probably won’t get this right) that my use of the @patch decorator in my tests is dangerous and won’t scale. (It’s not clear to me whether he meant it won’t scale to projects with large amounts of code, or large numbers of programmers, or large numbers of processes and/or threads, or something else entirely different.) He suggested we have a look at the video of him talking about this. Update: I thought that may be this YouTube video from Google Developer Day 2008 on “Python Design Patterns” (added June 4, 2007?), but Alex comments (below) that the video is not currently available, but we can find the slides at http://www.aleax.it/yt_pydi.pdf. And Alex described how the code would use “self” to refer to any dependency, which would allow that dependency to be changed by tests, or by other code that might use it.

I then asked a question (which I can no longer recall) and Alex stood up and came up to the microphone to elaborate. After sitting down, I (and others) asked further questions (which I also can’t recall) and Alex got some exercise sitting down and getting back up a few times until the two of us stayed at the podium and Alex gave me (and the audience) an impromptu (but very meaty) lesson on dependency injection. (I hope Google makes the video of this available—I’d like to watch it again myself. I haven’t found the video of my previous newbie nugget presentation yet though.) I do remember one (rather foolish) question I asked: I described how I had tried dependency injection using keyword arguments to functions with defaults, and how that gets complicated to test when testing a function which calls a function which calls a function with such an argument. Alex kindly repeated that dependency injection should be implemented using “self”, which I took to mean that one would use an object’s attributes to hold the dependencies. (This would certainly make the scenario I described much simpler to test.)

In a discussion after the meeting, I explained to one of several people I had very interesting follow-up conversations with (I didn’t get his name) on a white-board how this would work. The Google engineer who had graciously agreed at the last minute to host our meeting (I didn’t get is name either—but learned later that he was visiting from Australia) was watching (probably because we were over-staying our welcome) and as we were walking how he explained how the accepted technique is actually to use class attributes for the dependencies, since it’s less work to inject new dependencies into multiple objects, but one can still override dependencies for a particular object (since an object attribute will “replace” a class attribute with the same name).

One other question I recall (it may have been the last) was -jj’s. He asked (shrewdly) asked me to explain why one would want to use mocks. I decided to (figuratively) take a step back and first describe why one would want to test and use test-driven development (and what that is). I described the benefits those practices and then explained (or tried to) how mocks are used in testing to replace dependencies and allow unit tests to be truly “unit” tests and run fast to keep the test-implement-refactor cycle quick. TODO: Write this up in detail.

I concluded by thanking everyone for their questions and comments (especially Alex)—I stated I felt like I had learned more from everyone else than they probably did from me. When I sat down afterward, Alex turned to me and quoted Richard Bach: “You teach best what you most need to learn.”. Absolutely.

Next: Write up my experiences modifying my code and tests to use dependency injection, and contrast the new versions of the two examples from my slides.

Thursday, July 10, 2008

Newbie Nugget: Introduction to the with statement

I’ve signed up to present the first BayPIGgies “Newbie Nugget” at tonight’s meeting, on the with statement. I know from past experience that the best way to learn something is to explain it—and one has to learn it very well to explain it to a large group of smart people.

What follows are the notes I compiled as I taught myself about the with statement. I’ll turn these into slides for tonight. Since I only have 10 minutes for the presentation, I’ll keep these notes brief. (Think of this as an introduction to the with statement.) You’ll find links to further reading below.

We’re all used to (or should be used to) writing code that looks like this:

f = open(filename)
try:
print f.read()
finally:
f.close()

The with statement (introduced in Python 2.5) simplifies that to:

from __future__ import with_statement
with open(filename) as f:
print f.read()

(“with” and “as” are keywords in Python 2.6, and “from __future__ import…” is not needed.)

The specified syntax for the with statement is:

with EXPR as VAR:
BLOCK

The “as VAR” part is optional.

The EXPR must result in a context manager. Briefly, a context manager is an object with a __context__() method, which returns a context object. A context object has an __enter__() method, which is invoked as soon as a context object is returned. Its return value is assigned to VAR (if there is one). When the with statement terminates, the context object’s __exit__() method is called. (I won’t take the time to go into the arguments to __exit__().)

The contextlib module provides “utilities for common tasks involving the with statement.”

The contextlib.closing function returns a context manager that closes its argument. For example:

from __future__ import with_statement
from contextlib import closing
import urllib

with closing(urllib.urlopen('http://www.python.org')) as page:
for line in page:
print line

And the contextlib.contextmanager function “is a decorator that can be used to define a factory function for with statement context managers, without needing to create a class or separate enter() and exit() methods.” For example:

from __future__ import with_statement
from contextlib import contextmanager

@contextmanager
def opened(filename):
f = open(filename)
try:
yield f
finally:
f.close()

with opened(filename) as f:
print f.read()

There is plenty more detail in PEP 343, including plenty of fun examples. My favorites are:

  • (2) The opened() context manager used above
  • (3) transaction() context manager for committing or rolling back a database transaction
  • (5) stdout_redirected() context manager for temporarily redirecting stdout
  • (7) A context manager for blocking signals. (This one is actually left as an exercise to the reader.)

To learn about the with statement (before reading PEP 343 and the contextlib module documentation), I first turned to section 10.4 (Context Management) of Core Python Programming (2nd Edition) by Wesley J. Chun. It provided an excellent introduction.

Tuesday, April 1, 2008

PyCon 2008 Notes

I just completed writing up my notes on PyCon 2008 for my employer (who paid my way), and thought I should also share them here. I apologize for using one long blog post; I decided the ability to refer to a single post outweighed the advantages of several smaller ones. (And you'll have to ignore the "TODOs" I've sprinkled in the text.)

I attended PyCon from March 13 through 20, 2008.

Tutorials

Thursday April 13 was a day of tutorials. I attended three (of a maximum of three).

Secrets of the Framework Creators Tutorial

The first tutorial I attended was "Secrets of the Framework Creators", presented by Feihong Hsu and Kumar McMillan. This was an excellent tutorial that opened my my to some much more sophisticated ways of using Python. (See the three previous posts I've written on this tutorial so far.)

You'll find the materials they prepared for the tutorial in the Google Group they created. They covered four topics:
  • Frame Hacks
  • Decorators
  • Metaclasses
  • Magic Methods
(Of these four, frame hacks and metaclasses were completely new to me. I already had some experience with decorators and magic methods.)

It would be a waste of my time to summarize any of the material presented, since the tutorial materials do such an excellent job introducing these subjects. If you want a quick overview, you can probably read through them in an hour or less. And if you really want to learn to use these techniques, you can spend a few hours on the exercises they've provided.

Here are the few brief notes I took during the tutorial:
Generator Tricks for Systems Programmers

The second tutorial I attended was "Generator Tricks for Systems Programmers", presented by David Beazley. David has also made his tutorial materials (including excellent slides and plenty of code samples) publicly available: http://www.dabeaz.com/generators/.

At the start of the tutorial I had written iterators, and had a vague understanding of generators. After the tutorial, I now feel like my understanding of generators is much deeper. I very highly recommend you take the time to read through the slides (probably one to two hours) and look through his code samples.

TODO - add a summary here

My notes:
  • I didn't know that "Conditional Expressions" (aka "the Ternary Operator") was added to Python in 2.5. See an example at the bottom of slide 64.
  • There's a nifty trick in using the max function under the first bullet of slide 71. TODO - explain.
  • I wrote on slide 135 that the find_404() function could be modified to take a receiver arg and call it instead of the print (and the print could be wrapped in a consumer function). TODO - explain
  • TODO - I also gave myself a TODO on slide 135 to compare this to the previous use(s) of broadcast
  • TODO - Someone mentioned the "etree" (I think they meant ElementTree), which either uses generators for parsing XML or can be used with generators.
Django Code Lab

The last tutorial I attended was the Django Code Lab. The presenters, Adrian Holovaty (one of the creators and core developers of Django), Jacob Kaplan-Moss (a core developer of Django) and James Bennett (release manager for Django) asked people to send in questions and code samples, and then reviewed them in the lab. (I did submit some code developed at the last minute--my main question was how best to integrate models that don't use the database with others that do--but mine was not chosen.)

The slides are posted at http://toys.jacobian.org/presentations/2008/pycon/codelab/. They may not be too useful out of context. But I refer to them in my notes (below):

The first up was Pim Van Heuven:
  • he had a huge urls.py file - see the slides
  • they also presented "Forms with extra parameters"
Next was Justin Lilly:
  • Jacob ranted about TDD
  • then he presented some useful information about testing Django
    • look at the model examples in the Django documentation, they're all working unit tests
    • see slides for example of using django.test.TestCase
Next was Richard House:
  • he had a long list of BooleanFields in his model
    • they pointed out that one should use NullBooleanField rather than BooleanField(null=True)
    • the slides present a pattern for using "two pairs of models"
    • Adrian pointed out this is called EAV (Entity-Attribute-Value) and noted this has problems with searching and queries
Next, Peter Herndon:
  • he suffered from slow queries (see the slides)
  • someone recommended a Malcolm Tredinnick blog post (I think this one); though his blog is definitely recommended reading for anyone using Django
  • see the optimization notes for your database - Jacob says the PostgreSQL optimization notes are especially good
  • James recommends especially django-sphinx for searching (David Cramer used it for a very busy site with a single database server)
  • Jacob mentioned Google Co-op several times
Next, J. Clifford (Cliff) Dyer:
  • Handling previous/next links
  • I didn't fully understand the discussion, and there were no code examples provided unfortunately (and no slides)
Next, Dave Lowe:
  • the subject was when not to use the admin
    • the advice was that the admin should only be used by someone trusted with access to the entire database (even after the newforms-admin branch is merged back into the trunk)
  • Adrian said that the key to mastering Django as a tool is to learn (through experience) when to use different parts of Django and when not to
Next, Bob Haugen:
  • prepping for deployment
    • make sure that django.views.static.serve is blocked out inside settings.DEBUG
    • see the HOME trick with os.path.join() for TEMPLATES_DIR
    • see the urlresolvers.reverse slide
    • worth looking at mod_wsgi first (before mod_python)
    • Jacob admitted (reluctantly) that MySQL is easier to setup than PostgreSQL
    • see the slide 97, which summarizes "Develop" vs. "Deploy"
      • Jacob forgot to include creating 500.html & 404.html pages
Next, Wiley Kestner:
  • see the slides for details on the Django dispatcher
Then they ran out of time, but the slides also contain some details on REST APIs,

Slides for other tutorials

I got the URL for the slides from the Introduction to SQLAlchemy tutorial from IRC: http://utahpython.org/jellis/. The slides are in "sa-intro.pdf". (But it looks like there's some other interesting stuff there too.)

Conference Sessions

Friday, April 14 through Sunday, April 16 were the Conference Days. Each day contained keynote talks, scheduled talks, lightning talks, and "open space". (See the previous link for details.)

I'll go into detail on the scheduled talks (from all three days) first, and then dump all my notes from the lightning talks.

The A/V team as begun posting recordings from PyCon 2008 to their YouTube channel: see this blog post. As of 2008-03-28 they've got 12 up, some from regular sessions and some from lightning talks. None are from tutorials--I don't know if they intend to post tutorial videos.

Guido van Rossum - "Python 3000 and You"

The first technical talk of the conference was fittingly Guido's talk on Python 3000. Guido has posted the slides and some advice when porting to Python 3.0 on his blog.

I didn't take any notes, but if you're interested in Python 3000, PEP 3000 is work reading. (In addition to the slides.)

Brett Cannon - How Import Works

This was useful information. You'll find the slides in http://us.pycon.org/2008/conference/schedule/event/12/.

See also http://www.python.org/dev/peps/pep-0328/ and http://www.python.org/dev/peps/pep-0302/.

Dr. Tim Couper - Python references and practical solutions to reference-related problems

This was an excellent talk. The video isn't up yet, but I hope it is soon. (Unfortunately no slides are attached to http://us.pycon.org/2008/conference/schedule/event/16/.)

Here are my (sketchy) notes:
  • see the getrefcount() function
  • weakref.ref()
    • can get the actual reference from a week reference, with "()"
  • getweakrefcount()
  • getweakref()
  • garbage collector will clean up objects not deleted because of a circular reference
  • can turn off GC if you're sure you'll have no circular references
  • GC will not clean up objects with a __del__ magic method
    • but you can get a list of these objects
  • then I got distracted and got lost when he talked about pickling and weak references as a solution for this?
    • something interesting about a pattern using __getstate__ and __setstate__ methods
    • also talked about finalizers
Noah Gift - Using Optparse, Subprocess, and Doctest To Make Agile Unix Utilities

The presentation and code samples are at http://code.noahgift.com/pycon2008/.

TODO - I need to take a look at the subprocess module.

Kevin Dangoor - Rich UI Webapps with TurboGears 2 and Dojo

This was a very slick, fast-moving presentation with excellent examples on how to use Dojo. There was also an example of (bleeding edge, I think) use of Comet to push data from the server to the client.

I'll want to see this one again when it's available on video. I haven't been able to find the slides, but Kevin promises to post a screencast version in this blog post.

Adrian Holovaty - The State of Django

I haven't been able to find the slides, and the video isn't up yet. But I recommend it (if you're interested in Django).

Here are my notes:
  • What's new in the last year:
    • 0.96 - released March 23, 2007!
    • Unicode branch - is it still in a separate branch? (I think it was merged to trunk)
    • auto-escaping in templates
    • GeoDjango - still a branch
      • "hope to get it integrated into the trunk soon"
    • Sprints in Sept. & Nov.
    • 2432 checkins
  • Community stuff:
  • What's coming
    • mostly mature/stable
    • queryset-refactor to be merged to trunk
      • includes support for model subclassing
    • select_related - can now specify args
    • newforms-admin
      • admin options defined in separate class; register
      • has_change_position() method in ModelAdmin classes
      • can regulate which objects show up in the admin
      • can have multiple admin sites on the same website
    • want to add more sophisticated INSTALLED_APPS (using classes)
      • can define database prefixes & labels (?)
    • model validation
    • 1.0!
Marty Alchin - Django: Under the Hood

Plenty of useful information that I plan to review again (more slowly). His slides are at http://www.slideshare.net/Gulopine/django-under-the-hood/.

Some notes:
  • the quick review of how metaclasses are used was useful as used a metaclass in the code I wrote during the Django sprint
  • the signals discussion may be useful to users of Django too
  • Utilities
    • functional utilties
      • several, including turn middleware into per-view decorators
    • text utilities
      • various
  • "remember, Django is Python, just read the source"
    • but beware of query.py and related.py
See this Adam Gomaa blog post for links to the source files to look at for model metaclasses, the "signalling stuff" and "some neat functional utilities".

Chris McDonough & Mike Naberezny - Supervisor as a Platform

This looks like a very useful tool. I plan on using it very soon. The slides are in http://supervisord.org/wp-content/uploads/2008/03/ and are very good--I don't have anything to add. (Except I read a rumour somewhere that Guido wrote the original version of this--details are in http://supervisord.org/contributors/.)

Mike Bayer - SQLAlchemy 0.4 and Beyond

I missed this because it was at the same time as the Supervisor session. But I heard this was very good. You'll find the slides at http://techspot.zzzeek.org/?p=22.

Rodney Drenth - Decorated State Machines

I missed this one too (it was at the same time as the above two sessions). But I see the slides are at http://us.pycon.org/2008/conference/schedule/event/43/. TODO - I haven't looked at them yet.

Maciej Fijalkowski - The State of PyPy

This was interesting. I learned a bit about PyPy (a subject I know very little about). I can't find the slides, and the video isn't up yet.

My notes are very brief:
  • Currently:
    • quite compliant 2.4/2.5
    • still missing some standard libs
    • new: ctypes!
I vaguely recall an interesting demo.

James Bennett - Developing Reusable Django Applications

This was a very good talk that would be worth reviewing by anyone using Django. The slides are in http://us.pycon.org/2008/conference/schedule/event/50/.

I'll repeat some of the slides that I noted:
  • Four big ideas:
    • do one thing, and do it well (the UNIX philosophy)
    • don't be afraid of multiple apps
    • write for flexibility
    • build to distribute
  • He wrote django-registration
    • people asked for profiles (in django-registration)--he said no
    • but then he wrote django-profiles (as a separate app)
  • The Django mindset
    • application == some bit of functionality
    • site == several applications
    • tend to spin off new applications liberally
  • What reusable apps look like
    • single module directly on Python path
    • related modules under a package
    • no project cruft whatsoever
  • Good examples:
  • More information:
Alex Martelli - Don't call us, we'll call you: callback patterns and idioms in Python

This was at the same time as the Developing Reusable Django Applications talk, so I missed it. But the slides are in http://www.aleax.it/pyc08_cback.pdf. TODO - I haven't looked at them yet, but I've read good things about this talk.

Brandon Rhodes - Using Grok to Walk Like a Duck

You'll find the slides and example source code at http://rhodesmill.org/brandon/adapters/. (And this one is on YouTube. The title is misleading--this was really primarily about the Adapter pattern and the Zope Interface class. I left wishing he provided some examples of using Grok.

I got a little lost when he gets into zope.interface.Interface. TODO - The best way to understand this would be a play around with it.

Steven Wilcox - The Power of Django Admin (Even For Non-Django Projects)

The "slides" (actually a paper) are at http://devpicayune.com/pycon2008/django_admin.html. This doesn't look familiar, so I must have been busy with something else at the time--though I can't recall what. Perhaps that presentation wasn't as compelling as the material deserved. TODO - This does appear to be worth reading.

Brian Dorsey, Maciej Fijalkowski - py.test: Towards Interactive, Distributed and Rapid Testing

This was a very good presentation that I would recommend watching when it's available on YouTube.

See http://codespeak.net/py/dist/test.html

Here are my notes:
  • tests for py.test are almost identical to nose
  • tab completion in bash after -k flag slick
  • I missed the test reporting page thing, but heard it was slick - and I believe that's one thing that distinguishes py.test from nose
  • 1.0 soon
    • more plug-in architecture
  • yes, there is a tool for converting unittest tests to py.test tests
    • you can convert tests, or just run them without (permanently) converting them
  • what does py.test do that nose doesn't?
    • the introspection magic
  • py.test.raises(Exception, stuff, arg1, arg2)
  • TODO check out py.execnet (http://codespeak.net/py/dist/execnet.html - "A new view on distributed execution")
Jim Baker - More Iterators in Action

My notes say the slides are at http://zyasoft.com/, but that page strangely describes Zyasoft consulting services and software development but provides no links or contact information. The slides are attached to http://us.pycon.org/2008/conference/schedule/event/75/ though.

Here are my notes: Raymond Hettinger - Core Python Containers - Under the Hood

An excellent presentation that I will most definitely watch again when it's available on YouTube. I haven't been able to find the slides.

I didn't take very good notes: Collin Winter - 2to3: Translating Python 2 to Python 3

I haven't been able to find the slides.

My notes are very brief:
  • collinw@gmail.com
  • workflow:
    • maintain in 2.x
    • fix 2.6 -3 warnings
    • run 2to3
    • test suite!
    • release 3.x version
Lightning Talks

There were three Lightning Talk sessions, after the scheduled talks on Friday, Saturday, and Sunday. Here are my notes. (I marked my favorites with *.)

Friday (April 14) Saturday (April 15)
  • Bazaar - "if you can run Python 2.4, then you can run Bazaar"
Sunday (April 16)
UPDATE (2008-04-03)

As I read through blog posts on PyCon, I'll add links here.

To start, I recommend -JJ's posts on:
Did I miss any? (You just may want to scan all his PyCon 2008 posts.)

UPDATE (2008-04-11)

Yesterday evening I presented my "PyCon 2008 Notes" to BayPIGgies, in slide form. The slides don't have any of the links above, and I stripped out much of my notes, but if you really want them you'll find them at http://www.spitzer.us/daryl/baypiggies/pycon_2008_notes/ (exported to HTML from Keynote) or http://www.spitzer.us/daryl/baypiggies/pycon_2008_notes/pycon_2008_notes.pdf.