String Interpolation

One of the prerequisites for recipe #1 ("Ruby-style string interpolation") from the Frame Hacks section of the "Secrets of the Framework Creators" PyCon tutorial is the string.Template class. Before I took the tutorial I was vaguely aware of the string.Template class, but I don't think I've ever used it.

I'm going to consider using it though. There are probably plenty of cases where it would be more readable than using the % operator--I can recall a couple times in code I wrote very recently.

In step 1 of Frame Hacks recipe #1, Feihong and Kumar how how one can write a function that wraps string.Template and the substitute method so the mapping argument (and optional keyword arguments) don't need to be provided.

SPOILER WARNING

def interpolate(templateStr):
  import sys
  from string import Template
  t = Template(templateStr)
  frame = sys._getframe(1)
  return t.substitute(**frame.f_locals)

name = 'Feihong'
place = 'Chicago'
print interpolate("My name is ${name}. I work in ${place}.")

Here's what it could look like without the sys._getframe() call:

def interpolate(templateStr, **kws):
    from string import Template
    return Template(templateStr).substitute(kws)

name = 'Feihong'
place = 'Chicago'
print interpolate("My name is ${name}. I work in ${place}.",
                  name=name, place=place)

I guess I could argue that the latter is more explicit, but perhaps the former is explicit enough. I guess I'll be the judge of that (for myself) the next time I write (or am ready to refactor) an ugly string using the % operator. I'll let you know if I decide to add the interpolate function to my utilities module.

The solution for step 2 of recipe #1 (which I showed a partial solution for when I wrote about Frame Hacks) allows for expressions in the template strings. I don't think I've run across a use case for that, but I'll write about it if I do.

I should also mention that Python 3.0 will include a new (third) method of string interpolation, the format method to be added to the built-in string class. (There's more to it than just that. You can read all about it in PEP 3101. And if--like me--you don't know just what a PEP is, I recommend reading the "PEPs" section hear the top of the description of Python's Development Process. And you'll find plenty more detail in PEP 1.)

The above example would look like this using the new format method:

name = 'Feihong'
place = 'Chicago'
print "My name is {name}. I work in {place}.".format(name=name, place=place)

There doesn't seem to be a spec for the format method yet, so it's not clear whether one could wrap it and use sys._getframe(). I don't know if that's an argument for avoiding using it now though. (I don't see any reason why the interpolate() function above wouldn't work in Python 3.0.)

TODO

I haven't installed the latest Python 3.0 alpha yet. I should make time to do so and play with the above.

Frame Hacks

Someone (I forget who--I've listened to so many smart people here at PyCon) said yesterday that Python treats us like adults--if we want go "off-road" we can. (Other languages try to protect programmers from themselves and others. With Python we need to write tests to get that protection. Not that statically-typed languages with protected attributes and methods make testing unnecessary. But that's a subject for another time.) But until I took the "Secrets of the Framework Creators" tutorial at PyCon, I had no idea just how far into the wilderness one could go.

As Leihong and Kumar explain in the getting started chapter of the frame hacks section, one can call sys._getframe() to get the "frame" of the calling functions, all the way to the bottom of the call stack. This made immediate sense to me, since one of the first bits of real code I wrote when I started at Altera did a stack trace on the Nios (Classic) processor. (That was tricky, since the original Nios used register windows--but I'd better stop before I go down a rat hole.)

As I wrote previously, I'm working my way through the tutorial again on my own to give myself a chance to get to the point where I can use the techniques (rather than just understand them when I see them used).

I worked my way through the three steps of the first frame hacks recipe without a problem, though (to be honest), I've never used the string.Template class and the re.finditer function before (though they did seem familiar, so I guess I've read about them). I don't think I've used the yield statement either, though I'm eager to now that I've been immersed in generators and co-routines in the "Generator Tricks for Systems Programmers" tutorial (which was also excellent and which I'll write about later).

But I noticed that the code in the third step (step 2) of that recipe actually works without using sys._getframe():

SPOILER WARNING

import sys, re
from string import Template

def getchunks(s):
   matches = list(re.finditer(r"\$\{(.*?)\}", s))

   if matches:
       pos = 0
       for match in matches:
           yield s[pos : match.start()]
           yield [match.group(1)]
           pos = match.end()
       yield s[pos:]

def interpolate(templateStr):
   result = ''
   for chunk in getchunks(templateStr):
       if isinstance(chunk, list):
           result = ''.join((result, str(eval(chunk[0]))))
       else:
           result = ''.join((result, chunk))
   return result

name = 'Guido van Rossum'
places = 'Amsterdam', 'LA', 'New York', 'DC', 'Chicago',

s = """My name is ${'Mr. ' + name + ', Esquire'}.

I have visited the following cities:  ${', '.join(places)}.
"""

print interpolate(s)

That's because the eval in the interpolate function can find "name" and "places" in the global namespace. I can't recall if Leihong or Kumar noticed this in the tutorial, but I imagine they intended something like this:

import sys, re
from string import Template

def getchunks(s):
 matches = list(re.finditer(r"\$\{(.*?)\}", s))

 if matches:
     pos = 0
     for match in matches:
         yield s[pos : match.start()]
         yield [match.group(1)]
         pos = match.end()
     yield s[pos:]

def interpolate(templateStr):
 """Implement this function"""

def main():
 name = 'Guido van Rossum'
 places = 'Amsterdam', 'LA', 'New York', 'DC', 'Chicago',

 s = """My name is ${'Mr. ' + name + ', Esquire'}.

    I have visited the following cities:  ${', '.join(places)}.
    """
 print interpolate(s)

if __name__ == '__main__':
 main()

You'll need to use sys._getframe() in interpolate() to get this to work.

TODO

I should look into Trellis, Elixer and lxml, which were used in the tutorial to give use cases for frame hacks.

And before I move on to the next section of the "Secrets of the Framework Creators" tutorial, I should describe the string.Template class and the re.finditer function. Stay tuned.