Print

Mod_python's PSP: Python Server Pages

by Gregory Trubetskoy
02/26/2004

The new 3.1 version of mod_python introduces several major additions and enhancements over the previous 3.0 version. They are PSP, Cookie, and Session support. This article will introduce the first addition on the list, PSP.

Python Server Pages (PSP), as you probably guessed already, is a way to inline Python in HTML or XML documents. The server interprets this inlined code to produce the final HTML sent to the client. This approach has become popular with tools such as JSP, PHP, and ColdFusion.

Several other projects share the name "PSP", including Webware PSP and Perl Server Pages. The canonical name of what this article describes is therefore "Mod_python PSP", but for simplicity I will often refer to it as simply PSP.

PSP Story

Inlining Python in HTML is a subject of some controversy. Some people consider code inside HTML a bad programming practice, since it utterly violates the Model-View-Controller paradigm by placing the application logic inside the presentation layer. (I am inclined to agree with this, but as you will see later on, there are ways to use PSP that actually comply with the MVC model.) Bad as it may be, millions of PHP users show a clear demand for this style of programming — developers are having success implementing sites this way.

The inclusion of PSP in mod_python was rather unexpected. I have always maintained that something like this is outside the scope of mod_python, whose main objective is Apache-Python integration, not high-level web application tools and frameworks. Meanwhile, expecting "batteries to be included", many new mod_python users had expressed disappointment on the mailing lists that mod_python lacked PSP-like functionality. Quite a few frameworks worked with mod_python, but it seems like to choose a framework, one would have to try them all out first and make a decision personally.

Then one day an announcement came across the Python mailing list from Sterling Hughes (a PHP core developer, but obviously a Python fan as well!) regarding mod_psp. Mod_psp was an Apache module written in C that provided bare-bones Python-in-HTML functionality in a clean and simple way. Mod_psp was not thread-safe and had some syntactical short-comings, but it appeared as a great starting point for something that could be included in mod_python, mainly because it was fast and written specifically for Apache. Even better, Sterling had agreed to donate his code to ASF and had spent time on the initial work of integrating mod_psp into mod_python.

PSP Objectives

Early in the PSP-related discussions on the mod_python development list, I compiled a list of criteria that I felt were key to a successful implementation:

  • There should be no new language to learn. Many other frameworks, mainly as way to overcome the white space conflict between Python and HTML, introduced alternative syntax, often complex enough that it appeared as a separate language somewhat resembling Python. In my opinion this contradicts the spirit of simplicity and clarity of Python. New syntax can also be a significant deterrent for developers who rely on language syntax support features of their editor.

  • It should be as fast as possible. Mod_python solves many complex and low-level operating system and networking issues for the sake of performance, something that most Python developers do not have the skill or time to deal with. A PSP implementation should comply with this notion by being fast and clean. There is no value in hacking together yet another slow parser implemented in Python. The fact that PSP is a flex-generated C scanner adds real value in that it is not something that most people "can try at home".

  • It should require no Python semantics in HTML. Python uses indentation to denote blocks of code. Indentation has no significance in HTML. PSP should not require HTML to be indented.

  • It should require no HTML semantics in Python. Since HTML pays no attention to indentation, it seems there should be another way to denote blocks within PSP. The first temptation is to introduce a code block delimiter character into Python. PSP should avoid this.

PSP Syntax

PSP is similar to early JSP, delimiting its code using the greater/less-than/percent tokens (<% and %>).

Similarly to JSP, PSP has four types of entities:

  1. Code represents the Python source code that drives the logic of how the final output is produced. It's enclosed in <% and %> markers.

  2. An expression is Python code whose resulting string representation becomes part of the final output. They are enclosed in <%= and %> markers.

  3. Directives are special instructions to the PSP processor. Directives are enclosed in <%@ and %> markers.

  4. Unlike HTML comments, PSP comments are removed by the PSP parser and never make it into the final output. They are enclosed in <%-- and --%>.

Admittedly, this syntax is rather primitive and not XML compliant, but it's a start. Most likely, later versions will feature an alternative XML-compliant syntax, opening the door for XSLT-generated PSP pages and other XML goodness.

The thorny issue of indentation is addressed by making the last indentation in effect "stick" throughout the HTML that follows it, with a recommendation to use meaningful comments to make code more readable, for example:

<%
if x == y:
   # begin
%>

  ... some html ...

<%
# end
%>

The above code snippet also demonstrates a subtle syntactic difference introduced by PSP — the indentation of last line of code matters even if it is a comment.

The PSP syntax also introduces a small gotcha:

<%
if user in administrators:
   level = 'admin'
else:
   level = 'user'
# end
%>

  ... some html ...

In the above code, it's easy to forget the block-terminating comment (# end in this case). Without it, some html becomes part of the else block and will only be included in the final output when the user in administrators condition is false.

In general, this syntax works quite well. Its only minor limitation is that it takes more space (e.g., three lines to terminate a block), though some will consider it a feature.

Hello World Example

For the impatient, here is a quick Hello World without much explanation. To use PSP you have to configure Apache and mod_python to use the mod_python.psp handler. Here is the relevant part of the Apache config:

<Directory /some/path>
   AddHandler mod_python .psp
   PythonHandler mod_python.psp
   PythonDebug On
</Directory>

Here is the PSP code. It must be in a file ending with .psp, at least, according to the above configuration:

<html>
<%
if form.has_key('name'):
   greet = 'Hello, %s!' % form['name'].capitalize()
else:
   greet = 'Hello there!'
# end
%>
  <h1><%= greet %></h1>
</html>

The above example will produce "Hello there!" if you simply invoke the page. If you append the URL with ?name=john query argument, then the result will be "Hello John!".

Under the Hood

The mechanics of PSP are quite simple. The PSP parser converts the PSP page into pure Python code suitable for execution in mod_python. This is best demonstrated by an example:

<html>
<%
import time
%>
<h1>Current time is
<%= time.ctime() %> </h1>
</html>

will become something like:

req.write("""<html>
""")
import time
req.write("""
<h1>Current time is
""");req.write(str(time.ctime()));req.write("""</h1>
</html>""")

The above Python code is then compiled into a code object using Python's built-in compile() function. PSP caches that object and reused it fo subsequent requests, unless the source file changes on disk.

The strange use of semicolons in the resulting Python code is there to try to preserve line numbering. The PSP parser is not capable of producing any errors. Bad PSP will simply result in bad Python, which will cause compilation errors from the Python interpreter. Having the line numbers reported by the interpreter correspond to the line numbers in the original PSP page helps tremendously in debugging.

Global Variables

Several variables exist in the global namespace at the PSP page execution time. These variables, therefore, can be used without assigning a value to them first. They are:

1. req

req, the mod_python Request object. This means that all of the advanced functionality of mod_python is still available within the PSP pages.

2. psp

psp, an instance of PSPInstance, which provides access to a small PSP-specific API. This object has the following methods:

  • set_error_page(filename)

    This allows you to specify a PSP page to be invoked when a Python error occurs. This is useful for customizing error output, similar to the errorPage directive in JSP.

  • apply_data(object)

    This method will call the callable object object, passing form data as arguments and return the result. If you are familiar with JSP, this works much like setProperty. If, for example you have an object defined as follows:

    class Car:

   def __init__(self, color):
       self.color = color
       # etc.

    Then a PSP page called as result of a form submission (the form contains a field named color) can do this:

    <%
car = psp.apply_data(Car)
%>

    This will call the callable object Car (classes are callable), passing it the value of form field named color, resulting in an instance of Car which is assigned to car.

  • redirect(location)

    This can be used for redirection from within PSP pages. It's important to call this function absolutely first in the PSP page, because redirection cannot happen after there is any output sent to the browser.

3. form

form is the form data in a dictionary-like object (mod_python FieldStorage). Merely mentioning this in the code causes a FieldStorage instantiation, thereby consuming all POST input. PSP uses Python's introspective qualities to determine whether a piece of code uses the form variable. If the form variable is not mentioned in the code, then no FieldStorage is instantiated.

4. session

session Similarly, PSP's the behavior changes if you mention this variable inside the PSP page. When PSP detects that a page refers to the session variable, it automatically creates a mod_python.Session object, which will generate session cookies and turn on session locking ensuring that each unique session can only have one active request to this page at a time.

Directives

At this point PSP supports only one directive.

<%@ include file='filename'>

The PSP parser will replace this directive by the contents of the file filename. This can be a very useful feature, although it presently carries the limitation of complicating debugging because it throws off any correspondence of original line numbers to the line numbers in the resulting Python code.

Debugging

As I already mentioned, the PSP parser produces no errors. This may change in the future, but for now this is the case. Only the Python interpreter produces errors, but those errors will refer to the PSP-generated Python code, which is not visible to the developer. It can be difficult at times to associate the error condition reported by Python with the original PSP source.

To aid in this, the PSP handler provides the ability to peek at the intermediate Python code produced by the parser. This only works when PythonDebug Apache configuration directive is On. If you append an underscore to the PSP URL, you will receive a nice listing showing original PSP on the left and the resulting Python code on the right.

If the original link were http://localhost/test.psp, then http://localhost/test.psp_ will show the PSP-generated Python source code. For this to work, you must register the .psp_ extension (with the underscore) with AddHandler:

AddHandler mod_python .psp .psp_

Using PSP as a Templating System

Most *SP's only support the mode of operation where the code is inlined in a web page referred to in the URL. As I mentioned above, this is not the best programming practice because it places the application logic inside the presentation component. Mod_python PSP can also work as a flexible templating system from within Python code. I advocate this method of using PSP because it allows for a clean programming style, separating presentation from logic.

Let's look at an example of such usage. Since we're not using PSP as a handler, we will use the Publisher handler instead. Since the Publisher handler is outside the scope of this article, I will instead list the code that I think is self-explanatory. If you need more information on the Publisher, I recommend looking at the mod_python tutorial section in the standard documentation.

Here is a snippet of the relevant Apache configuration for using the Publisher:

<Directory /your/document/root>
SetHandler mod_python
PythonHandler mod_python.publisher
PythonDebug On
</Directory>

Since we're using PSP as a templating mechanism, we need a template. Let's assume it is in a file named /your/document/root/hello.tmpl:

<html>
   <h1><%= greet %></h2>
</html>

Here is the Python script. Let's assume it is in a file called /your/document/root/pubpsp.py.

from mod_python import psp

def hello(req, name=''):
   s = 'Hello, there!'
   if name:
       s = 'Hello, %s!' % name.capitalize()

   tmpl = psp.PSP(req, filename='hello.tmpl')
   tmpl.run(vars = { 'greet': s })
   return

By the magic of the Publisher handler, you can invoke the hello() function in the script via http://localhost/pubpsp/hello. Pass a name to the hello() function with http://localhost/pubpsp?name=joe.

In the above example, we load the PSP template by creating an instance of the psp.PSP class. At the time of instantiation, PSP either translates it into Python source code and then compiles it or loads it from the PSP cache. (Yes, caching is on even when PSP is used this way).

The next line calls the template's run() method, passing it a vars dictionary. This is when the template is actually executed and its output is sent to the client. The vars dictionary is a list of variable names that will be added to the global namespace of the template just before it is executed. We pass it a variable called greet which is referred to in an expression inside the template.

Nested PSP Templates

A nice feature of the PSP templates is that one template can contain references to another in an expression. Let's expand the above example. The Apache configuration stays the same, but we'll add another template file called time.tmpl:

<h2>
   And the time is <%= now %>
</h2>

We'll modify the hello.tmpl template:

<html>
   <h1><%= greet %></h2>
   <%= time_tmpl %>
</html>

Finally, our new script code is:

from mod_python import psp

import time

def hello(req, name=''):
   s = 'Hello, there!'
   if name:
       s = 'Hello, %s!' % name.capitalize()
   time_tmpl = psp.PSP(req, filename='time.tmpl',
                       vars = {'now': time.ctime()})
   hello_tmpl = psp.PSP(req, filename='hello.tmpl')
   hello_tmpl.run(vars = { 'greet': s, 'time_tmpl':time_tmpl })
   return

In the above example we instantiated an additional template, time_tmpl. Note that this time we passed a variable called now as part of the constructor rather than an argument to the run() method. PSP allows you to pass variables either or both ways. Then we pass the time_tmpl PSP object to the hello_tmpl.

In this example, the time_tmpl is parsed and compiled at instantiation time, but is executed only when the container template's run() method is called.

This example is, of course, completely impractical, but it demonstrates a very useful feature. Imagine that you have a complex site that contains a dynamically generated menu. You can place such a menu into a template of its own and include it in the other templates of the site.

Conclusion

The PSP functionality included in mod_python 3.1 is powerful and versatile, even though it is only the first version. Its ability to be used as a class from within the Python code makes it suitable for clean web development in accordance with the MVC paradigm.

Gregory Trubetskoy is the lead developer of mod_python and a member of the Apache Software Foundation.

Return to the Python DevCenter.