December 2007 Archives

AddThis Social Bookmark Button

I’m glad to see that TPF’s public relations group has spread the Perl 5.10 press release far and wide, and it’s getting some coverage. However, some of that coverage reminds me why I don’t watch television news and why I treat the newspaper as entertainment and not information. Consider eWeek’s First Release of Perl in Five Years Arrives:

Perl is a dynamic scripting language widely used in everything from Linux system utilities to Web servers to full-blown graphical enterprise applications.

What’s a “dynamic scripting language”? Is there such thing as a non-dynamic scripting language? (No one seems to know what a scripting language is anyway.) Minor nit.

During its 20-year history, it gained massive popularity by assimilating the syntax from many predecessors, making it really easy to use for anyone already versed in sed, awk, grep, csh, C/C++, Lisp, and so on.

Syntax, maybe (but Lisp? Really?). Features, sure. Easy to use? That’s debatable. Easy to start to learn, yes. I don’t know that anyone will suggest that Perl is easy to master, though I’m happy to argue that its learning curve is gentle if long.

… languages like python with rigid syntax structure have arguably gained ground in recent times over perl, for applications that are developed collaboratively.

“Arguably” is a weasel word, so you can throw out this whole sentence. I’m not aware of any statistics that show that Python is more popular than Perl. (Arguably, the Maginot Line gained a lot of ground in the southward direction. There. Now Python fans and the French can berate me in the comments.)

Additionally, scripting languages specially-made for use on the Web, like PHP and Ruby, have eroded some of perl’s once formidable share of the dynamic Web server scripting scene.

There’s that “dynamic scripting” mess again. What does that mean? What’s static Web server scripting anyway, and why would you need a programming language for that?

My favorite part however isn’t about Perl at all. Did you catch that? Apparently Matz was really busy in 1993 writing Ruby not as a general purpose language but specifically to use on the nascent Web. How prescient.

If a journalist can rephrase a press release and make this many errors in five paragraphs in a subject I know something about, how many errors are there in subjects I don’t know as well?

Oh, and the title comes from the last television news promo I ever watched, during the X-Files finale. The local Fox affiliate played a blurb for the evening news where the newsbimbo said, and I am not making this up, “Now that the X-Files is ending, let’s see what the series taught us about real aliens. Stay tuned at 10.”

Noah Gift

AddThis Social Bookmark Button

I haven’t read many RSS feeds lately due to time constraints from writing a book, but I came across this post by Adam Gomaa, by way of some of the responses from Jonathan LaCour’s blog.

If there is one thing to learn from the endless, and pointless, comparisons of frameworks in Python, i.e, popularity contests, it is that the most important things for a web framework’s popularity, in order, are:

1. Documentation
2. Marketing

In a popularity contest, the “best” framework, is going to have the best documentation, and the best marketing, like tons of screencasts, etc. Every other discussion is an exercise in futility. If you want to be home coming queen, people have to know who you are, or they won’t vote for you. One of the reasons why Django is considered the most “popular” or “best” framework to many people, is that the Django people did an incredible job of documentation, and marketing.

While Adam brings up some very valid points in his criticisms of Turbogears and Pylons, he ultimately misses out on the real problem and the real solution. While I also like to find reasons to stretch my intellectual muscles and use legendary books to compare and contrast ideas against real world problems, it doesn’t work for this comparison.

The real problems doesn’t involve fancy computer science terminology, or “Conceptual Integrity”, it involves something much more mundane…..documentation. It is a known fact, that smart people, often the very smartest, don’t like to document their work, as they are too busy “inventing”, and being mad scientists. I am quite certain, that core Turbogears and Pylons developers, and power users, like Bob Ippolito, know how to do things that are unbelievable, but creating bulletproof documentation is tough work. Documentation is a job, and that is why millions of technical books are published making millions of dollars per year. If you consider documentation to be full time job, then people need to get paid to do it properly.

The reason why everyone doesn’t have killer documentation and marketing, is that it is perhaps, the most difficult part of being, “in the framework business”. Documentation is grunt work, plain and simple. To quote Bruce Lee, “I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.” In a similar sense, I fear not the framework that has 10,000 features, but the framework that documents one feature 10,000 times!

Of course, ultimately, a popularity contest is mostly meaningless for people who actually develop in a specific framework and make it their job, unless they are consultants for hire who only build a “insert framework here” websites. As anyone who has attended a 10 year high school reunion can attest, being “popular” isn’t always best, but due to human nature they will always be confused. Look at all of the “quiet”, corporations, all over the world, that just do Plone development, they are so busy making money, the framework issue must seem like background noise.

Also, it is interesting, but not surprising, that Zope 3, ZODB, Plone 3, and Grok are virtually non-existent in discussions about Python Web Development, in some circles. I wonder how anyone can feel comfortable getting into a massive diatribe on Python Web Development and completely dismiss any discussion about Zope related technology. Listen to the people who write the PEPS. If someone has gotten more than one PEP approved then their opinion is worth 1 million times the regular joe. Last time I checked there are only a handful of people who can say that, Phillip Eby, being one of them. He seems to think Zope is relevant, why don’t you?

Another troubling thing I see when Python Web Frameworks are brought up, is a lot of discussion about people writing wiki’s and blogs, etc, in the hot new framework. I suppose I should probably bring up that fact that Plone 3 is one of the most kick*** Content Management Systems on planet earth, see PyATL. If you want a blog, install Plone, if you want a wiki, install MoinMoin, or Plone. Unless you want to learn about writing a CMS or Wiki, you are essentially, self-gratifying yourself, to put it diplomatically, by recoding a solution to a problem that has already been solved.

Finally, I think WSGI is going to change everything, because it is about reusing components and products. Some of the current web framework comparisons are just silly. It would great to live in a world in which the real issues are discussed like, “My documentation kicked your documentation’s *ss!”. He who wins the documentation and marketing, wins the hearts and minds! To steal from a famous political campaign, “It is the documentation stupid!”.

Links:
Noah’s Personal Blog
OS X Automation
PyAtl


AddThis Social Bookmark Button

Doug Hellmann

AddThis Social Bookmark Button

Map files directly to memory using mmap.

Curtis Poe

AddThis Social Bookmark Button

Just before I started my job at the BBC, one of our developers committed code which reduced our test suite run time from an hour and twenty minutes down to twenty-two minutes. One of my first tasks was to improve that. However, improving performance begs the old question of “cpu or developer performance?” Both are equally important, but I’ll just talk about making the tests run faster. Right now, it looks like we’re on track to get our test suite to run in under ten minutes. Here’s how we did this.

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

In a previous article , we introduced the reduction operators (like '[*]' and '[~]') which produced list operators from infix operators (like '*' and '~').

There is a variant of the reduction operator that operates over its list argument producing
all intermediate results along with the final result of the ordinary reduction.

[\+] 1..5   # (1, 3, 6, 10, 15)

which is equivalent to

([+] 1),
([+] 1, 2),
([+] 1, 2, 3),
([+] 1, 2, 3, 4),
([+] 1, 2, 3, 4, 5)
Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

Binary '=>' is no longer just a “fancy comma”. In Perl 6, it now constructs a Pair object that can, among other things, be used to pass named arguments to functions.

my $pair = (one => 1);
$pair.isa(Pair)        # Bool::True
$pair.key              # 'one'
$pair.value            # 1
Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

We already have seen two Perl 6 meta-operators in articles of this series: namely, the
negate and the reduction operators. These are two of the five standard meta-operators of the language. What makes meta-operators interesting is how Perl automatically generates new operators from others (user-defined or builtins) with some straightforward semantics derived from the transformation of the base operators.

This time, we approach mutating operators, which are a shortcut for typical assignments where the assignment target and the first operand are the same variable.

my $s = 'foo';
$s x= 3;          # $a = 'foofoofoo'

my $x;
$x //= 'default'; # $x = 'default'
Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

And that’s time to take a look at another of the Perl 6 meta-operators: the reduction operator.

By surrounding with square brackets an (associative) infix operator, a new list operator is created.

[*] 1..10      # that's 1*2*...*10 = 10!
[~] <m oo s e> # 'moose' - [~] is basically Perl 5 join
[,] 'a'..'e'   # <a b c d e> - [,] is a list builder
Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

If you are wondering how processing the lines of a file will look in Perl 6, the answer is something like this:

my $h = open '<', $filename;

for =$h {
   ...
}

(Yes, we need error handling yet. I just ommitted the details for brevity.)

Noah Gift

AddThis Social Bookmark Button

Let’s say you customized your Zsh prompt by editing ~/.zshenv


#customize prompt:
PROMPT=$'[%n@%m][H:%B%!%b][J:%B%j%b]> '

You should see something like this:

[root@bigbadunixbox][H:2487][J:0]>

If you want to see all 2487 lines of your Zsh history buffer do this:


history 1 | less

Andy Oram

AddThis Social Bookmark Button

A modern, well-maintained web site should be valid, well-formed XML. If you want to rely on XML tools (XQuery, XSLT, etc.) for all your documentation and database access, you can now implement web sites with a new MVC framework named Flower. It hasn’t reached the 1.0 stage yet, and developer Thomas Lord warns that you’ll need help installing and building the system, but once you’ve got it going you can apply your XML tools to dynamic document creation.

Doug Hellmann

AddThis Social Bookmark Button

If you haven’t already seen it, check out Michael Trier’s new podcast “This Week in Django“. The first episode was good, so I’m looking forward to listening to the second.

Subscribed!

Doug Hellmann

AddThis Social Bookmark Button

The zipimport module can be used to import and run Python code found inside ZIP archives.

Adriano Ferreira

AddThis Social Bookmark Button

Jon Allen announced that http://perldoc.perl.org/ has been updated with Perl 5.10 documentation. This is great news as the look of the rendered PODs with syntax highlighting and many other nice effects makes me feel good.

AddThis Social Bookmark Button

Damien Seguy just sent me more stats on the versions of PHP in the wild. From their analysis of PHP Statistics for November 2007:

  • PHP 5 still vigourous, up to 26%
  • PHP 4.4.7 is the last growing PHP 4 version
  • PHP 5.2 will take over PHP 4.3 in the next months

With support for PHP 4 ending in a couple of weeks, hopefully the 50% or so of PHP installations running PHP 4 or earlier will take the opportunity to migrate. (See the PHP 4 to PHP 5 migration guide.)

In a related note, I really like the plush PHP elePHPant.

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

Perl 6 provides an operator 'X', the cross operator, which combines its list operands into a sort of cartesian product of these arguments.

1,2 X 3,4        # (1,3), (1,4), (2,3), (2,4)

1,2 X 3,4 X 5,6  # (1,3,5), (1,3,6), (1,4,5), ..., (2,4,6)
Andy Lester

AddThis Social Bookmark Button

This is a copy of the official announcement about Perl 5.10.

Today the Perl Foundation announces the release of Perl 5.10, the first major upgrade to the wildly popular dynamic programming language in over five years. This latest version builds on the successful 5.8.x series by adding powerful new language features and improving the Perl interpreter itself. The Perl development team, called the Perl Porters, has taken features and inspiration from the ambitious Perl 6 project, as well as from chiefly academic languages and blended them with Perl’s pragmatic view to practicality and usefulness.

Significant new language features

The most exciting change is the new smart match operator. It implements a new kind of comparison, the specifics of which are contextual based on the inputs to the operator. For example, to find if scalar $needle is in array @haystack, simply use the new ~~ operator:

  if ( $needle ~~ @haystack ) ...

The result is that all comparisons now just Do The Right Thing, a hallmark of Perl programming. Building on the smart-match operator, Perl finally gets a switch statement, and it goes far beyond the kind of traditional switch statement found in languages like C, C++ and Java.

Regular expressions are now far more powerful. Programmers can now use named captures in regular expressions, rather than counting parentheses for positional captures. Perl 5.10 also supports recursive patterns, making many useful constructs, especially in parsing, now possible. Even with these new features, the regular expression engine has been tweaked, tuned and sped up in many cases.

Other improvements include state variables that allow variables to persist between calls to subroutines; user defined pragmata that allow users to write modules to influence the way Perl behaves; a defined-or operator; field hashes for inside-out objects and better error messages.

Interpreter improvements

It’s not just language changes. The Perl interpreter itself is faster with a smaller memory footprint, and has several UTF-8 and threading improvements. The Perl installation is now relocatable, a blessing for systems administrators and operating system packagers. The source code is more portable, and of course many small bugs have been fixed along the way. It all adds up to the best Perl yet.

For a list of all changes in Perl 5.10, see Perl 5.10’s perldelta document included with the source distribution. For a gentler introduction of just the high points, the slides for Ricardo Signes’ Perl 5.10 For People Who Aren’t Totally Insane talk are well worth reading.

Don’t think that the Perl Porters are resting on their laurels. As Rafael Garcia-Suarez, the release manager for Perl 5.10, said: “I would like to thank every one of the Perl Porters for their efforts. I hope we’ll all be proud of what Perl is becoming, and ready to get back to the keyboard for 5.12.”

Where to get Perl

Perl is a standard feature in almost every operating system today except Windows. Users who don’t want to wait for their operating system vendor to release a package can dig into Perl 5.10 by downloading it from CPAN, the Comprehensive Perl Archive Network, at http://search.cpan.org/dist/perl/, or from the Perl home page at www.perl.org.

Windows users can also take advantage of the power of Perl by compiling a source distribution from CPAN, or downloading one of two easily installed binary distributions. Strawberry Perl is a community-built binary distribution for Windows, and ActiveState’s distribution is free but commercially-maintained. ActiveState’s distribution is available now, and Strawberry Perl’s is imminent.

Noah Gift

AddThis Social Bookmark Button

The more I write WSGI code, and read WSGI posts like this one on the Repoze blog, the more I wonder if Phillip Eby didn’t invent Python’s own Howard Roark in WSGI, when he wrote PEP 333.

Zope Corporation was founded in 1995, according to the, about Zope,page on their website. This means there are a heck of a lot of Python programmers who know a heck of a lot about Python Web Development. If you look at say, Ruby on Rails, which deserves a great deal of respect for what it has accomplished, you will notice it was released to the public in 2004, there is about a decade of experience difference there. In terms of computer science, this is an eternity.

Python now has mod_wsgi, a Zope TM that works inside of WSGI with any other WSGI application, people that spend every waking minute developing an ORM, and new hybrid, WSGI specific, Python web frameworks like Pylons,and,Grok. Not to mention incredibly mature full web applications, like Plone 3.0. Let’s also not forget Deliverance, which makes “skinning” multiple WSGI applications, and/or products like Plone trivial. In plain english, all of this stuff works together! As Ian Bicking explains, it is just a bunch of tubes. At this EXACT moment, people have completed the rest of the tubes, that connect the rest of the technologies in Python!

Python Web development reminds me of a very large battleship, which was slowly turning toward the shore, all the while loading the guns with large shells. Well, the ship has turned, the guns are loaded, and now, it is time to see what Python web development can really do in 2008. The perceived weakness of many different frameworks in Python, has now turned into a strength with WSGI, and these host of other technologies all working in harmony. Plus, lets not forget this is Python, perhaps, the most human readable language in existence.

Python has the potential to become the dominant Web Development language in the next couple of years, due to its massive, yet growing, pool of experienced developers, WSGI, the incredible, battle tested Standard Library, tested and proven scalability, with a plethora of concurrency solutions, readability, and outstanding leadership of the core developers. I suppose, armed with knowledge of these new developments, I wonder why anyone would not use Python Web Development for projects in the next two years?

Update: Here is the PyAtl/Repoze Talk, which discusses WSGI/Repoze/Deliverance.

Update: To see why Plone is such a big deal, watch this link, Better Web Application Development, and why having Plone combined with WSGI is almost an unfair advantage for Python

Matthew Russell

AddThis Social Bookmark Button

After a batch of feedback about the way that our previous Short Cut on Dojo was titled and marketed, we decided to update it for version 1.0 (the previous version targeted 0.9), make the title more descriptive of the Short Cut’s focus on creating custom widgets, and try it all again.

Here’s the link to the updated Short Cut.

One caveat is that the URLs for the code examples in it point to version 1.0.0 of the code on AOL’s Content Delivery Network. The latest Dojo build features the 1.0.2 code (a significant bug fix release). While the example code should work the same way either way, you’ll want to use the 1.0.2 code in any actual development you do over the CDN.

For those who haven’t heard of Dojo, it’s a fantastic JavaScript toolkit that you really don’t want to live without if you are a web developer in this day and age. In addition to providing facilities that comprise a JavaScript standard library, you also get a library of amazing out-of-the-box widgets and build tools. You can read a short ONLamp article about it here if you’re looking for a drive-by overview.

Also, stay turned for the upcoming book that’ll be available early next year.

PS - What more could someone possibly ask for on Christmas morning than a Short Cut on Dojo? It makes a great stocking stuffer :)

AddThis Social Bookmark Button

I was just commiserating with David Wheeler about a problem he had with mod_ssl on Mac OS X. The .dylib extension on shared libraries seems “arbitrary” in his words, and I’ve wrestled with it in cross-platform code a few times myself.

Then I realized where so many of my frustrations with Mac OS X came from as a developer:

Unix circa 1986 via NeXT is different from Unix circa 1998 or 2008 via Linux.

I’m happy to stick with POSIX when I want software to run somewhere outside of the nice cozy GNU/GCC/Linux/glibc universe, but those NS* functions just don’t quite feel right, you know?

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

The syntax of an if-then-else expression in Perl 6 is composed by the conditional operator.

say "My answer is: ", $maybe ?? 'yes' !! 'no';

The expression above is equivalent to that, which uses the if-then-else statement within a do.

say "My answer is: ", do {
    if $maybe {
        'yes';
    }
    else {
        'no';
    }
};
Andy Oram

AddThis Social Bookmark Button

The Peer to Patent project (which I’ve reported on before) just pointed me to a particularly broad patent that could encumber user interfaces on the web and desktops for years to come.

The idea in this patent, submitted by Yahoo!, is a clever little idea: if someone is starting to drag an icon or mail message somewhere, why not bring the mountain to Mohammed, so to speak? If she is dragging a photo, for instance, the browser or operating system can guess that she wants to open it with PhotoShop or the Gimp, and present that choice right next to the icon for the photo.

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

The design of Perl 6 includes some unification mechanisms to bring some extra power to syntax and developers. One of such artefacts is the notion of meta-operators. With them, it is possible to construct augmented operators from existing ones.

The first of such meta-operators we’re going to look at is the quite simple '!' negation prefix.

AddThis Social Bookmark Button

As you may know already, Perl 5.10 came out today. Today is also Perl’s 20th anniversary (see also Perl Simplifies the Labyrinth that is Programming Language — and to be fair, Perl doesn’t include David Bowie in extra eye makeup and tight pants).

You might not know that the Parrot porters have released Parrot 0.5.1. It’s pure happy coincidence that the monthly release cycle of Parrot coincides with Perl’s 20th anniversary, but in honor of the occasion, there’s a nice puny easter egg in this release that might bring back a stab of nostalgia. It’s an interesting comparison of how far the language and platform have evolved in two decades. Oh yes, and the Parrot tarball also contains an implementation of Perl 6 which has made tremendous progress in the past month.

brian d foy

AddThis Social Bookmark Button

Perl 1 was released to the public by Larry Wall 20 years ago today. To celebrate, Perl5Porters have released Perl5.10, the latest stable version of Perl 5. Happy Birthday Perl!

Perl 5.10 isn’t just a bug fix version: it’s full of new features that I’m eager to use: named captures in regular expressions, state variables for subroutines, the defined-or operator, a switch statement (called given-when, though), a faster regex engine, and more. You can read more about the changes in perldelta.

The perl-5.10.0.tgz file is making its way to all of the CPAN mirrors, but if you can’t wait for that, you can its torrent file. Once it makes it to the CPAN mirrors, it will the new stable.tar.gz

This time around, Perl 5.10 installation will work the same on unix and Windows: Strawberry Perl is a Perl distribution for Windows that comes with a C-compiler and everything else you need to do it yourself. Give it a couple of days to catch up, though.

Noah Gift

AddThis Social Bookmark Button

I setup trac 0.11dev last night on CentOS 5 to manage the review process for our book, using svn 1.4.5 and python 2.4.3. It was very nice! The new admin interface is great, and allows you to easily load plugins:

One gotcha, that I always forget is, if you setup tracd running behind an Apache rewrite you need to setup the trac.ini file to have both:


base_url = http://trac.example.com
use_base_url_for_redirect = True

And your rewrite stanza should look like this:



    ServerAdmin trac@example.com
    ServerName trac.example.com
    RewriteEngine on
    RewriteRule ^(.*) http://192.168.1.1:8000$1 [P]

Just a note, it is not 100% beta yet, but from the traffic on the list, it appears it should be beta any day now. Nice work Trac team…this rocks!

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

In Perl 6, you may construct ranges with expressions like

$min  ..  $max
$min ^..  $max
$min  ..^ $max
$min ^..^ $max

and even

^$limit

AddThis Social Bookmark Button

If you read my feed, you probably saw that I changed the name of the project mentioned in SAP’s Composition on Grails. After I published that, my contacts at SAP said “You know, we had been meaning to change the name.”

If you’re interested in the project, you should definitely follow Will Gardella’s weblog, starting with Composition on Grails - New Version, New Name. He answers some of the most common questions he’s heard over the past few months about the project.

I realize that SAP doesn’t want to offer official support for languages other than Java and ABAP, but every demo I saw there with interaction between services over WS-* makes me think that the company could spend $50k in tech marketing very well by posting bounties with the appropriate F/OSS communities to beef up WSDL and SOAP support in languages such as Perl, Python, Ruby, and PHP such that they can be first-class consumers of SAP middleware services in the same way as Groovy now is.

Again, it doesn’t have to be official support, and it’s just the price of a full-page ad in a decently-read magazine, but it would be a shame not to take advantage of these rich ecosystems with an architecture that already supports such a thing….

Adriano Ferreira

AddThis Social Bookmark Button

Another article of the series “Yet Another Perl 6 Operator”

Among the new Perl 6 operators, there is the handy operator '//', known as defined-or or the default operator. This novelty was anticipated by the introduction of this syntactic bit in Perl 5 (see the upcoming 5.10 release) — so you won’t need to wait for Perl 6 to start using it.

# dor.pl
use 5.010;
print "arg: '", shift // "?", "'\n";

$ perl dor.pl one
arg: 'one'
$ perl dor.pl ""
arg: ''
$ perl dor.pl
arg: '?'
Noah Gift

AddThis Social Bookmark Button

Most people forget that python is both procedural and Object Oriented. If you find yourself wanting to use a procedural, Bash style of programming with Python, take a look at the global statement.

It lets you do something like this, because without the :


In [34]: x = 1; y = 2

In [35]: def func():
   ....:     global x, y
   ....:     print "X = %s, Y = %s, in func" % (x,y)
   ....:
   ....:     

In [36]: func()
X = 1, Y = 2, in func

Here is an even better example, based on the comment by Ant, as the top example was too vague, probably because I was trying to make the example too "tiny" :) That is a challenge, to make a tip tiny, yet explicit.


In [46]: x = 1; y = 2; z = "out here"

ln [47] def func():
   ....:         global x,y,z
   ....:         print "X = %s, Y = %s, in func.  Z says: %s " % (x,y,z)
   ....:         x = 1000;y=1000;z="in here"
   ....:         print "X = %s, Y = %s, in func.  Z says: %s " % (x,y,z)

In [48]: func()
X = 1, Y = 2, in func.  Z says: out here
X = 1000, Y = 1000, in func.  Z says: in here 

I should also say, thanks to Tres, who showed me this tip to begin with!

Noah Gift

AddThis Social Bookmark Button

The last 7 days, have been very crazy for me. Last weekend, I ended up in Los Angeles, for a couple days, and was able to grab some Cuban Pastries:

Before hopping back on a plane for Atlanta:

Back in Georgia, I felt very much at home when I spotted some familiar local scenery, a guy with a mullet in a camaro (with a killer “G-Force” bumper sticker), next to a man in a costume in the middle of traffic, asking for money:

Next, it was a Tuesday visit to the monthly Atlanta-Plone meeting. Where we discussed the upcoming Repoze Sprint/Visit:

On Thursday, we met with Tres and Chris, who happened to write supervisor, and they gave a tremendous talk on WSGI, Repoze, and Deliverance, that blew the PyAtl crowd away.

One of the more dramatic, show and tell, pieces, was a local demonstration of their “theme trac like Plone trick”. The crowd was blown away, when Tres and Chris stole, borrowed, pick your favorite word, the pyatl plone 3.0 site, and themed a localhost trac instance. We also saw a great debugging middleware WSGI tool, that “leaked” objects in the WSGI stack. WSGI is truly an incredible technology, and I am so excited about it, I almost can’t sleep.

Next on Friday, we hunkered down at Georgia Tech, and starting playing with Repoze a little more:

One silly idea that came up after a few beers at lunch, was writing the simplest possible WSGI application using the WSGI spec from Pep 333. By using Ian Bicking’s pythonpaste, Tres was able to walk me through setting up the most simple possible WSGI application. We used string substitution and pickle, and gave birth to A******Glue, AGlue, for short. AGlue is just a proof of concept, with a funny name.

If you are use virtualenv, and pythonpaste, it is quite simple to make a little web application using WSGI. You really only need to create an /etc directory in your virtualenv, that includeds a .ini file, such as this:

Step 1: Create a .ini file


[server:main]
use = egg:Paste#http
host = 127.0.0.1
port = 8080

[app:aglue]
paste.app_factory = aGlue.app:factory
path = %(here)s/../var

[pipeline:main]
pipeline = egg:Paste#evalerror aglue

Step 2: Next make some simple model.py like this:


class Book(object):
    """A book object"""

    def __init__(self, ISBN, title, reviewer=None):
        self.ISBN = ISBN
        self.title = title
        self.reviewer = reviewer

Step 3: Finally, make a app.py, or controller:



import os
import pickle
from paste.request import parse_formvars

from model import Book
template = """
<html>
<body>
<p>

<a href = "http://pyatl.org">pyatl.org</a>
</p>
"""
row = """
<p>%(title)s <form method="post"><input type="hidden" name="index" value = "%(index)d">
<input type = "submit" name = "delete" value="delete">
</form>
</p>
"""

epilogue="""

<form method="post">
<input type="text" name = "ISBN">
<input type="text" name = "title">
<input type="submit" name = "submit" value = "add">
</body></html>
"""
def middleFinger(environ, start_response):
    """Why did you use this, punk?
    """
    form = parse_formvars(environ)
    if form:
        if 'submit' in form:
            book = Book(ISBN=form['ISBN'],title=form['title'])
            books.append(book)
            saveList()
        elif 'delete' in form:
            index = int(form['index'])
            del books[index]
            saveList()
    print form
    page = [template]
    for index in range(len(books)):
        book = books[index]
        page.append(row%{'index':index,'title':book.title})
    page.append(epilogue)
    status = '200 OK'
    response_headers = [('Content-type', 'text/html')]
    start_response(status, response_headers)
    return [''.join(page)]

def saveList():
    file = open('/tmp/persistant.db', 'w')
    pickle.dump(books,file)
    file.close()

def factory(global_config, persist = '/tmp/persistant.db',**local_config):
    global books
    books = []
    if not os.path.exists(persist):
        saveList()
    else:
        file = open(persist)
        books = pickle.load(file)
    return middleFinger

With that little bit of code, you get something like this:

One thing I learned from the last few days, is that Ian Bicking is amazing! Between virtualenv, and pythonpaste alone, it is an incredible, how many tools he creates to help other Python programmers. Tres and Chris, are also equally amazing, and I would recommend trying to get them to come to your local user group for a Repoze/Deliverance talk too!

There will be a video posted this week of their talk this week on YouTube, and I will also upload a more refined version of AGlue, to the cheeseshop in a few days.

Doug Hellmann

AddThis Social Bookmark Button

The zipfile module can be used to manipulate ZIP archive files.

Jeremy Jones

AddThis Social Bookmark Button

First off, the “I” in the title does not refer to me; it refers to my coworker Jenny Walsh. Jenny was recently setting up a new Macbook Pro with Leopard on it and ran into a snag with PIL. Running setup.py for PIL for the first time said that she had no jpeg support. So, she installed libjpeg and tried again. This time, it said that she had jpeg support, but then it threw this error:

ld: in /Developer/SDKs/MacOSX10.4u.sdk/usr/local/lib/libJPEG.dylib, file is not of required architecture for architecture ppc

She tried compiling libjpeg and followed suggestions she had found from across the interweb including compiling as a “fat binary” (intel and ppc), but she still had the same level of unsuccess as before. Finally, in a fit of hacking frenzy, she removed all “-arch ppc ” from
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/config/Makefile,
re-ran setup.py for PIL, and all was right with her world again.

In trying to diagnose and help her with her problem, I stumbled across
this
blog post
. They didn’t seem to have any trouble getting PIL and libjpeg installed on leopard. The compiled libjpeg as follows:

wget http://www.ijg.org/files/jpegsrc.v6b.tar.gz
tar zxvf jpegsrc.v6b.tar.gz
cd jpeg-6b/
./configure
make
sudo make install-lib

I tried the same thing on my laptop (which runs Tiger and not Leopard, btw), and it worked for me. But this just took my PIL installation from saying that it did not have jpeg support to saying that it did. Jenny was having a different problem. So, if you are having a problem with PIL blowing up when trying to set it up, you may want to give Jenny’s solution a spin. But make sure to make a backup copy of the Makefile.

Jeremy Jones

AddThis Social Bookmark Button

I’ve been wondering that lately. I’ve been using Subversion for … well … what seems to be a lot of years now. Looking back at the dates for Subversion’s history and coinciding them with events that were happening in my life, I’m guessing that I started using Subversion no later than the end of 2003. (So, maybe that’s not “a lot of years now”…) And I was using CVS from about 2001 until I started with Subversion. During the majority of that time, I have been either the sole commiter of the code base that I was working on, or one of very few people working on the same code.

Recently, I started a job where I’ll likely be working more mingled in with other developers on the same code at the same time. Everything is set up using Subversion. Before starting this new job, though, I began looking into distributed source control, which is the cool new kid on the block. I’ve created some personal projects using Bazaar and have glanced at Darcs, Mercurial, and Git. I like Bazaar a lot. It can be a little sluggish at times (like pushing, pulling, and merging), but not unbearably so - and I expect that it’ll get better. I keep running through my mind how moving to a distributed model would impact the work flow with my co-workers and I’m not totally convinced that distributed is always the way to go.

I just finished reading this piece of a conversation with Linus Torvalds regarding Git, and I remain unconvinced that going distributed would be the best thing for us. And I’m guessing that maybe most small teams of “closed” development probably don’t need a distributed source control system, either. It seems that the problems that have spawned this new model of source control is more of a problem for open source development, particularly of larger projects, and less of a problem for smaller proprietary development. For example, it’s really important for Linus that Linux kernel developers (or anyone, really) be able at any moment to create a new branch. It’s important for Linux that people be able to experiment with new kooky ideas and maybe come up with a cool new feature to go into the kernel. It’s also important to Linus that people be able to do so in anonymity. I can see how this would be important for a project that has potentially tens of thousands of developers interested in experimenting with the code and are doing so on their own free time. I think this is less important when a small team is being paid to work on a code base. Typically, you don’t have the leisure time to perform experiments. If you do need to work on an experimental feature set, it’s not a problem for a repository admin at work to create a branch for you. And anonymity isn’t typically necessary at work. At least, not anywhere I’ve ever worked.

I can see how having an “off-line” repository could be helpful. But from what I’ve heard, svk should address a lot of those issues. And most of these distributed systems are reputed to handle merging between branches better, which would be nice. I love Bazaar and would love to use it at work, but I’m just not convinced that it buys us enough benefits to switch from Subversion. Does anyone have convincing reasons that a small, closed source development team should consider switching to a distributed tool?

AddThis Social Bookmark Button

Perl 5.10 will be out soon — within a week — and pumpking Rafael Garcia-Suarez will likely release a final release candidate for final testing and polishing. If you have Perl code you care about, it’s worth downloading RC3 to see how well things run with the new version.

If you have a platform more exotic than GNU/Linux or FreeBSD on x86, getting pass or fail results would be wonderful too.

Here’s how.

AddThis Social Bookmark Button

Mike Shaver has a deconstruction of false statements from Adobe about Flex’s openness. In particular:

[Adobe evangelist James Ward] has the nerve to call them “the community” and indicate that their work is a remedy for Adobe simply not being willing to remove the field-of-use restrictions on their existing documentation.

… and:

… they don’t want people to think too hard about the fact that writing to Flash is committing yourself to proprietary platform…

Maybe Monopolight will save the free software/free data/free community communities. (Why Monopolight? There’s a single vendor for licensed Silverlight codecs, Moonlight doesn’t provide the Silverlight codecs, and they’re x86 binary blobs only. Gee. Thanks.)

Adriano Ferreira

AddThis Social Bookmark Button

The series “Yet Another Perl 6 Operator” is back with this brand new article

In the article on coercion operators, we got to know the prefix operator '?' which converts values into Bool::True or Bool::False. Like it happens with '~' for strings, '?' is recurrent for boolean operators.

In Perl 6, the usual infix boolean operators are:

?& - and
?| - or
?^ - xor

AddThis Social Bookmark Button

I was in Silicon Valley last week, and caught part of a demo of Composition on Grails. This is a project within SAP built on top of SAP NetWeaver Composition Environment.

I decided to change my plans, and had lunch with Will Gardella and his team. (Will gave the demo, and his team is building Composition on Grails within SAP.)

I’d never considered SAP or its business very interesting, because I misunderstood what SAP actually does. Though I very happily left the world of the Enterprise (twice, first as a system adminstrator and then as a consultant who wrote and maintained business software), and though SAP’s choice of technologies (ABAP, Java, WSDL, WS-*, shared-everything) are different from my preferences, it took just a few minutes for Will’s demo to convince me that I had overlooked something very interesting.

Noah Gift

AddThis Social Bookmark Button

Want to control Active Directory with Python? Well now you can, Python-AD was just released over the weekend.

AddThis Social Bookmark Button

There’s a lot of speculation at any given time about what the future of the web is. I don’t know what it is, but I think I know what it’s not. Facebook.

Yeah, yeah, I can hearing the booing from here. Easy now, fan boys.

The truth is that the web rides waves of innovation and everyone on of those waves has a trough between it and the next wave. Facebook is currently experiencing the slide down the backside of the wave and is headed into the trough. Why?

Doug Hellmann

AddThis Social Bookmark Button

The BaseHTTPServer module includes classes which can form the basis of a web server.

Jeremy Jones

AddThis Social Bookmark Button

From Guido’s blog, Python 3.0alpha2 has just been released. And here is the release page. Good work folks!

Curtis Poe

AddThis Social Bookmark Button

Before you dash of a nasty reply, hear me out. There’s an excellent lesson that all companies can learn from this.

By now I suspect that many of you have already heard of Perl On Rails, an internal BBC project. Naturally, this made Slashdot, Reddit and Digg, amongst many other sites and one chap put up a very scathing post entitled Why the BBC Fails at the Internet. Most of the comments you’ll read about the “Perl on Rails” project are pretty far off the mark, but the “Fails at the Internet” post was, despite the vitriol, probably the most spot-on analysis of the problem facing the BBC.

Admittedly, I’ve only worked for the BBC for a month, but I’ve already had several friends ask me for the “inside dope” about this project and I wouldn’t say much for two reasons. One, I had only heard about it second hand and was concerned about getting technical details wrong. Two, I live in the perpetual fear that anything I say in my blog will be held against me. I’m probably right on the first count, but I’m definitely wrong on the second.

Doug Hellmann

AddThis Social Bookmark Button

Leslie Hawthorn, Program Manager on the Open Source Team at Google and lead wrangler for GHOP has posted a video summary of the status one week into the contest, including some feedback from Guido.

AddThis Social Bookmark Button

Patrick Michaud is truly earning the Mozilla Foundation’s Perl 6 on Parrot Grant. He’s just published a Perl 6 on Parrot Roadmap for December 4, 2007, which describes the progress the project has made in the past couple of weeks.

In particular, he and Jerry Gay just switched the Perl 6 compiler in languages/perl6/ from the old compiler toolkit to the new compiler toolkit. This sounds like a small thing, but it’s actually a very large step.

Jeremy Jones

AddThis Social Bookmark Button

Here is a pro-Python propaganda comic. If you’re in to Perl, don’t take the alt tag (by hovering over the image) personally - it’s an xkcd comic. You know, the same comic that depicted Stallman as a ninja? Enjoy! (I know I did.)

Noah Gift

AddThis Social Bookmark Button

As Jeremy and I get more down the road of finishing our book on Python for *NIX systems administration, working title, we are both going to start dishing out some meatier nuggets of Python that we expose in our book. I have a background in Feature Film Animation Pipelines, and I thought I would share an interesting Python module that Python programmers in the Animation world might enjoy. The module is called pyinotify, and it “monitor’s filesystem events with Python under Linux”. Sound cools right, well, it is, so lets get a little background.

Inotify went into the linux kernel in release 2.6.13, and, according to Wikipedia, “Inotify uses an API that uses minimal file descriptors, allowing programmers to use the established select and poll interface”, in plain english it notices changes to the filesystem and reports those changes to applications. In animation pipelines, generally, files need to make a round trip between CGI, and Editorial. The Animators are given a scene to create, and very large, raw image files, like 16 bit TIFFs, are created. The files generally need to be processed in many different ways depending on where those images need to go. They may need to be converted into a HD Quicktime Movie for viewing on a 2K or 4K, stands for thousands of pixels, digital projector, or they may need to be converted into a compressed format like MXF, or DNxHD, a variant on the MXF file format that Avid has developed. If the files are to get sent to the editorial department for editing, then they will need to be processed with a timecode, or keykode, value embedded into the metadata of the file.

Embedding either a timecode, or keykode, is mandatory, as it allows image files to be assembled automatically according to a shot list, or else they would need to be visually arranged by hand, which just wouldn’t work in an animation pipeline. One of the reasons why it wouldn’t work, is that the image files could be cut out of a scene, and it would be an incredible waste of money to animate a frame or many frames, that were cut out in editorial. This is just one of the reasons why embedded metadata in files is so important in animation. Of course things need to be kept track of, and that is why using something like SQLAlchemy could make a lot of sense. I am actually building a metadata management system into open source tool I am creating called Liten, and, when I get more time, it will use SQLAlchemy.

All of this rambling background material, leads me back to Python, and Pyinotify. Pyinotify, by talking the Linux Kernel API, can watch a directory or a whole filesystem, for the moment that say, a Maya Artist, has exported a sequence of frames to the “shot tree”, or file server, in plain english. At that point, when Pyinotify notices these changes, it could begun to process these files and perhaps move them to a High Speed Fibre SAN like Avid Unity, or XSAN, that is built for playing back HD media files.

Lets take a look at how that might work:

I threw this “toy code”, together in about an hour or so, and all it does is pretend to do things when a file is added to the /tmp directory and it closes. I might get around to actually making a useful tool this weekend and adding threading, etc.

If you would like to check this code out, I put up a Google Code Project here. These are some pictures of my “toy code” pretending to do things, when I create a file in a directory, I am watching. Thanks to author of pyinotify for making this so easy to work with!


import os
import sys
import optparse
from pyinotify import WatchManager, Notifier, ProcessEvent, EventsCodes

class PClose(ProcessEvent):
    """
    Processes on close event
    """

    def __init__(self, path):
        self.path = path
        self.file = file

    def process_IN_CLOSE(self, event):
        """
        process 'IN_CLOSE_*' events
        can be passed an action function
        """
        path = self.path
        if event.name:
            self.file = "%s" % os.path.join(event.path, event.name)
        else:
           self.file = "%s" % event.path
        print "%s Closed" % self.file
        print "Performing pretend action on %s...." % self.file
        import time
        time.sleep(2)
        print "%s has been processed" % self.file

class Controller(object):

    def __init__(self, path='/tmp'):
        self.path = path

    def run(self):
        self.pclose = PClose(self.path)
        PC = self.pclose
        # only watch these events
        mask = EventsCodes.IN_CLOSE_WRITE | EventsCodes.IN_CLOSE_NOWRITE

        # watch manager instance
        wm = WatchManager()
        notifier = Notifier(wm, PC)

        print 'monitoring of %s started' % self.path

        added_flag = False
        # read and process events
        while True:
            try:
                if not added_flag:
                    # on first iteration, add a watch on path:
                    # watch path for events handled by mask.
                    wm.add_watch(self.path, mask)
                    added_flag = True
                notifier.process_events()
                if notifier.check_events():
                    notifier.read_events()
            except KeyboardInterrupt:
                # ...until c^c signal
                print 'stop monitoring...'
                # stop monitoring
                notifier.stop()
                break
            except Exception, err:
                # otherwise keep on watching
                print err

def main():
    monitor = Controller()
    monitor.run()

if __name__ == '__main__':
    main()

Sound fun? if so, let me know.

Andy Oram

AddThis Social Bookmark Button

I spent yesterday at the XML 2007 conference in Boston. It’s smaller than last year’s conference, which is a shame because I liked the sessions I attended better than last year’s. The knowledge and skills of the attendees as well as the presenters seemed impressive. Here are a few musings that resulted.

Andy Oram

AddThis Social Bookmark Button

The next time you have to search for information on any topic, try recording your efforts in a survey I’ve just put up:

http://www.praxagora.com/search_survey/

Easy searches usually aren’t interesting, so I’m seeking submissions just about searches that covered three or more documents (besides search engines). Relevant searches can be done online, using print media, or both–and even other media such as radio or film.

Doug Hellmann

AddThis Social Bookmark Button

The SocketServer module is a framework for creating network servers. It provides base classes for handling TCP, UDP, Unix streams, and Unix datagrams and supports both threading and forking servers, depending on what is most appropriate for your situation.

Noah Gift

AddThis Social Bookmark Button

Need to monitor remote disk usage? Use snmpdf

1. Start and configure snmpd (edit /etc/snmp/snmpd.conf):

rocommunity superSecret
disk /

2. Run the command on a local machine and be amazed:

snmpdf -v 2c -c superSecret localhost

3. Setup and use SNMP v3 on a real machine you admin.

Noah Gift

AddThis Social Bookmark Button

Do you have a thousands of lines of debug code slowing you down…use __debug__ and optimize it out with -O command line option.

Advertisement