I’m glad to see that TPF’s public relations group has spread the Perl 5.10 press release far and wide, and it’s getting some coverage. However, some of that coverage reminds me why I don’t watch television news and why I treat the newspaper as entertainment and not information. Consider eWeek’s First Release of Perl in Five Years Arrives:

Perl is a dynamic scripting language widely used in everything from Linux system utilities to Web servers to full-blown graphical enterprise applications.

What’s a “dynamic scripting language”? Is there such thing as a non-dynamic scripting language? (No one seems to know what a scripting language is anyway.) Minor nit.

During its 20-year history, it gained massive popularity by assimilating the syntax from many predecessors, making it really easy to use for anyone already versed in sed, awk, grep, csh, C/C++, Lisp, and so on.

Syntax, maybe (but Lisp? Really?). Features, sure. Easy to use? That’s debatable. Easy to start to learn, yes. I don’t know that anyone will suggest that Perl is easy to master, though I’m happy to argue that its learning curve is gentle if long.

… languages like python with rigid syntax structure have arguably gained ground in recent times over perl, for applications that are developed collaboratively.

“Arguably” is a weasel word, so you can throw out this whole sentence. I’m not aware of any statistics that show that Python is more popular than Perl. (Arguably, the Maginot Line gained a lot of ground in the southward direction. There. Now Python fans and the French can berate me in the comments.)

Additionally, scripting languages specially-made for use on the Web, like PHP and Ruby, have eroded some of perl’s once formidable share of the dynamic Web server scripting scene.

There’s that “dynamic scripting” mess again. What does that mean? What’s static Web server scripting anyway, and why would you need a programming language for that?

My favorite part however isn’t about Perl at all. Did you catch that? Apparently Matz was really busy in 1993 writing Ruby not as a general purpose language but specifically to use on the nascent Web. How prescient.

If a journalist can rephrase a press release and make this many errors in five paragraphs in a subject I know something about, how many errors are there in subjects I don’t know as well?

Oh, and the title comes from the last television news promo I ever watched, during the X-Files finale. The local Fox affiliate played a blurb for the evening news where the newsbimbo said, and I am not making this up, “Now that the X-Files is ending, let’s see what the series taught us about real aliens. Stay tuned at 10.”

I haven’t read many RSS feeds lately due to time constraints from writing a book, but I came across this post by Adam Gomaa, by way of some of the responses from Jonathan LaCour’s blog.

If there is one thing to learn from the endless, and pointless, comparisons of frameworks in Python, i.e, popularity contests, it is that the most important things for a web framework’s popularity, in order, are:

1. Documentation
2. Marketing

In a popularity contest, the “best” framework, is going to have the best documentation, and the best marketing, like tons of screencasts, etc. Every other discussion is an exercise in futility. If you want to be home coming queen, people have to know who you are, or they won’t vote for you. One of the reasons why Django is considered the most “popular” or “best” framework to many people, is that the Django people did an incredible job of documentation, and marketing.

While Adam brings up some very valid points in his criticisms of Turbogears and Pylons, he ultimately misses out on the real problem and the real solution. While I also like to find reasons to stretch my intellectual muscles and use legendary books to compare and contrast ideas against real world problems, it doesn’t work for this comparison.

The real problems doesn’t involve fancy computer science terminology, or “Conceptual Integrity”, it involves something much more mundane…..documentation. It is a known fact, that smart people, often the very smartest, don’t like to document their work, as they are too busy “inventing”, and being mad scientists. I am quite certain, that core Turbogears and Pylons developers, and power users, like Bob Ippolito, know how to do things that are unbelievable, but creating bulletproof documentation is tough work. Documentation is a job, and that is why millions of technical books are published making millions of dollars per year. If you consider documentation to be full time job, then people need to get paid to do it properly.

The reason why everyone doesn’t have killer documentation and marketing, is that it is perhaps, the most difficult part of being, “in the framework business”. Documentation is grunt work, plain and simple. To quote Bruce Lee, “I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.” In a similar sense, I fear not the framework that has 10,000 features, but the framework that documents one feature 10,000 times!

Of course, ultimately, a popularity contest is mostly meaningless for people who actually develop in a specific framework and make it their job, unless they are consultants for hire who only build a “insert framework here” websites. As anyone who has attended a 10 year high school reunion can attest, being “popular” isn’t always best, but due to human nature they will always be confused. Look at all of the “quiet”, corporations, all over the world, that just do Plone development, they are so busy making money, the framework issue must seem like background noise.

Also, it is interesting, but not surprising, that Zope 3, ZODB, Plone 3, and Grok are virtually non-existent in discussions about Python Web Development, in some circles. I wonder how anyone can feel comfortable getting into a massive diatribe on Python Web Development and completely dismiss any discussion about Zope related technology. Listen to the people who write the PEPS. If someone has gotten more than one PEP approved then their opinion is worth 1 million times the regular joe. Last time I checked there are only a handful of people who can say that, Phillip Eby, being one of them. He seems to think Zope is relevant, why don’t you?

Another troubling thing I see when Python Web Frameworks are brought up, is a lot of discussion about people writing wiki’s and blogs, etc, in the hot new framework. I suppose I should probably bring up that fact that Plone 3 is one of the most kick*** Content Management Systems on planet earth, see PyATL. If you want a blog, install Plone, if you want a wiki, install MoinMoin, or Plone. Unless you want to learn about writing a CMS or Wiki, you are essentially, self-gratifying yourself, to put it diplomatically, by recoding a solution to a problem that has already been solved.

Finally, I think WSGI is going to change everything, because it is about reusing components and products. Some of the current web framework comparisons are just silly. It would great to live in a world in which the real issues are discussed like, “My documentation kicked your documentation’s *ss!”. He who wins the documentation and marketing, wins the hearts and minds! To steal from a famous political campaign, “It is the documentation stupid!”.

Links:
Noah’s Personal Blog
OS X Automation
PyAtl

Map files directly to memory using mmap.

Just before I started my job at the BBC, one of our developers committed code which reduced our test suite run time from an hour and twenty minutes down to twenty-two minutes. One of my first tasks was to improve that. However, improving performance begs the old question of “cpu or developer performance?” Both are equally important, but I’ll just talk about making the tests run faster. Right now, it looks like we’re on track to get our test suite to run in under ten minutes. Here’s how we did this.

Another article of the series “Yet Another Perl 6 Operator”

In a previous article , we introduced the reduction operators (like '[*]' and '[~]') which produced list operators from infix operators (like '*' and '~').

There is a variant of the reduction operator that operates over its list argument producing
all intermediate results along with the final result of the ordinary reduction.

[\+] 1..5   # (1, 3, 6, 10, 15)

which is equivalent to

([+] 1),
([+] 1, 2),
([+] 1, 2, 3),
([+] 1, 2, 3, 4),
([+] 1, 2, 3, 4, 5)

Another article of the series “Yet Another Perl 6 Operator”

Binary '=>' is no longer just a “fancy comma”. In Perl 6, it now constructs a Pair object that can, among other things, be used to pass named arguments to functions.

my $pair = (one => 1);
$pair.isa(Pair)        # Bool::True
$pair.key              # 'one'
$pair.value            # 1

Another article of the series “Yet Another Perl 6 Operator”

We already have seen two Perl 6 meta-operators in articles of this series: namely, the
negate and the reduction operators. These are two of the five standard meta-operators of the language. What makes meta-operators interesting is how Perl automatically generates new operators from others (user-defined or builtins) with some straightforward semantics derived from the transformation of the base operators.

This time, we approach mutating operators, which are a shortcut for typical assignments where the assignment target and the first operand are the same variable.

my $s = 'foo';
$s x= 3;          # $a = 'foofoofoo'

my $x;
$x //= 'default'; # $x = 'default'

Another article of the series “Yet Another Perl 6 Operator”

And that’s time to take a look at another of the Perl 6 meta-operators: the reduction operator.

By surrounding with square brackets an (associative) infix operator, a new list operator is created.

[*] 1..10      # that's 1*2*...*10 = 10!
[~] <m oo s e> # 'moose' - [~] is basically Perl 5 join
[,] 'a'..'e'   # <a b c d e> - [,] is a list builder

Another article of the series “Yet Another Perl 6 Operator”

If you are wondering how processing the lines of a file will look in Perl 6, the answer is something like this:

my $h = open '<', $filename;

for =$h {
   ...
}

(Yes, we need error handling yet. I just ommitted the details for brevity.)

Let’s say you customized your Zsh prompt by editing ~/.zshenv

#customize prompt:
PROMPT=$'[%n@%m][H:%B%!%b][J:%B%j%b]> '

You should see something like this:

[root@bigbadunixbox][H:2487][J:0]>

If you want to see all 2487 lines of your Zsh history buffer do this:

history 1 | less

A modern, well-maintained web site should be valid, well-formed XML. If you want to rely on XML tools (XQuery, XSLT, etc.) for all your documentation and database access, you can now implement web sites with a new MVC framework named Flower. It hasn’t reached the 1.0 stage yet, and developer Thomas Lord warns that you’ll need help installing and building the system, but once you’ve got it going you can apply your XML tools to dynamic document creation.

If you haven’t already seen it, check out Michael Trier’s new podcast “This Week in Django“. The first episode was good, so I’m looking forward to listening to the second.

Subscribed!

The zipimport module can be used to import and run Python code found inside ZIP archives.

Jon Allen announced that http://perldoc.perl.org/ has been updated with Perl 5.10 documentation. This is great news as the look of the rendered PODs with syntax highlighting and many other nice effects makes me feel good.

Damien Seguy just sent me more stats on the versions of PHP in the wild. From their analysis of PHP Statistics for November 2007:

• PHP 5 still vigourous, up to 26%
• PHP 4.4.7 is the last growing PHP 4 version
• PHP 5.2 will take over PHP 4.3 in the next months

With support for PHP 4 ending in a couple of weeks, hopefully the 50% or so of PHP installations running PHP 4 or earlier will take the opportunity to migrate. (See the PHP 4 to PHP 5 migration guide.)

In a related note, I really like the plush PHP elePHPant.

Another article of the series “Yet Another Perl 6 Operator”

Perl 6 provides an operator 'X', the cross operator, which combines its list operands into a sort of cartesian product of these arguments.

1,2 X 3,4        # (1,3), (1,4), (2,3), (2,4)

1,2 X 3,4 X 5,6  # (1,3,5), (1,3,6), (1,4,5), ..., (2,4,6)

This is a copy of the official announcement about Perl 5.10.

Today the Perl Foundation announces the release of Perl 5.10, the first major upgrade to the wildly popular dynamic programming language in over five years. This latest version builds on the successful 5.8.x series by adding powerful new language features and improving the Perl interpreter itself. The Perl development team, called the Perl Porters, has taken features and inspiration from the ambitious Perl 6 project, as well as from chiefly academic languages and blended them with Perl’s pragmatic view to practicality and usefulness.

Significant new language features

The most exciting change is the new smart match operator. It implements a new kind of comparison, the specifics of which are contextual based on the inputs to the operator. For example, to find if scalar $needle is in array @haystack, simply use the new ~~ operator:

  if ( $needle ~~ @haystack ) ...

The result is that all comparisons now just Do The Right Thing, a hallmark of Perl programming. Building on the smart-match operator, Perl finally gets a switch statement, and it goes far beyond the kind of traditional switch statement found in languages like C, C++ and Java.

Regular expressions are now far more powerful. Programmers can now use named captures in regular expressions, rather than counting parentheses for positional captures. Perl 5.10 also supports recursive patterns, making many useful constructs, especially in parsing, now possible. Even with these new features, the regular expression engine has been tweaked, tuned and sped up in many cases.

Other improvements include state variables that allow variables to persist between calls to subroutines; user defined pragmata that allow users to write modules to influence the way Perl behaves; a defined-or operator; field hashes for inside-out objects and better error messages.

Interpreter improvements

It’s not just language changes. The Perl interpreter itself is faster with a smaller memory footprint, and has several UTF-8 and threading improvements. The Perl installation is now relocatable, a blessing for systems administrators and operating system packagers. The source code is more portable, and of course many small bugs have been fixed along the way. It all adds up to the best Perl yet.

For a list of all changes in Perl 5.10, see Perl 5.10’s perldelta document included with the source distribution. For a gentler introduction of just the high points, the slides for Ricardo Signes’ Perl 5.10 For People Who Aren’t Totally Insane talk are well worth reading.

Don’t think that the Perl Porters are resting on their laurels. As Rafael Garcia-Suarez, the release manager for Perl 5.10, said: “I would like to thank every one of the Perl Porters for their efforts. I hope we’ll all be proud of what Perl is becoming, and ready to get back to the keyboard for 5.12.”

Where to get Perl

Perl is a standard feature in almost every operating system today except Windows. Users who don’t want to wait for their operating system vendor to release a package can dig into Perl 5.10 by downloading it from CPAN, the Comprehensive Perl Archive Network, at http://search.cpan.org/dist/perl/, or from the Perl home page at www.perl.org.

Windows users can also take advantage of the power of Perl by compiling a source distribution from CPAN, or downloading one of two easily installed binary distributions. Strawberry Perl is a community-built binary distribution for Windows, and ActiveState’s distribution is free but commercially-maintained. ActiveState’s distribution is available now, and Strawberry Perl’s is imminent.

The more I write WSGI code, and read WSGI posts like this one on the Repoze blog, the more I wonder if Phillip Eby didn’t invent Python’s own Howard Roark in WSGI, when he wrote PEP 333.

Zope Corporation was founded in 1995, according to the, about Zope,page on their website. This means there are a heck of a lot of Python programmers who know a heck of a lot about Python Web Development. If you look at say, Ruby on Rails, which deserves a great deal of respect for what it has accomplished, you will notice it was released to the public in 2004, there is about a decade of experience difference there. In terms of computer science, this is an eternity.

Python now has mod_wsgi, a Zope TM that works inside of WSGI with any other WSGI application, people that spend every waking minute developing an ORM, and new hybrid, WSGI specific, Python web frameworks like Pylons,and,Grok. Not to mention incredibly mature full web applications, like Plone 3.0. Let’s also not forget Deliverance, which makes “skinning” multiple WSGI applications, and/or products like Plone trivial. In plain english, all of this stuff works together! As Ian Bicking explains, it is just a bunch of tubes. At this EXACT moment, people have completed the rest of the tubes, that connect the rest of the technologies in Python!

Python Web development reminds me of a very large battleship, which was slowly turning toward the shore, all the while loading the guns with large shells. Well, the ship has turned, the guns are loaded, and now, it is time to see what Python web development can really do in 2008. The perceived weakness of many different frameworks in Python, has now turned into a strength with WSGI, and these host of other technologies all working in harmony. Plus, lets not forget this is Python, perhaps, the most human readable language in existence.

Python has the potential to become the dominant Web Development language in the next couple of years, due to its massive, yet growing, pool of experienced developers, WSGI, the incredible, battle tested Standard Library, tested and proven scalability, with a plethora of concurrency solutions, readability, and outstanding leadership of the core developers. I suppose, armed with knowledge of these new developments, I wonder why anyone would not use Python Web Development for projects in the next two years?

Update: Here is the PyAtl/Repoze Talk, which discusses WSGI/Repoze/Deliverance.

Update: To see why Plone is such a big deal, watch this link, Better Web Application Development, and why having Plone combined with WSGI is almost an unfair advantage for Python

After a batch of feedback about the way that our previous Short Cut on Dojo was titled and marketed, we decided to update it for version 1.0 (the previous version targeted 0.9), make the title more descriptive of the Short Cut’s focus on creating custom widgets, and try it all again.

Here’s the link to the updated Short Cut.

One caveat is that the URLs for the code examples in it point to version 1.0.0 of the code on AOL’s Content Delivery Network. The latest Dojo build features the 1.0.2 code (a significant bug fix release). While the example code should work the same way either way, you’ll want to use the 1.0.2 code in any actual development you do over the CDN.

For those who haven’t heard of Dojo, it’s a fantastic JavaScript toolkit that you really don’t want to live without if you are a web developer in this day and age. In addition to providing facilities that comprise a JavaScript standard library, you also get a library of amazing out-of-the-box widgets and build tools. You can read a short ONLamp article about it here if you’re looking for a drive-by overview.

Also, stay turned for the upcoming book that’ll be available early next year.

PS - What more could someone possibly ask for on Christmas morning than a Short Cut on Dojo? It makes a great stocking stuffer :)

I was just commiserating with David Wheeler about a problem he had with mod_ssl on Mac OS X. The .dylib extension on shared libraries seems “arbitrary” in his words, and I’ve wrestled with it in cross-platform code a few times myself.

Then I realized where so many of my frustrations with Mac OS X came from as a developer:

Unix circa 1986 via NeXT is different from Unix circa 1998 or 2008 via Linux.

I’m happy to stick with POSIX when I want software to run somewhere outside of the nice cozy GNU/GCC/Linux/glibc universe, but those NS* functions just don’t quite feel right, you know?

Another article of the series “Yet Another Perl 6 Operator”

The syntax of an if-then-else expression in Perl 6 is composed by the conditional operator.

say "My answer is: ", $maybe ?? 'yes' !! 'no';

The expression above is equivalent to that, which uses the if-then-else statement within a do.

say "My answer is: ", do {
    if $maybe {
        'yes';
    }
    else {
        'no';
    }
};

The Peer to Patent project (which I’ve reported on before) just pointed me to a particularly broad patent that could encumber user interfaces on the web and desktops for years to come.

The idea in this patent, submitted by Yahoo!, is a clever little idea: if someone is starting to drag an icon or mail message somewhere, why not bring the mountain to Mohammed, so to speak? If she is dragging a photo, for instance, the browser or operating system can guess that she wants to open it with PhotoShop or the Gimp, and present that choice right next to the icon for the photo.

Another article of the series “Yet Another Perl 6 Operator”

The design of Perl 6 includes some unification mechanisms to bring some extra power to syntax and developers. One of such artefacts is the notion of meta-operators. With them, it is possible to construct augmented operators from existing ones.

The first of such meta-operators we’re going to look at is the quite simple '!' negation prefix.

As you may know already, Perl 5.10 came out today. Today is also Perl’s 20th anniversary (see also Perl Simplifies the Labyrinth that is Programming Language — and to be fair, Perl doesn’t include David Bowie in extra eye makeup and tight pants).

You might not know that the Parrot porters have released Parrot 0.5.1. It’s pure happy coincidence that the monthly release cycle of Parrot coincides with Perl’s 20th anniversary, but in honor of the occasion, there’s a nice puny easter egg in this release that might bring back a stab of nostalgia. It’s an interesting comparison of how far the language and platform have evolved in two decades. Oh yes, and the Parrot tarball also contains an implementation of Perl 6 which has made tremendous progress in the past month.

Perl 1 was released to the public by Larry Wall 20 years ago today. To celebrate, Perl5Porters have released Perl5.10, the latest stable version of Perl 5. Happy Birthday Perl!

Perl 5.10 isn’t just a bug fix version: it’s full of new features that I’m eager to use: named captures in regular expressions, state variables for subroutines, the defined-or operator, a switch statement (called given-when, though), a faster regex engine, and more. You can read more about the changes in perldelta.

The perl-5.10.0.tgz file is making its way to all of the CPAN mirrors, but if you can’t wait for that, you can its torrent file. Once it makes it to the CPAN mirrors, it will the new stable.tar.gz

This time around, Perl 5.10 installation will work the same on unix and Windows: Strawberry Perl is a Perl distribution for Windows that comes with a C-compiler and everything else you need to do it yourself. Give it a couple of days to catch up, though.

I setup trac 0.11dev last night on CentOS 5 to manage the review process for our book, using svn 1.4.5 and python 2.4.3. It was very nice! The new admin interface is great, and allows you to easily load plugins:

One gotcha, that I always forget is, if you setup tracd running behind an Apache rewrite you need to setup the trac.ini file to have both:

base_url = http://trac.example.com
use_base_url_for_redirect = True

And your rewrite stanza should look like this:

    ServerAdmin trac@example.com
    ServerName trac.example.com
    RewriteEngine on
    RewriteRule ^(.*) http://192.168.1.1:8000$1 [P]

Just a note, it is not 100% beta yet, but from the traffic on the list, it appears it should be beta any day now. Nice work Trac team…this rocks!

Another article of the series “Yet Another Perl 6 Operator”

In Perl 6, you may construct ranges with expressions like

$min  ..  $max
$min ^..  $max
$min  ..^ $max
$min ^..^ $max

and even

^$limit

If you read my feed, you probably saw that I changed the name of the project mentioned in SAP’s Composition on Grails. After I published that, my contacts at SAP said “You know, we had been meaning to change the name.”

If you’re interested in the project, you should definitely follow Will Gardella’s weblog, starting with Composition on Grails - New Version, New Name. He answers some of the most common questions he’s heard over the past few months about the project.

I realize that SAP doesn’t want to offer official support for languages other than Java and ABAP, but every demo I saw there with interaction between services over WS-* makes me think that the company could spend $50k in tech marketing very well by posting bounties with the appropriate F/OSS communities to beef up WSDL and SOAP support in languages such as Perl, Python, Ruby, and PHP such that they can be first-class consumers of SAP middleware services in the same way as Groovy now is.

Again, it doesn’t have to be official support, and it’s just the price of a full-page ad in a decently-read magazine, but it would be a shame not to take advantage of these rich ecosystems with an architecture that already supports such a thing….

Another article of the series “Yet Another Perl 6 Operator”

Among the new Perl 6 operators, there is the handy operator '//', known as defined-or or the default operator. This novelty was anticipated by the introduction of this syntactic bit in Perl 5 (see the upcoming 5.10 release) — so you won’t need to wait for Perl 6 to start using it.

# dor.pl
use 5.010;
print "arg: '", shift // "?", "'\n";

$ perl dor.pl one
arg: 'one'
$ perl dor.pl ""
arg: ''
$ perl dor.pl
arg: '?'

Most people forget that python is both procedural and Object Oriented. If you find yourself wanting to use a procedural, Bash style of programming with Python, take a look at the global statement.

It lets you do something like this, because without the :

In [34]: x = 1; y = 2

In [35]: def func():
   ....:     global x, y
   ....:     print "X = %s, Y = %s, in func" % (x,y)
   ....:
   ....:     

In [36]: func()
X = 1, Y = 2, in func

In [46]: x = 1; y = 2; z = "out here"

ln [47] def func():
   ....:         global x,y,z
   ....:         print "X = %s, Y = %s, in func.  Z says: %s " % (x,y,z)
   ....:         x = 1000;y=1000;z="in here"
   ....:         print "X = %s, Y = %s, in func.  Z says: %s " % (x,y,z)

In [48]: func()
X = 1, Y = 2, in func.  Z says: out here
X = 1000, Y = 1000, in func.  Z says: in here