Related link: http://use.perl.org/~luqui/journal/26943

A recent journal entry from fellow Perl 6 cabalist Luke Palmer expresses a problem I’ve had with reading CS papers: I wouldn’t put up with that inscrutability in code I had to review or maintain.

Why should people put up with that in research papers? Does jargon-heavy, mostly mathematical notation serve a useful purpose (even as simple as keeping down word lengths), is it the sign of laziness from people who don’t have to touch the code (or prose) after it hits publication, or is it a sinister sign of a technocratic kabbalah, trying to keep out the unworthy?

Maybe that’s overstating it, but I have a similar dissatisfaction with the Haskell programming language. Autrijus Tang, who like Luke is both scarily smart and younger than I am, swears by the language. I’ve dabbled in it, even written some production code in it (including parameter binding and object destruction in Pugs, the latter of which exposed a bug in GHC’s garbage collector) and the syntax and idioms of the example code I see still drives me crazy.

I’m not a mathematician; I write code to solve real problems, not to calculate the Fibonacci sequence lazily and infinitely or to create a typesafe, type-agnostic function that appends to or counts the elements of a list. Yawn!

I think I’m a pretty good programmer. I understand closures, iterators, generators, list comprehensions, currying, continuations, type inference, co- and contravariance, and even (to some extent) monads.

For those of us who just want to do something productive, not show how elegantly we can transform numerical integration problems into source code, is it too much to ask for meaningful identifiers and intelligible examples? Don’t even get me started on the typing error messages GHC spews. The difference between IO and IO () was, well, awfully subtle the first dozen times I saw it.

I appreciate academic research and I know that researchers are solving problems that I don’t even recognize yet — but the gap between theory and practice is, in part, the fault of their poor communication.

Yeah, yeah, I’m a Perl hacker complaining about syntax. What do you think about the density and inscrutability of computing papers though?

What I don’t know about sports could fill three or four stadiums and a half-dozen indoor arenas, and throw in a swimming pool and a Little League sandlot just for good measure. Nevertheless, one of my favorite occasional stops on the Web is a section of the ESPN site — the section hosted by Bill Simmons, “The Sports Guy.”

Now, I don’t read Simmons for his insights into athletes’ contract negotiations, or various pro sports’ drafts, or record holders, or any of the rest of it. What I read Simmons for is two-fold:

1. The sheer pleasure he takes in writing about his topic of choice; and
2. His fearless (and almost always entertaining) digressions into areas other than sports.

Among the latter is a new feature he’s instituted, called The Curious Guy, which he described as follows in his first foray into that area:

I e-mail questions to somebody who’s successful — whether it’s the GM of a baseball team, an author, a creator of a TV show, another columnist or whomever — and we just start trading e-mails for an entire day. As with many of the new features I start up, you may never see this one again, or you might keep seeing it. I don’t know. Let’s see how this one works out.

The very first Curious Guy piece, linked to above, was an exchange of email messages with the creator and executive producer of the TV show “The OC”; the second, several thousand words of back-and-forth with Chuck Klosterman (”the best-selling author and pop culture guru who released his third book, ‘Killing Yourself To Live’, earlier this summer. Chuck also writes monthly columns for Spin, Esquire and Honcho. All right, I made that last one up”).

Why I’m rambling on about Simmons here is that I imagine this weblog will turn out to be something like Simmons’ column, transplanted to the technology realm. While my principal focus will continue to be (as it was on XML.com) on XML and related technologies, I look forward to the chance to write about other technologies as well — and sometimes, who knows?, to fall off the wagon and write about something completely unrelated to the transmission of electrons through silicon, copper, and fiber optic cable, or to any of the jillion side- and after-effects of such transmission. I’m kind of a Curious Guy myself.

Anything in particular you’d like me to look into — or stick my neck out about? Feel free to comment on this or any other post; if publicly sticking out your own neck gives you what used to be called the vapours, feel free instead to contact me at johnesimpson-at-earthlink-dot-net.

Related link: http://www.sdexpo.com

I want to pick up essentially where I left off with my last article. Before I get into that, let me first share a little about the conference. Today, the conference officially ended. I’ve had such a broad spectrum of emotions and experiences, that it is difficult to characterize. A few of the scenes that linger in my memory have nothing to do with software development. Or do they?

Increasingly as I ponder this never ending debate about open source versus proprietary models, I’m believing more and more that our focus in the software industry is severely misguided. One of my take aways from this conference is that, very simply, that the development of a software solution is less about the technology and more about people. Before you flame this article, consider this perspective. Think for a moment about the following software development memes: user involvement in XP and agile development, user-centered development, and a new one for me from the conference, usage centered design. Consider also that one of the tracks at the conference was “People, Projects & Teams”, in addition also consider part of the description for that track: “The most important success factor for any software development team is the people that comprise it.” I think that human centered software development is applicable along many different levels. It is equally important as a key philosophy while working with developers or teams as it is working with our clients and end-users. Ultimately, the tools we use, whether they are Linux or Windows, Java or C#, or for that matter, PHP, Perl, or Python, in the end are merely tools.

I learned a new metaphor for software development at this conference. In the past I wondered whether software development was art, engineering, or manufacturing. I’ve seen this new analogy before with her Kim Polese and some her references to Doc Searls’ construction metaphor. However, this analogy became very clear from some of the keynotes and other discussions I had during the conference. So I ask the following…Do you think that carpenters get into heated discussions over what brand of hammer they use? Is one brand of lumber better than another? It might be, I have no idea. I’m a geek so how would I know?

One of the activities held during the Software Development Best Practices conference was the announcement of the Software Development Magazine 2005 Readers’ Choice Awards. So bringing up the subject of my previous article, it is extremely curious that even though I witnessed very little convergence between the Software Development Magazine conference and the Open Source community, the readers of magazine voted open source projects as recipients of approximately 25% of the awards. I’m excluding the category solely about open source, but clearly 25% of the sessions were not about open source software. So there is a little bit of a disconnect between the readers of the Software Development magazine and the organizers of the conference.

For me, my last article generated a lot of comments. I’d like to address some of those comments. From the speakers and attendees that I met, a large percentage do NOT make their living from software license fees. A considerable amount were consultants, authors, or work within an IT software development shop. If we take a moment and recall that most of the software our industry produces is never distributed nor sold with any type of license, whether proprietary or open source, then how can we claim that Open Source is jeopardizing someone’s ability to earn a living? If I happen to work in the insurance industry and I implement a web-based system using Linux and Apache, is my job at risk? Sure, maybe someone working for a producer of operating systems or web servers might be at risk, but isn’t that the basis of a market-based economy? I could just as easily choose any provider in that marketplace and it wouldn’t be seen as out of the normal.

To tie these two streams of thought together, I really believe that, whether open source or proprietary, we are all focused on creating software that is useful, used, and brings us the recognition and respect that we crave as human beings. Consistently, I think most everyone would agree that the obstacles that we confront involve our inability to understand what users want, translate their wants into a functional and usable system, and manage our teams to deliver what we believe needs to be built. I fail to see how licensing or technology plays a role in these dysfunctions. In the end, I would have to admit that my project and presentation were both centered around people, and the major problems I confronted were not technical or licensing problems, but people problems. Perhaps, just perhaps, the area most critical, and most in need of our complete attention, is not licensing or technology, but human relation engineering (or creation or constuction, choose your metaphor).

What is your metaphor for improving software development?

Related link: http://www.foxnews.com/story/0,2933,170724,00.html

ODF is an open standard, free for any organization — for-profit, non-profit, hobbyist — to implement. Presumably, any software product that implements the format is acceptable to use for working with the Massachusetts government. If the right price to feature ratio is $100 for an organization, so be it. If the right price to feature ratio is $0, there are products that do that too.

For example, AbiWord, a Free Software word processor developed by a handful of people, already supports ODF. The same goes for other free software and open source products.

Is there a technical reason preventing Microsoft, with tens of thousands of programmers and billions of dollars available, from adding ODF support to Microsoft Office? That’s a ridiculous question! Of course not.

Instead of supporting what a large customer values — open standards, free to anyone to implement, without fear of patent traps, royalties, and planned obsolescence — Microsoft has chosen not to compete on the merits of its software. (An alternate explanation is that Microsoft is completly incompetent and can’t duplicate or even merely license the work of a handful of open source programmers. I reject that idea; it appears more likely that Microsoft really wants to levy a tax on collaboration as an NGO.)

All that prevents Microsoft from supporting ODF is its will not to compete. Don’t misunderstand me — this is an act of will and a refusal to compete on anything but entrenched proprietary lock-in. In other words, the dominant provider of office software prefers to compete on its own incompatibilities than on the quality of its software. That’s… hardly new.

With tens of thousands of programmers and billions of dollars, the best Microsoft can do is complain? That’s not a company to which I’d tie my future. Good for Massachusetts.

What’s it worth (in money or in time) to take back your future from a greedy, abusive monopoly?

Related link: http://www.odeo.com/show/120299/view

Ruby has it all: all the press, methods on integers, a string method named “chomp”…and they have a song to boot. Why can’t Python have a song??? I want a Python tutorial song! I think I’ll go sulk now….

Who’s going to volunteer for a Python tutorial song?

Related link: http://cheeseshop.python.org/pypi/SQLObject/0.7rc1

I couldn’t find news of the release on the main page of SQLObject, nor could I find release notes at SQLObject.org (or in the release itself), but I did find news of 0.7.0 which mentions features and bug fixes. I didn’t find a date on this news entry, so I don’t know if this is for 0.7.0 in general, or if it is for a specific build of the 0.7 line. Hopefully, this will be pertinent to rc1.

I’ve used SQLObject now on a couple of small projects and have been pretty impressed with it. I’m not afraid of SQL at all, but it is nice to have a well-built, general-purpose data abstraction layer nicely packaged and ready to use. In the projects in which I’ve used SQLObject, I haven’t had to write a line of SQL - even to create the database schema.

As I mentioned in this blog entry, I really like SQLObject, but I feel like I need to re-re-read the docs. I’m having a small issue creating (what I feel are) the “right” relationships between one object type and two other object types which the first type links to. I’m certain I’m just not doing something right. This experience doesn’t leave me with the feeling that SQLObject is an inferior library. It leaves me with the feeling that I need to spend a little more time with it and make it do what I want it to.

Great library. Good work, Ian.

Related link: http://pugscode.org/

When I first learned that several members of the core Perl 6 design team
(affectionately known as @Larry) had learned Ruby and/or Python, I wasn’t
terribly surprised. It pays to understand one’s competition after all,
and there are some nice features in each to assimilate. Perl, Ruby, and
Python all share some deep common design attributes, so they’re not
exactly new ground, however. They could be considered part of the same
language family, as for example French and Italian are.

What has surprised me of late is the number of Perl 6 designers and
developers that are learning languages from completely different language
families: Dylan, Forth, Haskell, Joy, Lisp, Prolog, Scala, Scheme,
Smalltalk, and who knows how many others. This is a very good thing.

Each of these languages offers a unique perspective on programming. Each
has its own philosophies, world views, conventions, and so on. Most of
them try to be extremely good at at least one thing, and can therefore
warp their entire syntax and semantics around that task. This gives the
opportunity to find features in their “natural environment” that Perl 6
can then assimilate.

And assimilate it does. Just as Perl 5 borrowed heavily from the languages
that came before it, so Perl 6 is borrowing heavily from today’s languages.
It’s also taking lessons learned from long experience with older languages
from which Perl 5 didn’t borrow enough. Here are a few examples:

C

• Having easy, efficient access to C libraries is important.
  Perl 5 of course has had access to C libraries for a long
  time, but the interface was clunky and slow. The Inline
  group of modules have helped, but it’s still nowhere near
  ideal. Perl 6 (and Parrot) have long had a goal of clean
  library call interfaces.
• Better handling of native data arrays and packed data
  structures is important, too. Again, Perl 6 has long
  had native (AKA unboxed) data types in the design.

Dylan

• Method resolution in the face of complex inheritance paths
  is a pain. Perl 6’s default MRO (Method Resolution Order)
  is based on Dylan’s C3 algorithm. In fact, the Perl 6
  OO MetaModel even uses some tests cribbed from examples in
  the original Dylan C3 paper.

Forth

• The best language for any situation is one designed
  specifically for the task at hand. Perl 6’s grammar system
  makes it much easier to do this in Perl. Like Forth and
  Common Lisp in their own ways, Perl 6 allows deep
  control over parsing and macro use. In fact, Perl 6’s
  grammar is exposed to the programmer and can be directly
  altered — or new grammars can be created, and swapped in
  as desired.
• That said, the base language should have a rich
  non-orthogonal vocabulary, allowing many common tasks to
  be coded clearly, instead of being a mess of contortions
  to get around false simplicity in the language design.
  Perl 6 has this kind of rich vocabulary (in spades).
• Refactoring is at its best when you can count on the
  compiler to keep everything running fast. Several
  implementors are working on various ways to keep Perl 6
  code running as fast as possible, without forcing the
  programmer to forgo clean, well-factored designs to do
  so.

Haskell (et al.)

• There’s more to Functional Programming than just having
  support for closures. Perl 6 is borrowing heavily from
  the powerful toolsets used every day in FP languages.
  From metaoperators to function signature pattern matching,
  Perl 6 is getting a big dose of higher-order concepts in
  the core.

Java

• Java teaches mostly by counterexample. Yes, it’s true,
  good garbage collection is important, and Perl 5’s
  reference counting just didn’t cut it. But the biggest
  lesson to be learned from Java is that all these nice new
  features shouldn’t result in a horrendous programmer tax.
  Anything the programmer doesn’t want to be faced with
  right now should just get out of the way. That’s been a
  Perl tenet for some time, but ugly things like parameter
  unpacking (especially hundreds of copies of
  my $self = shift;) have gotten in the way. Many
  of those things are now fixed.

Perl 5

• CPAN is all-important. In fact, it is probably the one
  thing other language designers are most jealous of from
  Perl 5. Perl 6 can use Perl 5 modules from CPAN directly
  (and Pugs has been able to do this for months, albeit with
  a few gotchas that Ponie should help with) while the
  whole mass is slowly migrated to Perl 6.
• Perl 5 did a lot of things pretty well. Perl 6 doesn’t
  throw them away, but rather tweaks them to be just a
  little bit better.

It’s important that it’s not just the core language designers that are
learning and borrowing from these disparate languages; the implementers
are as well. Both groups are getting the benefit of the broadened
horizons these languages afford. Sometimes in #perl6
(irc.freenode.net) links to language homepages and papers on language
design and implementation are flying so fast it’s hard to just keep up
with the general gist of things.

And it’s not just reading. Perl 6 people are going to conferences for
other languages (such as ICFP),
presenting Pugs and Perl 6, sharing knowledge with other language
designers, and soaking up vast piles of information. It’s a beautiful
thing.

People have complained about how long Perl 6 is taking to arrive; I used
to grumble about that myself. Now that I watch things growing and changing
on a daily basis, I’m glad everyone is taking their time. There is so
much good stuff to assimilate, care must be taken to not end up with a
huge ugly mess. As when cooking with many ingredients, sometimes the
best thing is to let the pot simmer for a while, to let the ingredients
get to know each other, to mellow out and produce a better whole.

Of course, the benefits of learning completely different computer
languages go beyond just language design. Having new viewpoints on the
problems you face every day can be a great help, even if you don’t plan
to change the language you code in. Which is why I’m joining the crowd
and learning Forth, even though I expect Perl will be my day-to-day
language for years to come.

What’s your favorite “very different” language? What could Perl 6 learn
from it?

Related link: http://www.oreillynet.com/pub/a/sysadmin/2005/09/15/qpsmtpd.html

In a recent use Perl; discussion about Matt Sergeant’s recent ORN/Perl.com article about qpsmtpd, the pure-Perl SMTP server, Bart Lateur pointed out that one buried sentence serves as a great Perl success story:

As an example of the scalability of qpsmtpd, the apache.org mail servers currently process more than 2 million emails a day on their primary MX, rejecting more than 80 percent of it as junk — all thanks to the power and flexibility of qpsmtpd.

That’s, on average, 23 messages per second. Go qpsmtpd. Go Perl.

Do you /really/ need to write your next network daemon in C?

Related link: http://www.python.org/2.4.2/

Here are the release notes for 2.4.2 final from the release notes page:

What’s New in Python 2.4.2 final?
Release date: 28-SEP-2005

Extension Modules

Patches #1298449 and #1298499: Add some missing checks for error returns in cStringIO.c.

Patch #1297028: fix segfault if call type on MultibyteCodec, MultibyteStreamReader, or
MultibyteStreamWriter.

Tests

Fixed failure in test_macfs on Mac OS X 10.4 (Tiger), which apparently doesn’t allow the creation time to be set later than the modification time. Fixed by changing the test data.

Build

The Windows .msi files are now compressed using lzx:21. This produces a significantly smaller installer.

Downloads for 2.4.2 are here and details of 2.4 in general are here.

Related link: http://www.turbogears.org/

About a week ago, I said a few preliminary, positive words about TurboGears before I had a chance to actually do anything with it. Well, I’ve written some code now and I’m ready to report on all the ugly details.

I recently bought my wife a Canon EOS Digital Rebel. For various and irrelevant reasons, I created a simple little digital photo managment program for it with CherryPy and SQLObject. My little app seemed to work OK. Just like anything, it has some rough edges and some room for improvement. When Kevin Dangoor recently announced TurboGears, I decided to rebuild my little photo app using TurboGears.

My nonGears version is just under 300 lines of pure Python code. No templates. And I must say, it’s not the prettiest coding I’ve done. No templates means that I’ve embedded HTML in my Python code. I know. That’s nasty. But I wrote it as a quick and dirty job. The Gears version turned out to be about 300 lines of Python and 225 lines of templates. This really is not a fair size comparison because I added a few niceties in the TG version that weren’t in the VCP (vanilla CherryPy) version.

I started the project with the quickstart option to the turbogears-admin.py script. This did a nice job of creating the directories and files I would need. It separated out all the parts in a nice MVC fashion. The database model went into a model.py module. The controller went into a controllers.py module. And the view went into a templates folder. This structure answers a whole class of questions for a developer about where to put which files and what to name them. Structure is good.

One thing that I had to get used to with TurboGears was writing code in the controller and then having to specify which template to “send” it to, then fleshing out the template. This is just an adjustment to an MVC structure. My original code didn’t have any of that; the HTML was inline. That made it easier to write initially, but a maintenance nightmare afterward. This logical separation made it really easy to re-use code (yes, I did re-use some code; I’m not talking abstract theory, here) and fix bugs while I was coding. Separation is good.

Probably my biggest hurdle was SQLObject. I need to thoroughly read the documentation. I created relationships between some of the tables which aren’t working as I would expect. I’m sure it’s my problem, but I just didn’t invest the time in working through it until it worked.

One of the really cool features of TurboGears is the turbogears-admin.py shell commmand. It opens a Python interpreter with the PYTHONPATH set exactly like it is for the running CherryPy process. This goes a long way in helping test code.

While I was not able to pound out my photo management app in 20 minutes, like Kevin Dangoor in the demo video, I felt very productive. Nothing was laid out in a non-obvious, surprising way. And - get this - I was having fun writing my little app. Fun!

I think TurboGears is an assembly of the right components. It answers a lot of the common questions developers will have in logical, well-thought-out ways. It is extremely easy to use. It has a growing community using and supporting it. I like what I see from technical, psychological, and sociological perspectives.

OK, now for the ugly details. My UI design is horrendous. I think it’ll take more than TurboGears has to offer to help me make a decent looking website. I think it’ll take something closer to a miracle.

Have you tried TurboGears? What are your thoughts?

I can’t scientifically say what makes code “good”, but my gut has a few things to say about it:

1. good code reflects the problem it solves
2. good code can be easily understood by programmers without experience in your language
3. good code is generally short
4. good code doesn’t require much thinking to understand
5. good code has very little coupling
6. good code is cohesive - definitions of functions and types required are not scattered throughout the software
7. good code documents any requirements for proper usage
8. good code makes just as much sense to you one year from now, as it does now

All of these properties I feel could be summed up simply as: “good code is reusable”, and taken a step further: “really good code is so simple it can be reused by a trained seal.”.

What do you feel are other properties of good code?

Today I’m witnessing the collision of my former self and who I have become. Today I’m at the Software Development Best Practices Conference and Exposition. When I was a software development manager for many years this was the one conference I was never able to attend (yes, I am bitter but that’s a whole other story). So I’m very excited and happy that I’m finally attending, even better I also get to share the SNAP Development Center story.

Leading up to the conference, I started to perceive that the Software Development community (as defined by this conference)and the Open Source community were separate. I observed that there didn’t seem to be many open source related sessions nor open source related exhibitors. Now that I’ve been to a few sessions, I have confirmed that these communities are fairly apart from each other. This really surprises me. On the one hand, as I led an open source project for nearly the last two years, it is easy to believe that everyone sees the benefits and value of open source software. On the other hand, most of the sponsors and many of their participants here earn their living from proprietary software. So it should be no surprise why these communities are not converging. There is some overlap between them, but I just some how expected more of it.

Two anecdotes from this morning illustrate how far apart these communities seem to be. First, one of the presenters was speaking on user interface design issues and habits when he excitedly shared that our browser was going to be coming with a tabbed interface. I was like, but, but, but, I’ve been using tabbed browsing for more than two years now. It then hit me (it took me a moment to switch my community viewpoint). He was speaking about Internet Explorer. I was completely shocked. Everyone I know uses Mozilla Firefox, so for me I implicitly thought that everyone did. It’s amazing how completely I have switched communities, and how that switch has changed my perception of what is normal.

This has made me think something I never contemplated before. I wonder how many Internet users have never heard of Firefox? If they have heard of it, I wonder if they have tried to download and install it? Finally, if they did try Firefox, do they still continue to use Internet Explorer? These questions are really tough to even conceive of, if one is locked into the open source world view. As a software developer and Internet user, it is almost trivial to find, install and switch to Firefox. So why aren’t more people using Firefox?

The second anecdote involves a discussion I had with a developer of a niche scientific software solutions. We were talking about C#, and he said he loved the language. However, he said that he would love it even more, but that he worried about people decompiling the intermediate language and gaining access to his source logic. I didn’t know what to say about that, except to mentally note that this was a great example of how far these two communities are apart from each other. Somehow, I believe that his perspective is now so foreign to me that I can no longer find a way to reconcile it with my new point of view. I also feel that perspective lock-in (from either community) is a risky situation, and that we should work together to minimize the adverse affects of the decisions we might make when stuck in either point of view.

Can these two world views live in peace?

The terminology “Software As A Service” (SaaS) is gaining momentum and is becoming widely understood.

But, SaaS is used interchangeably and sometimes used to replace the definition of “Application Services Providers” (ASP). Many of you may know about ASPs and ISPs (Internet Service Providers). SalesForce is a good example of ASP. ASPs deliver application functionality (rather than software) as an outsourced service and charge on usage basis. ISPs provide internet service as an outsourced service. AOL is a good example of ISPs. For lack of better term we can call service providers who delivers software as a service as “Software Service Providers” (SSP).

Unlike the existing definitions and differentiations between SaaS and ASPs, I think there is an important distinction between ASP and SaaS. In case of ASPs, the software is not shipped to the customers; the customers access the “required functionality” and use the functionality via a remote client i.e. the software execution happens in the ASPs’ end while the customers access the application. In case of SSPs, the software is shipped to (or downloaded by) the customers, deployed within customers’ premises (or their co-location) and gets executed within the customers’ environment. Anti Virus Software is a good example for SSPs, where subscriptions are charged for virus definition updates and not for the anti virus software itself. Moreover in ASP model when a service is stopped, the customer has to migrate their data to a different environment, where as in SSP model the customer can still run the software (along with their data) without any associated value-added services (like future updates).

Here are 5 things you want to adopt if you want to be a SaaS provider:

1. Initial activation cost as low as possible (better if it is free) to gain adoption
2. The customer must be able to make use of the functionality (out of the box) right from the get go
3. There must be incremental value-add (updates, patches, validated integrations) on an ongoing basis that are delivered seamlessly to customers and can be charged (like in free printers vs. paid cartridges)
4. The incremental value-add must be thoroughly tested and should not break backward compatibility. Think of whether you will get antivirus updates if it breaks your existing applications
5. The customer must be able to stop the subscription and still be operational (no vendor lock-in)

SaaS blends best of both traditional software model (customer owns the software) as well as ASPs (rent or lease the service). Hence in my opinion, SaaS business models will be widely adopted and preferred by customers in future.

If you are a customer, what will be your preference: ASP, SSP or Traditional Software Delivery Model? Please share your thoughts.

Related link: http://blogs.forrester.com/charleneli/2005/09/yahoo_launches_.html

I am fascinated with the divergence between the Yahoo mail beta and the (admittedly older) Google mail beta. After only one week of Yahoo’s mail beta, it’s getting high praise. It sounds like Outlook-meets-AJAX, and people like it a lot. If you want to try Yahoo mail beta, here’s a short application.

At first I thought: of course Google will fall in line. I am now fascinated with the idea that they won’t. Could you imagine Google copying Yahoo, who is copying Microsoft Outlook? Perhaps Google could roll out new Gmail skins, with one skin tailored after Outlook. That way they could spin their: “me too!” as a: “me first!” Anyhow, it’s a good idea.

Email isn’t as cool as search, but here are some numbers that suggest its importance to the leading web brands (emphasis mine): MarketingSherpa.com says: “Only 84% of the US Internet-using population uses search engines on a regular basis. However, 92.5% use email.” and: “Google’s Gmail has only captured a tiny 2.5% slice of the personal email usership pie” And Charlene Li says: “31% of North American online households use Web-based email at least once a week, up from 27% a year ago.”

As much as I love Google, I am also keenly sensitive to the bottom line: “what works?” Indeed, isn’t that why we all fell in love with Google to begin with? Its search simply works. Maybe they think we fell in love with the innovation? That would be so Web 1.0.

* update *

Here is a detailed review of Microsoft’s new AJAX-enabled Hotmail beta (code-named Kahuna).

Who do you love? Why?

You know, its fascinating watching people approach Open Source in so many different directions. Some people are engulfed by the ethical advantages, some by the technical prowess, and some because it is just interesting and strange. Irrespective of the basis for the attraction, it can also be fun to observe those who think they understand Open Source.

There are very few people in the world who fully understand Open Source. Sure, you can understand it from a technical perspective, but the social threads that cascade through the thousands of sub-communities are intensely difficult to truly understand. Unravelling this social fabric requires a keen interest in people-watching; an art form in itself. The only way you can really understand the community is to be an active member of it. You need to hang out on IRC, subscribe to mailing lists, go to LUG meetings, run websites, hang around shops giving out Open Source CDs and generally badger people about why free software is a good thing. You don’t have to be a software developer to do this, but you do need to be a part of the community.

Unfortunately, there are a lot of people out there who don’t really get it. These hapless individuals will typically guffaw their way into contributing, with every good intention, but their fundamental misunderstanding about the community will be their failing. They will expect that the Open Source way of doing things can mimic the commercial development culture, they will think that an Open Source project will attract a flock of developers at the drop of a hat, and they will also expect all good discussions to result in real working software as opposed to a This project has made no releases conclusion.

Total idiot

To illustrate such misunderstandings, I have a case study. This is a prime example of somebody who had great intentions, but just didn’t understand the community at the time. Unfortunately, this hapless fool was me.

Back in 1999 I was 19 and about to go off to University. When I was 18, my brother got me into Linux and I was instantly wooed by it. Fascinated by the concept of distributed, collaborative community, I set up Linux UK; one of the first UK websites dedicated to Linux. I worked hard setting it up and promoting it, dedicating hours to it every night, and it was doing rather well. As time went on, I was looking for the next challenge and back then, KDE was rocking my world - I wanted to contribute. As a 19 year old gangly youth who had a reasonable reading knowledge of software development but not much of a practical-in-the-trenches knowledge, I figured I could just set up a team and they would come. Hey, Matthias Ettrich set up KDE and they came, Miguel de Icaza set up GNOME and they came, Linus Torvalds set up Linux and they came. I was certainly not expecting projects the size and esteem as those, but I figured it would be cool to run a software development team creating KDE software. What a clueless muppet I was…

So, I loaded up my trusty Usenet client and fired off this rather embarrassing collection of witterings to comp.windows.x.kde. This post demonstrates a clear misunderstanding of Open Source software development, from somebody who was genuinely convinced he knew it and could contribute in a pragmatic way. The intentions were innocent and certainly not driven by ego, but the expectation that I could just form a development team and tell them what to do was ridiculous.

One problem with my post back then is that I was laying out a list of requirements and expecting adherence from people who threw together code in their spare time for fun. Paragraphs such as “Programmers will need to have experience coding in C++ and preferably CORBA, and be able to code the Qt widget set. Experience at coding KDE applications is neccessery” are crazy. Aside from maybe spell-checking before I posted out such an important, ground shaking, everybody-must-see post that would attract an army of ‘yes pick me!’ contributors, it is is crazy to list job-like requirements when trying to attract people to a project.

The major problem with the post was that I was somehow expecting that people could be driven by a project management approach to Open Source software development. I remember thinking back then that it seemed crazy that the only way you can develop software is by coding a chunk of code and then encouraging others to join in. Surely this will only result in overly-complex unusable software? To be fair, this concern had reasonable merit - some of the software back then was awful when it came to usability, and I think my rather pathetic efforts to cobble together a team were a subconscious effort to contribute to the usability of software. Back then I remember looking at some of the GUI desktop applications and identifying what seemed like clear usability flaws. Although the intention was reasonable, my solution and subsequent cringe-worthy post demonstrated a mis-understanding of Open Source development.

The major lesson to learn from this post is the separation between what is theoretically possible and what is actually possible. Over the years I have seen so many people approach Open Source in a similar way - they think of something that could theoretically work, but in reality, the culture and climate of the community means that it just won’t happen. When someone typically identifies ‘theoretical possibility’, this usually refers to something that is possible without any scientific, legal or financial restrictions. Running an Open Source development team is certainly not subject to these constraints, but it is subject to the hidden unwritten constraints of the community.

Hindsight is always 20/20

Since those days, I have made Open Source my ambition and my career and I have spent the last seven years exhaustively working to contribute to Open Source in different ways. I have been fortunate enough to be around great people who have contributed to my own development and understanding of the community and I am now confident enough to say that I believe I know the community and its constraints fairly well. Although it makes me cringe when I read that awful post, that post has itself been a useful learning curve in understanding the nuances of this social fabric that brings people together in the Open Source community.

I certainly don’t profess to know everything about Open Source and its community, and I never will, but I am utterly convinced that the only way to understand the community is to become a part of it and build up a body of knowledge to compare which projects and ambitions work and which don’t. When you can identify the hits from the misses, you can identify patterns in structure, and which factors made which effort a success. Aside from anything, it is always important to step back and identify where you missed the point, and I hope this rather embarrassing little tale will help to clarify some aspects of the community for others.

What do you think? Would you have joined my super elite team of KDE developers? Do you have any similar experiences/thoughts? Go on…share them with us…

Related link: http://www.news4neighbors.net/article.pl?sid=05/09/23/1538200&mode=nested&tid=1

Portland-area hacker Eric Wilhelm recently received a generous $10,000 grant from UK-based Xara to continue to develop a converter between multiple vector graphics formats. Eric’s passionate about this and has worked on it anyway, but major credits to Xara for recognizing and promoting the idea that connectivity and interoperability drives competition and invention. News4Neighbors has more information.

Related link: http://www.gotdotnet.com/workspaces/news/newsitem.aspx?id=ad7acff7-ab1e-4bcb-99c…

From the release page:

We have just released IronPython 0.9.2. In addition to focusing on the CPython regression test suite
and fixing bugs, we also spent time prepairing IronPython for PDC 2005 conference where Jim spent the
whole of the last week. There were many changes in IronPython codebase, mostly inspired by getting
the regression tests running and adding CPython 2.4 features.

Given their focus and the bugs they’ve been fixing, it looks like they are getting ever closer to an environment which is (mostly) compatible with CPython.

It also looks like running IronPython under mono on a non-MS OS is becoming less painful. I was actually able to run the IronPythonConsole (which is just an IronPython interpreter shell) under mono 1.1.9.1 on Linux (Ubuntu Hoary specifically). This isn’t a huge deal, but I was further able to compile a simple C# module with mono 1.1.9.1, import it using IronPython 0.9.2, instantiate an object from it, and then call a method on the object. I have had some issues with only certain versions of IronPython being able to do that with certain versions of mono. I’m hopeful that what we’re seeing is progress.

If/when IronPython becomes 100% CPython compatible and 100% mono runnable, would you use it? Why or why not?

Related link: http://www.bbc.co.uk/imp/

As anticipated by BBC and Slashdot, and covered in the 9/21 WJS (page A15), the BBC will begin a three month trial of their Internet television system this month. They hope to then roll it out next year in the UK. Unlike their current Online Radio Service, their online television service will be available only in the UK.

Its DRM is provided by Microsoft, its geolocation is provided by Quova, and its P2P software was developed by Kontiki. The client software, iMP (integrated Media Player), appears to be a Windows-only program.

The BBC hopes its iMP software will become the iTunes of online television. However, the service currently promises to deactivate downloaded programs after seven days, making it more restrictive than a TiVo. Still, it’s a start.

Pioneering the field, the BBC could end up licensing their internet television technology overseas.

Part of the iMP trial? How’s it going?

Related link: http://www.prnewswire.com/cgi-bin/stories.pl?ACCT=104&STORY=/www/story/09-21-200…

Citizens Against Government Waste released a press release today criticizing the Massachusetts plan for migrating to open document formats. There are two main arguments. One is that the migration will incur conversion and training costs. That part is likely true — no migration is ever completely free.

I don’t understand the second argument. CAGW president Tom Schatz said, “It is bad procurement policy for any state to unilaterally lock itself into one set of technologies.”

That was my argument to the Oregon Legislature two years ago supporting a move to open standards! Any vendor, proprietary or free, has the opportunity to support open formats in its products — even Microsoft.

It’s possible that Schatz means that migrating unilaterally to open document formats may be a bad short-term decision if the software that supports those formats does not perform the job as well as the alternatives. However, I don’t like that kind of short-term thinking, especially in the government. It’s more important to me to provide open and unfettered access to documents in the long term than to stick with (perhaps better) proprietary software in the short term, especially because proprietary document formats make it more difficult to migrate to open formats in the future.

In my mind, that’s the real waste.

I also wonder about the rest of this press release that criticizes the licensing model of software instead of the real issue — openness of document formats. Hmm.

Is there a nuance here that I’m not seeing, or is this just a silly criticism from CAGW?

Related link: http://www.turbogears.org/

First of all, let me say that I’ve gotten past the cool demo video and have downloaded TurboGears and am beginning to tinker with it. I have a really small project I’m converting from vanilla CherryPy and SQLObject to TurboGears. I’ll report later on how that goes. Right now, I’m digging into the Kid documentation.

I had a couple of people point out projects similar to TurboGears in a recent blog entry. The two projects they pointed out were Subway and Django. I’ve heard some buzz about these and have even gone to their websites, but haven’t been compelled enough to download them and give them a spin. But I was compelled enough to give TurboGears a spin. Why? I watched the TurboGears video. Man, that sounds so shallow, but it’s the truth. I watched the TurboGears video and was thoroughly impressed.

With Django and Subway, I glanced through the documentation, what they said they could do, thought, “OK - that’s nice”, and just kept right on surfing. I’m not saying that Django or Subway are in any way inferior to TurboGears. They may be vastly superior. But there wasn’t enough in the documentation to capture me and make me interested enough in the projects to download and give them a try. That definitely says more about me than Django or Subway.

I think what TurboGears has done with its marketing is going to get people excited about it, gain it market share, and will help drive its credibility. Credibility? Yes, credibility. I’m not saying that it makes it any better, but when people see with their own eyes how someone can create a wiki in 20 minutes, it will make them believe that it is a capable web framework. It will make them believe that it’s easy to use. It will make them believe that they can make a website with it with less work than an alternative.

I know, all this is mostly psychological and has nothing to do with the goodness of the project. That’s why I feel a little dirty even saying anything about it. But good marketing can be the difference between a project being widely used and a project dwindling into obscurity.

What place should marketing have in open source projects? Can good marketing really make or break a project?

Related link: http://codinginparadise.org/weblog/2005/08/ajax-creating-huge-bookmarklets.html

From Brad Neuberg:

“I’ve put together a way to have huge, arbitrarily sized bookmarklets, where most of the code resides outside of the bookmarklet link. I’ll explain how this works in this mini-tutorial.

” … The essential idea is that we dynamically insert a new script element into the DOM through our bookmarklet. …”

“This code has been tested in IE 6+ and Firefox.”

Cool.

Related link: http://www.jamesshore.com/Blog/Five-Design-Skills.html

I’ve often wished that there were some field of higher education between the theoretical computer science and the practical IT/IS degree programs for people interested in careers in software development. Both perspectives are valuable, but I see the purpose of undergraduate education to teach you how to learn and to do research and to organize and express your thoughts clearly. If it’s just vocational training, you might as well join a vocational training program at the cost of far less time and money.

I do regret not studying more mathematics, but everything else I know about software I learned outside of the classroom. When I see that my naive data structure isn’t working out, I can always look up an algorithm for a heap or a binary tree (and I always have to look up algorithms for dealing with graphs), but when my problem is more philosophy than fact I have to deal with my experience.

Imagine how far ahead we would be toward writing high-quality, effective, and maintainable software if we could count on new developers already knowing how to read, analyze, understand, and improve code. As Jim says, they may be a little slow and will probably need a little help, but even just knowing that these skills exist and are valuable and necessary could be a huge improvement.

Perhaps free software would be more effective if our projects actively encouraged this kind of review and maintenance — not just with words, but with actions as well. Of course, software development in general has to grow up to acknowledge maintenance as not only the largest part of the software lifecycle but the most important and, by far, the most valuable.

What do you wish they had taught you in school?

Related link: http://www.cherrypy.org/

I’ve tinkered with CherryPy a bit in the past, from 0.7 on up. With the recent announcement of TurboGears, I’ve had reason to give CP some fresh attention. This is an easy to use web framework which, even by itself (meaning without TurboGears), allows you to put together a web site in little time. This project has done well standing on its own and has gained momentum and appears to be gaining market share in the Python web frameworks arena.

New features for CherryPy are listed here. One of the new features is a “session authenticate filter”. Somewhere between 0.7 and 2.(something), the builtin support for a single session authentication mechanism was stripped out (although there were various recipes for authentication on the CP wiki). I really hope that has found its way back in for good.

How do you think CherryPy compares to other Python Web Frameworks?

Related link: http://www.google.com/search?&q=failure

If you don’t know the term, here’s a definition of google bomb:

Google bomb: to hyperlink a term to a website in order to raise the linked site’s ranking in a search return result set on Google.

I’ve intentionally written an opaque (but accurate) definition of Google bombing. It’s easy to understand Google bombing in practice.

The most notorious current example of Google bombing - in fact, of Google bombs at war - relates to a search for the term failure. President George W. Bush’s official White House biography comes up first, although film maker Michael Moore’s official site is a close second. These sites have been Google bombed and connected to the word failure in web pages.

The Google PageRank algorithm encourages this kind of “voting” by webmasters - both a strength and weakness of the methodology. (Click here for more of my take on the PageRank algorithm.)

I’ve already voted twice in this piece for George W. Bush as the failure of the two, and now let me stack the votes some more: failure, failure, failure!

Any other Google bomb wars we should know about?

Related link: http://www.datadirect.com/products/xquery/index.ssp

Jonathan Robie, the XQuery Language Program Manager at DataDirect, Jonathan Robie has some important news over on his blog.

I love collgiate sports, especially Division 1 football and hockey. One aspect I follow in college sports is recruiting. Now if I had to go out and “sell” Computer Science to the many college students undecided about their major, especially those who are scientifically and mathematically inclined, I have five recruiting pitches:

• Computer Science is not just programming - The popular belief is that Computer Science is all about programming, and it is so not true. There are many problems that make up the field. How can you make a computer think (e.g. recognize speech)? How do you design an interface for a product that suits the need of a happy medium? How do you ensure users of their privacy and security when performing banking and commercial transactions online? How do you make realistic and low-cost terrains for computer games? Programming is an important component, but documentation and design are equally critical.

Which leads to my next pitch…

• You WILL learn how to think - Do you know why Computer Science and Mathematics are so intertwined? Both fields deal with abstractions, the high-level information, which you can then apply to build concrete objects. A professor once told me: “The true point of college is to teach you to learn how to think, and not to be an apprentice of one subject.” If you can think, then you can solve and dig out of a lot of problems in life. You can also delve into other areas of specialty. Especially in Computer Science, you will learn to solve many problems, and perhaps walk out a well-oiled machine.

Which leads to my next pitch…

• You will have tremendous opportunities to delve into other fields of study - Computing is critical in all areas of studies, from the humanities to the sciences. Projects range from constructing digital libraries, analyzing complex economical datasets, developing a system to manage industrial processes (probably involving more than one field of study), and building robots. Nowadays, it is very common, if not required, for computer scientists to delve into other fields including Physics, Economics, Psychology, Biology, and Classics.

• The jobs, and the money, are still there - Of course, one of the biggest reasons for the sharp decline in Computer Science majors in the last few years is the job outlook for CS graduates. And news of IT jobs being outsourced to India sure doesn’t help. Sure the Internet bubble burst, but that doesn’t mean it is the end of Computer Science. Hey, the media say a lot of thing. Again, there are tremendous CS opportunities in fields such as the pharmaceuticals, education, security (yes, computer security is still abysmal), healthcare, and the list goes on. The fact to the matter is, there is always room for good developers and good IT folks. In fact, the number of good developers and IT folks is shrinking in this country, with a growing number of problems to solve –old and new.

Which leads to my last point…

• If you are in the United States, this country needs you - The sharp decline in Computer Science majors, especially among Americans, it is no joke. Not just in Computer Science, but also in Mathematics, Chemistry, and Engineering. Students from countries such as China, Japan, India, and Russia have embraced the fields. Add it all up, and what do we get –this country is starting to fall way behind the rest of the world in math and science. That is no way to compete with the rest of the world in the short and long-run, and as Coach Lee Corso says, that deserves a major-league YO!

There are still many problems to solve, from even the most basic computational tasks, to grand-challenge problems. Computer Science is required in all of the problems. A degree in the field will provide you not only a rigorous background in math and science, but also the problem solving abilities to get though anything in life.

P.S. Major kudos to IBM for encouraging its employees to help fill in the math and science teaching voids around the country.

Any other recruiting pitches?