Last week I was in Copenhagen for YAPC::Europe. One of the announcements at the conference was the location of next year’s conference which will be in Lisbon. The theme of next year’s conference will be “Corporate Perl”. And that (along with a couple of conversations last night) got me thinking about a talk that I’ll submit to next year’s conference which might well be entitled “Why Corporates Hate Perl”.

It’s not true, of course. There are a still large number of large companies who love Perl. I could probably work through to my retirement enhancing and extending systems that are written in Perl at many of the big banks in the City of London. There are, however, also many companies who are moving away from Perl for a number of reasons. Here’s one of the reasons that will be included in my talk.

I was talking to people from one such company last night. The Powers That Be at this company have announced that Perl is no longer their language of choice for web systems and that over time (probably a lot of time) systems will be rewritten in a combination of Java and PHP. Management have started to refer to Perl-based systems as “legacy” and to generally disparage it. This attitude has seeped through to non-technical business users who have started to worry if developers mention a system that is written in Perl. Business users, of course, don’t want nasty old, broken Perl code. They want the shiny new technologies.

And so, in a matter of months, the technical managers at this company have created a business environment where Perl is seen as the cause of most of the problems with the current systems. It’s an impressive piece of social engineering.

It’s also, of course, completely unfair. I don’t deny at all that this company (like many others) has a large amount of badly written and hard to maintain Perl code. But I maintain that this isn’t directly due to the code being written in Perl. It’s because the Perl code has developed piecemeal over the last ten or so years in an environment where there was no design authority which encouraged developers to think beyond getting their immediate task done. Many of these systems date back to this company’s first steps onto the internet and were made by separate departments who had no interaction with each other. It’s not really a surprise that the systems don’t interact well and a lot of the code is hard to maintain.

There are, on the other hand, a number of newer systems which are also written in Perl which follow current best practices in Perl development and are far easier to to maintain and enhance - as easy, I would contend, as anything written in the new approved languages.

It’s certainly true that this company has a large number of systems that need to be rewritten over the next few years. But throwing away all of the company’s accumulated Perl expertise and moving to new languages seems to be a step too far. Management are blaming Perl for the problems when really they should be blaming the management and design procedures that were in place (or, more likely, weren’t in place) when the code was originally written.

Many organisations are in the same situation, with large amounts of unwieldy Perl code. Ten or twelve years ago everyone was writing web systems in Perl and we were all making mistakes. We all have to deal with those mistakes but we’ve, hopefully, learned from them and can rewrite our systems to take account of everything that we’ve learned in the last ten years.

It’s too late for the company I’ve been talking about in this article. The anti-Perl social engineering has probably insinuated itself too deeply into the culture. It’s unlikely that Perl’s reputation can be rescued.

But if you have similar problems in your own company, then please try to ensure that blame is apportioned correctly and that you don’t use Perl as a scapegoat.

A while ago, I published a teaser on this blog about using the Web as a whole as a data storage object. At that time I said that “the Web right now is cut down into a million pieces that don’t talk to each other properly”. Almost two years have gone by since that article and it looks like not much has changed.

One of the early questions was how interoperable are Web services when they’re not envisioned and created by them same company. This problem lead to a number of initiatives that are trying to push forward Web services creation standards. DataPortability, for example, is evangelizing a number of different standards that will create a better interoperable Web:

• end user authentication through OpenID;
• inter-application authorization through OAuth;
• information syndication and distribution through RSS, RDF and OPML;
• information meaning and automatic extraction through microformats;
• user attention profiling through APML;
• messaging and information brokerage through XMPP.

This collection of standards and best practices is great when a large number of companies start following them. For us, developers, it means that by following these standards our Web services will be interoperable with all other Web services that use the same standards. It means that creating a Web service now is much easier than it would have been two years ago.

What about the end users? How can they take advantage of this interoperability? I’m not just talking about Web services that let you consume data, because that problem was solved a long time ago by aggregators. Aggregators are a good example of a class of Web application that survives because there’s a de facto standard in place: RSS.

So, my point is, how can end users take advantage of Web services that let you publish, transform and assemble information? We’re moving to a point where a number of emerging services give you a one-to-many publishing approach:

• Ping.fm and HelloTxt publish your status across multiple services, like twitter, jaiku and Pownce;
• Typepad’s Blog It publishes blog articles across different platforms and also announces them on different status services;
• twitxr publishes your pictures across different services like flickr and Picasa.

Is it just me or there’s a pattern emerging here? Users see value in these services because they save you precious time by automating repeatable actions, like publishing a picture across different services. One thing to notice, though, is that these services only provide half of all that’s possible with the existing Web.

All these services let you choose among a number of services and then broadcast your information to all of them. Forgetting minor format and content adaptations, they won’t give you the possibility of programming the flow of your information. One thing is to shoot a picture and send it to different services, another thing is letting users tell how that picture flows through different services.

One service that’s offering you the capability of configuring this flow of information is switchAbit. It evolved from an original idea by Dave Winer that you could grab your pictures from flickr and post a tweet for each one of them. Quoting Dave’s original post:

The SwitchABit platform was developed because we noticed that an ever more complex flow of ideas and information is being facilitated by editorial systems and aggregators such as Flickr, Facebook, Twitter, FriendFeed, Seesmic, Qik, Ustream, YouTube, BlogTalkRadio, Disqus, Wordpress, Tumblr, TypePad, Blogger, etc.

switchAbit is basically an RSS to publish mechanism. It’s built around the pub-sub paradigm which means that it will get your information from a number of services, filter it according to your instructions and publish part of it into other services.

With this approach you’d still have to publish your information on at least one supported service, so that switchAbit grabs it and routes it somewhere else. Another approach is acting like a reverse aggregator, extending the functionalities of Ping.fm and others by adding the possibility of configuring information flow.

You could, for instance, add a watermark or a copyright notice to the picture, extract EXIF geo-location information and send it to Fire Eagle, publish the transformed picture on a number of services, and announce it to your contacts on some social networks. And this is just an example of what can be done in the near future.

I’ve been working since January on such an application. It has an interface similar to Yahoo! Pipes, but it lets you compose the flow of information from a starting point through a set of Web services that exist on the cloud. Because of the obvious similarities of this concept with the familiar UNIX pipe, it’s called tarpipe. Quoting tarpipe’s blog original post:

tarpipe will also create an ecosystem where Web applications and services will be able to receive and transform media content. Users will take advantage of this ecosystem by defining delivery and transformation workflows for their documents.

With tarpipe you can direct the output of one Web service into the input of another one. This makes different services virtually interoperable, even if they’re not able to talk to each other individually. It also gives end users the ability to compose flows of actions (or workflows) for their information. It currently accepts information sent through email and a REST endpoint, meaning you can extend your application by connecting it to tarpipe.

So, my initial thought that “the Web right now is cut down into a million pieces that don’t talk to each other properly” is not so true anymore. There are ways of making the Web more interoperable, like following de facto standards and creating programmable service adapters.

At last Thursday’s Ignite Boston, which I wrote up in a previous blog, provided an unexpected mirror in which two opposing views shined on each other, each view provided by one of the two keynotes by John Viega and Jonathan Zdziarski.

Both Viega and Zdziarski.are security experts and authors of books by O’Reilly and other publishers. Viega used the bully pulpit for an entreaty against the “full disclosure” philosophy, a fundamental article in the open source catechism. Zdziarski, who had not consulted with Viega beforehand, endorsed full disclosure whole-heartedly and with a doggedly pragmatic intent. The context for Zdziarski’s approach is the Apple iPhone, which has security vulnerabilities that, in his experience, Apple doesn’t fix until they’re made embarrassingly public.

Today Zdziarski sent me a long and frightening article from the National Journal about the threat of cyberwar. Although the basic premises in the article have been circulating for years, many of the details were new to me. And despite the focus of the title on China, the article makes it clear that governments as well as individuals (the “cyber-militia”) are engaging in disruptive behavior around the world. In fact, the article cites worries about what may be happening in the NSA.

It seems to me that the National Journal article provides more fodder for Viega than Zdziarski. Veiga insisted that the black hats planning DDOS attacks and identity theft aren’t as smart as they are commonly made out to be. They couldn’t create as much havoc if they had to rely only on the vulnerabilities they found themselves. They are helped immeasurably, he said, by the revelations of vulnerabilities in major software products by people with no malicious intent. The worldwide database of known vulnerabilities is swelled by individuals trying to show off their technical chops, and by companies in the security business trying to demonstrate the indispensibility of their products.

So long as software vendors are slow to fix bugs, full disclosure has to be an option, a kind of last resort, and I think Viega allowed for this. Open source projects have to promote a sense of responsibility among contributors to be discreet in reporting bugs with security implications. Perhaps it doesn’t matter much anyway–because most people keep using unpatched versions of software long after fixes come out.

Giles Bowkett’s Never Hate. Only Destroy. (disclosure: contains language your local third graders probably use and your work filter might block as inappropriate) contains a side point which crystallized something I’ve pondered for several weeks:

The whole point of the Cory Doctorow Problem is that the fundamental assumption with Internet celebrities - that a very smart person will always be interesting - is false…. What irritates me is essentially a search failure; I can seek excellent insight on social software and end up reading pointless trivia about a corporate amusement park filled with plastic birds on plastic trees.

This is my problem with current social networks as well. Your information is either public or it’s not. You’re either connected to someone or you’re not. There’s little to no sense of context.

The ever-creative Wade Olson (of KDE fame) tells an interesting story of immediately losing interest in otherwise-interesting hardware due to “Intellectual Property” protections. He caught himself going from caring to not caring in the time it took to read the phrase “Don’t expect Linux support anytime soon.”. His conclusion is:

Vendors need to beware: Intellectual Property gains, once thought to be a Competitive Advantage, will continually over time become a negative branding attribute.

I’ve noticed this myself. I don’t particularly care what Microsoft does, what NVidia does, or what Adobe does. Their products don’t really matter to me when I can use other products without giving up freedoms I consider essential.

Have you had similar experiences?

A proposal to help editors work better with dynamic languages — by not pretending they are static, and by leveraging their unit tests.

As a Test Driven Developer, using dynamic languages, editors frequently disappoint me. The main thrust of editor research, for the past few decades, targets debugging static languages. This post suggests a very simple fix.

For years, many people have argued that one of PHP’s big successes is deployment. The language has little to recommend it for anything beyond simple database-backed HTML templating, but there’s little easier than dropping a couple of .php files in a directory through FTP.

While there are still millions of wonderful (and ultimately unproductive) flamewars about how mod_php is faster than vanilla CGI Perl and Ruby uses too much memory and FastCGI is unstable and shared-everything on a monster JVM is obviously more scalable, none of that will ever matter to most of the deployed PHP code in the world today.

A Perlbuzz commenter named Yudel made the deployment/colonization point very clearly:

I still think in Perl, but as an only occasional programmer, I seldom find it the best tool for the job. The Perl community failed to successfully colonize the new ecosystems of programmers who don’t have root access. Simply asserting that PHP is linguistically inferior won’t convince anyone who has had to argue with a web hosting company about the load MovableType was placing on their servers.

mod_perl is great for what it does, but it’s clear that mod_perl isn’t what hosting providers most wanted. A slim Perl distribution — including perhaps a new Apache httpd module which only embeds Perl — with a good templating module, the DBI, and perhaps an XML parsing module or two could have put Perl on more $4.95/month hosting plans. The corollary to that of course is an easily installable bundle of Pure Perl for an application.

Sure, that doesn’t cover everything. You probably can’t get RT orPlagger or Angerwhale in such a system, but it’s a start.

Ceding the very low end of a technology to an upstart is just one of the ways to let distruptive innovation eat your lunch.

One flaw in this argument is that approximately zero webhosts supported Ruby before the Rails lovefest. As well, the Rails deployment strategy went through several iterations. Here’s the interesting point which subverts my argument somewhat: Rails hosting suddenly became lucrative enough that several Ruby-friendly hosts appeared.

I haven’t yet figured that out.

Yesterday Google celebrated the opening of a larger Cambridge, Massachusetts office, which takes up a substantial part of a building right next to the Kendall/MIT subway stop in the higher-than-high tech area of East Cambridge. I got a look at their new Friend Connect service (covered in a related Radar blog) and heard some fascinating comments that the staff kindly let me reproduce here.

Google staff certainly know how to say the right things and react in ways I approve to the situations Google finds itself in. More and more people I know (including authors) are Google employees, which is statistically predictable because more and more people in general are Google employees. The Cambridge office has been growing wildly since it began with the purchase of the company that created Android. And this office is one of 45 Google offices around the world.

This raises the question of whether the empire can be supported through continued sales of advertising, and whether Google’s stated openness carries through to employee behavior on the ground. I explored these questions with managers and staff at

I like numbers. They can mean a lot of things.

Rather than continuing silly arguments over obfuscated and flawed measurements of “language popularity”, perhaps a better way of measuring the viability of a language or platform is to measure the freshness of its ecosystem.

LaPerla’s How Fresh is the CPAN? measures the upload dates of one of the world’s largest and most active repositories of free software. Of the 12,000 (or is it 14,000 now?) distributions on the CPAN, 25% have a most recent upload date of February 2008 or newer. Half have an upload date of 2007 or newer.

You don’t get those kinds of statistics by putting “Ruby Programming” into Google and pretending the results are meaningful.

I promised to explore the theme of Free-loading Adoption of F/OSS in more detail. Alan Rimm-Kaufman’s Why Small Businesses Should Support Open Source is a great place to start:

It doesn’t matter if your donation is large or small. It doesn’t matter if you give money or code.

What does matter is this: if you’re benefiting from the Open Source Movement, try to give something back.

It makes good business sense. And it is the right thing to do.

Before I joined O’Reilly, I worked in a small consulting company. As I joined, the company was migrating away from proprietary platforms to open platforms. We saved a tremendous amount of money and gave our customers far better service. Being able to use, modify, and redistribute free software let us finish jobs we’d never have been able to do otherwise.

In return, we submitted bug reports. We occasionally submitted patches, both on and off the clock. We knew that we owed a great deal of our business to a healthy commons of free and open source software… and now I know that keeping that commons free, open, and healthy was vital to the business.

The owner of the company allowed the Portland Perl Mongers to meet in our offices once a month. He rented chairs for the meeting. Maybe it’s not a big thing, but it was a way to pay back part of one of the communities which had produced so much great software integral to our business.

You don’t have to hire a core developer. You don’t have to release your own (non-derivative) source code. You don’t have to donate money to a foundation, or host a conference or a meeting.

You don’t have to contribute back to the communities which produce software you rely on… but if you rely on it now, aren’t you interested in its healthy future as well?

Issue 16 of [IN]Secure Magazine is available. Mirko Zorz interviewed me in this edition (Page 41). If you decide to read it, I’d be delighted to hear your thoughts and feedback. The magazine edition of the interview is much better looking and highly recommended (as are the other articles), but for the sake of convenience, the interview session is below.

Small ISPs such as Glass’s (and yes, they do exist, even today) have none of the incentives that network neutrality advocates attribute to major carriers to discriminate against voice, video, or other content. In fact according to Glass, new applications such as VoIP are great because they provide new business. “This week we hooked up a VoIP company which was dissatisfied with the quality of service it was getting from the incumbent in our area. We deployed a low-latency, high-bandwidth radio link just for them, at a cost (parts and labor) of about $1,000. We can justify the cost because we will be paid for the service. It’s cost-shifting without compensation that’s the big issue for all ISPs–large and small.”

Glass has a stake at least as precarious in the current Internet economy as the media companies using peer-to-peer transmission as part of their business plans. Laws or regulations that fail to take economics into account, in his view, could put small ISPs out of business. He defends his position fiercely, and gets plenty of flak in return. I consider Glass a friend and have even planned to tap him as an author on some projects. So I want his view heard as a balance to the “just throw more bandwidth at it” proposals.

That said, I wonder whether the problem is really peer-to-peer protocols, which Glass focuses on, or high-volume media such as video. What architecture could handle the video experiences Internet users want. Compression can achieve impressive quality at reasonable bandwidth, but the sheer volume of everybody sharing the network stresses current transmission systems.

Matt Asay kicked up a small controversy in MySQL adoption: Deep and wide when he wrote:

Now the only thing missing in that conversation is the enterprise stepping up to pay for some or all of its free-loading adoption of MySQL. This is what is prompting MySQL to consider new licensing models. It would be very easily resolved by enterprises for owning up to and paying for the value they derive from open source, very little of which comes down to a lower price tag.

I’d like to extend that to projects beyond MySQL and to a definition of “contribution” far beyond opening a checkbook. Here’s my thesis: if your organization derives some benefit from a community-driven software project, you have a moral obligation to contribute to the health of that community in some way.

I’ll write more about this tomorrow.

There’s been some recent chatter and speculation on the upcoming enhancement to the PCI standard. Among the discussions, I’d like to publicize my opinion on one argument I’ve heard multiple times during the last few days. The argument goes something like this: The cost of performing security code reviews is too high, but the cost of performing black box reviews and/or implementing web application firewalls is lower. Therefore, the solution is to recommend that organizations rely on penetration assessments and/or web application firewalls.

Remember Andy Lester’s rant about Can’t You Just…?. There aren’t often easy answers in any field.

I really like what Chris Cummer had to say in a comment on “All I Need is a Programmer”:

Every time you use “just” to describe a feature or a process it tells me you’ve made a gross assumption about what I’ll need to do.

(Of particular amusement is when non-contributors tell volunteers what to do in free software projects.)

A conference attendance that tops 2000 suggests that a technology involves a certain number of subtle angles. MySQL became a hit because installing it and manipulating tables were so simple–and yet when you get serious, the simple things start growing hair.

If you look at the CPAN test reports for Parrot, you’ll see that the pernicious and persistent problems relate to odd bits of not-quite-always-cross-platform math, specifically floating point numbers and not-a-numbers.

It’s reasonably easy to find and read the C89 and POSIX specifications. They’re well-published. Even if there are confusing parts and contradictions, you can look for them and find the specifications.

Now try to find current information about how various operating systems and the various compilers and major versions and minor versions and libc versions in all of those combinations interact. In short, if I want to figure out exactly what OpenBSD does in its C library in its standard configuration to return a negative floating point 0.0 (or not, as the case may be), where do I go, what do I read, who do I talk to, and — most importantly — how do I change the software I work on to deal with those platform-specific quirks such that the users of my software don’t even have to know that these quirks exist? Ditto GNU/Linux, Cygwin, FreeBSD, Mac OS X, et cetera.

I know that knowledge exists somewhere. Perl 5’s core encodes it somewhere. I suspect the same is true of Python and Emacs and Guile and Glib and kde-base or related projects. Yet the knowledge in code is only useful in two ways. First, in projects that use the code directly and only need to trust that it does the right thing. Second, if other people read the code.

There ought to be a third option: encapsulating that knowledge outside of code somewhere. Maybe this specific case is documented and I just can’t find it. The same goes for alignment concerns (64-bit Intel/AMD, PA-Risc, Sparc, ARM), pointer sizes, and other information.

Does this information exist in one or two good places, or does everyone have to track it down on his or her own? Worse, does everyone just hope these problems never come up?

Didn’t get one of the 10,000 golden tickets in special Google-brand chocolate bars? Python isn’t your favorite language? Not sure about hosting your code and data with the world’s largest ad broker? Never fear — Google’s not the only supercomputing grid in the world. It may not even be the largest.

Computerworld reports that the top botnets control over a million computers and can deliver over a hundred billion advertisements per day. MapReduce and AdWords have nothing on this.

Yes, the deployment platform is mostly Windows, and you don’t exactly have professional system administrators in charge of every whim and need of the machines, but you do have root access, and there’s little chance the box owners will suddenly yank your code and data if your business model conflicts with theirs. Google’s offering has a ways to go if it wants to compete.

Benjamin Otte’s Open Source will scale brings up an interesting point.

If currently 1% of the world uses GNOME and it suddenly were 100x as many, we’d be at 40 million bugs right now.

The persistent lie that increased usage guarantees hordes of available volunteers descending from heaven to wipe out all signs of resistence (accompanied by the appropriate Wagnerian soundtrack) is one of the most pernicious Myths Open Source Developers Tell Ourselves.

Though Benjamin finds hope in the fact that users tend not to report bugs (and distributions don’t work well enough with upstream), the entire scenario still bothers me. I don’t mind fixing an interesting segfault now and then, and I’m always happy to fix a well-reported bug with a test case and a sensible description which helps me reproduce the problem. Yet I don’t scale, especially for projects where I can devote only a few hours a week.

That’s not a few hours every week, either.

Bug reporting, bug triaging, and bug fixing are all activities present in healthy project communities. In return for the freedom of using great software (often at low or no cost) with no usage restrictions and few (if any) distribution restrictions, users have the responsibility of ensuring the community’s long-term health. That may mean submitting a bug report, testing a development version, posting a bug bounty, producing a patch, or even sponsoring a developer. Otherwise, you’re relying on the goodwill of volunteers who’ve already more than paid their obligations to you — and I’m concerned about the long-term sustainability of that model.

Sometimes I wonder of the dual-licensing model is actually healthier in some ways. At least there the costs are explicit and fungible.

I’m at a unique symposium this week named Codework, which I do not dare to describe because it has barely begun. I can only say that snagging Ted Nelson to deliver the opening talk was not only a great motivation for attendance but an exquisitely appropriate historical marker for the workshop, which bills itself as “Exploring relations between creative writing practices and software engineering.” Nelson, of course, is one of the first people to recognize the benefits literature and computing could offer each other.

Readers have plenty of ways to learn about Nelson’s famous Xanadu and his more recent project Zigzag, one of the best ways being to hear him speak as we did in a full hall last night. Thanks to the World Wide Web that Nelson perennially maligns, he is much more famous than he otherwise would be, and also has much more opportunity to spread his views. But here I’ll just jot down a few observations I haven’t seen others offer about Nelson’s ideas, and that aren’t immediately obvious from his talks, fascinating and well-argued as they are.

Adobe has released a beta of AIR for Linux. Good news, everyone with a 64-bit processor, or PPC, or Sparc, or ARM, or anything more exotic than 32-bit x86. AIR for Linux Release Notes say that all you need is:

Processor - Modern processor (800MHz or faster)

Finally, web applications have freed us from the tyranny of worrying about such difficult issues as endianness, alignment, and struct padding!

(Sorry “by the end of the day” turned into four days. I was in rural Pennsylvania with no Internet!)

In 2006 I was taking a look at the then unreleased XenSource XenServer 3.0. The server was running on a Dell laptop on a filing cabinet next to my desk where the XenServer management interface was open on my desktop. My wife walked into our office, looked over my shoulder, and while pointing to the monitor on my desk asked, “Is that the Xen thing you said you were going to be reviewing?” I responded that the laptop next to me was running Xen, and that she was just looking at the management software.

The fact of the matter is that to most people, the software that manages virtualization *is* virtualization — a fact that may save companies like VMware. See, the virtualization management interface is the most public facing component of the virtualization ecosystem, and two crucial parts of this ecosystem are quickly becoming commoditized: the hypervisor and the virtual machine (VM). The entire virtualization ecosystem is being redefined, and in a few years the companies that wish thrive in this market will need to focus on an entirely different set of technologies than they like to tout now.

This blog briefly discusses the commoditization of the different parts of the virtualization ecosystem and what areas companies like VMware will need to pay attention to in order to survive software giants like Microsoft and open source alternatives such as Xen and KVM.

I chuckled at a couple of quotes in Java performance improvements touted, specifically one from Cliff Click:

As your program grows in size, the lack of strong typing basically kills your ability to handle a very large program and so you don’t find the million-line Perl program.

I’ve met Cliff, and he’s very smart, but I have to disagree on two points. First, no one who’s used anything with a better static type system than Java consider’s Java type system “strong”. (If you can still get a NullPointerException from a generic-enhanced collection, Java has a ways to go.)

Second, the reason that there aren’t many million-line Perl programs is that the people who are capable of writing and managing million-line Perl programs have better ways to organize their projects than glomming a million lines of Java into a single shared-everything instance. That’s setting aside the qualities of encapsulation and abstraction that Java-the-language doesn’t have, preferring instead to push that problem to tool vendors and AbstractFactoryFactoryInjectors which consume vast swaths of XML to get around Java’s static code fetish. I can only imagine how much larger the Java code would be without all of those XML files.

I also recommend James Robertson’s take on things, from Earth to Sun.

I’m curious to hear how many million-line Java applications exist in the world and what they do. I suspect that they’re primarily web applications that speak SOAP or REST over strict SOA or HTTP boundaries — just the sort of boundaries beyond which it doesn’t matter if your code is Java, Perl, C++, or the Korn shell. You know, because they’re completely network bound.

Hello. My name is Andrew Kutz, and I am honored to be blogging for ONLamp on the topic of virtualization. Please watch this space for news on VMware, Xen, KVM, and other virtualization technologies. I’ll be creating my first real post later today. If you have any ideas with regards to what type of content you would like to see, please let me know by shooting me an email at akutz at lostcreations dot com.

Thought for the day: If the preferred scaling strategy of Java web applications is shared-everything in a beefy JVM with plenty of threads in myriad pools (and it seems to be) and the preferred scaling strategy of LAMP applications is a shared-nothing architecture across plenty of boxes with memcached in front of a replicated database, what changes will be necessary to run popular apps written with shared-nothing in mind in a shared-everything environment?

Bonus question: besides web applications and language research, are dynamic languages on the JVM interesting? (The clever reader will see where this line of thought leads.)

In my (painfully) long career as a software engineer, I’ve often run across the attitude that code has intrinsic value. You see this frequently in the industry when ‘code reuse’ is used as a metric of efficiency. At several companies I’ve worked at, old and badly bit-rotted products have not been rewritten because “we’ve invested umpty-umph million dollars in that code, we’re not just going to throw it away.” This whole attitude is bull-doodoo, and here’s why.

In amusing synchronicity, I was reviewing Bernard Golden’s Open Source Maturity Model earlier today. Then I read My Visit to Sun, where he describes a conversation he had with Simon Phipps before giving a talk at Sun.

In particular:

Many enterprises seeem to operate in a vendor-centric model: they select a vendor and from then on rely on the vendor to define when new technologies should be adopted, when new releases should be rolled out, even what complementary technologies should be implemented. It’s obvious that this causes middle-of-the-pack performance, lock-in, and lack of pricing power. Without rehashing all of those arguments, consider the other implication of this approach: it fosters dependence — an inability to self-direct in technology direction, custom architecture, and unique business offerings. If all you can offer is off the standard menu, you will never serve up differentiation.

When you give away software and trade license fees and pre-sales for support contracts and free downloads, you break the passive-adversarial model between vendor and customer that has served IT so poorly for the past two and a half decades.

That’s not a safe thing, nor an easy thing. That’s still a good thing.

Companies are constantly opening new veins of ore as they attempt to mine the Internet for useful information. Developers and open source system users will be particularly interested in a SourceLabs announcement of a service called Self-Support Suites that has been in beta since December. This tool combines enormous amounts of information indexed by SourceLabs from bug trackers, technical mailing lists, and other sites to help open source users diagnose problems. They’ve just put up a free download.

The proof of concept I heard from Byron Sebastian, CEO of SourceLabs, concerned a site that spent two weeks trying to track down the failure of an Apache Project module. SourceLabs’s system found a bug report with the fix in a few minutes by finding a match between a stack trace provided by the user and a stack trace provided by a question in a public forum message. This search was more difficult than it might sound, because stack traces don’t match precisely and their contents are not unique strings that are easy to search for. Sebastian says that stack traces and log files tend to have the most useful information–but if other information was organized better, it might rise in value.

Oh, joy. Adobe is at it again.

AIR applications are deployed as a single AIR file that works identically cross-platform. The api’s within AIR are identical across different operating systems so any application behavior will work the same regardless of where it is running. Regardless if you use HTML/AJAX or Flash/Flex to build your application the API’s are identical and run on MAC/WIN/LIN without issue.

Ted Patrick, Why Adobe AIR?

Given that Adobe’s evangelists have a very difficult time telling the truth about which platforms Adobe actually supports (particularly pernicious with regard to Flash; see Uh, Thanks for the “Linux” Support for one example), does anyone really think that AIR will run on anything more exotic than 32-bit x86 GNU/Linux? Set aside the fact that, as much as Ted’s quote may make you think that AIR runs on “Linux” right now, it sounds like no one outside of Adobe will see that binary blob until later this year.

When I think about cross-platform support, I think about the first time I sent e-mail on the Internet via a FidoNet gateway accessed through a PC bulletin board from my Commodore 128 over modem-to-modem dialup in the very early ’90s.

Again (I always have to disclaim this), Adobe has every right to support only the platforms and processors it wants to support. I have no problem with that.

As usual, I offer any Adobe evangelist, manager, or developer the chance to prove me wrong, publicly, by successfully installing a publicly released version of Adobe Flash on the GNU/Linux laptop sitting six feet behind me in my office. (Good luck; it has a PPC CPU.)

Just don’t tell me that you offer cross-platform support and then stick me in a ghetto because I’m using the wrong operating system and the wrong processor. I know what cross-platform support means — you can still browse the web on a Commodore 64 — and your walled garden isn’t it. For all its flaws (don’t get me started on the codec licensing nonsense), Moonlight has a better claim to cross-platform compatibility. For starters, it doesn’t lock you out if you happen to be using the wrong type of CPU.

(I thought one of the goals of high-level programming languages and frameworks and virtual machines was so that you don’t have to worry about the details of the lower levels. Of course, I thought one of the goals of web applications was independence of platform at the level of operating system and below. Shows what I know.)

Inkscape developer (and fellow PDXer) Bryce Harrington mused about fixing critical bugs in Inkscape 0.46 to be in The paradox of FOSS projects supporting Windows.

Unlike the philosophical argument described in The Dubious Benefits of Porting FOSS to Windows, Bryce makes a more concrete, more pragmatic argument. In particular, the ratio of potential contributors to users on non-free platforms is measurably smaller than the same ratio on free platforms.

I’ve noticed this problem in some of my own projects. There are plenty of users willing to try a piece of software that may or may not work well on a non-free platform, but when it comes time to debug and fix these problems, their motivation goes away.

I’m sympathetic; it’s not fun to try to build and debug software on Windows. I don’t use it. I don’t understand it. I’m not the person you want telling someone how to install any of the free compilers that somehow don’t come bundled with Windows, just so that someone might be able to produce an interesting backtrace.

I’m not sure there’s a good solution, at least without enfranchising users to become contributors — and that seems to require Free platforms.

Back in December, I was one of the 170,000 eager Linux geeks who forked out $400 for the privilege of getting my hands on a One Laptop Per Child XO. To be honest, my initial impression was not great. For one thing, the keyboard was (and continues to be) the wrong size and feel for productive typing by an adult. I really can’t complain about this, as it wasn’t designed for adults, and I have managed to adapt somewhat to get something approaching a reasonable typing speed.

More troubling was the poor state of the WiFi stack, which seemed incapable of connecting to a WPA encrypted network. Even when using WEP or no encryption at all, getting and keeping a connection seemed to be a hit or miss adventure. And while the Sugar OS was certainly innovative, it really started to grate on me when I tried to do the kinds of things I wanted to do with it.

A quote from Steve Jobs during the iPhone SDK Press Conference last week:

If they write a malicious application we [will] track them down and tell their parents.

In other words, the iPhone applications will need to be digitally signed by Apple, and the developers will be required to register with Apple. It will be interesting to see what kind of information developers will be required to provide to Apple to register. Will they ask for the developer’s credit card number? How will the developers authenticate their identity with Apple before they are allowed to submit their applications to be included in the store inventory?

If observation is the first stage of scientific discovery, watching what people are doing in a field will tell you what the academics and theorists will write about in a few years. By this reasoning, SD West and SD Best Practices are important bellwethers for programming theory, even though there’s little theoretical about the conferences.

On March 10, O’Reilly will release Jonathan Zdziarski’s book iPhone Open Application Development, currently available as an online RoughCut. Online and brick-and-mortar bookstores will have the book as soon as shipping permits.

Apple has just released a toolkit for application development on the iPhone. Readers will naturally ask: what is the relationship between the material in the book and Apple’s development environment? Can they apply what they learn in the book to Apple’s toolkit?

InformationWeek interviewed Steve Balmer on several subjects, including open source. Here’s a choice quote:

We’ve always tried to get innovative work to happen on our operating system, and I want Windows to be the number one destination for open source innovation.

Steve, you could stop by dropping the Unreasonable and Discriminatory Pay-to-Interoperate Tax on commercial open source distribution. These developers already pay the Windows tax. So do their customers. Taxing them again with a licensing scheme that allows you to pull the rug out from under their feet if you decide to compete with them is, in my mind, something other than encouraging development.

I suspect that true innovation doesn’t have to put up toll roads.

In a recent discussion on lkml (found thanks to LWN, Ingo Molnar defended checkpatch, a Perl program which reviews proposed patches against the kernel’s coding standards. In particular:

you might know that Deja-Vu moment when you look at a new patch that has been submitted to lkml and you have a strange, weird “feeling” that there’s something wrong about the patch.

It’s totally subconscious, and you take a closer look and a few seconds later you find a real bug in the code.

That “feeling” i believe comes from a fundamental property of how human vision is connected to the human brain: pattern matching. Really good programmers have built a “library” of patterns of “good” and “bad” looking coding practices.

If a patch or if a file has a clean _style_, bugs and deeper structural problems often stand out like a sore thumb. But if the code is peppered with random style noise, it’s a lot harder (for me at least) to notice real bugs. I can notice bugs in a squeeky clean code base about 5 times easier than in a noisy codebase. This effect alone makes checkpatch indispensible for the scheduler and for arch/x86.

…

I’ve yet to see a _single_ example of a good, experienced kernel programmer who writes code that looks absolutely careless and sloppy, but which is top-notch otherwise.

I’ve spent a lot of time this past year cleaning up a medium-sized codebase written in cross-platform C89 (c’mon vendors, can’t you upgrade your compilers tosupport C99 already? I know Microsoft has a few programmers on staff.). Coding standards help immensely.

Even something as simple as aligning the equals signs in assignments and separating paragraphs in code with newlines increases skimmability greatly. Ingo’s right though, and I wish I’d realized it consciously earlier. Consistent code requires much less brainpower to decipher its structure, leaving that much more brainpower to find real problems.

While there may never be a refactoring called “Beautify code block” (and technically that’s not improving the design of the code), it’s an important tool for human analysis of code.

I’ve slaved away for the past 8 months or so writing a book on Dojo. It’s definitely not a lot of fun moonlighting for such an extended duration on a single project, but my hope is that it’s all going to be worth it when I am able to deliver what I believe will live up to the “definitive guide” standard that so many other O’Reilly books have delivered in the past. (Yes, that previous link shows that the book is titled Developing with Dojo, but it’s really just a placeholder. The book has since been retitled, there will be a different animal on the cover, etc.)

The fact that I’ve spent so long working on what’s essentially documentation of all things should tell you that I’m a huge Dojo advocate, but why did I like Dojo enough to spend hundreds and hundreds of hours writing a ~500 page book on it?

Here are some of the reasons that come to mind immediately:

• Breadth and Depth - It includes a highly optimized JavaScript standard library that protects you from the bare metal of the browser and allows you to write portable code, a terrific set of widgets that you can drop right into the page, and build tools that you can use to compress and consolidate your JavaScript to squeeze as much performance out of it as possible. Those are really broad strokes, but hopefully you get the idea that it’s not just about widgets or just about DOM manipulation. It’s about creating a great user experience and necessarily includes a swath of stuff that helps you to make that happen.
• Awesome Community - The help these people have given me in writing this book is nothing short of incredible. It’s kind of strange to “know” people for months and months via an IRC chat room and somehow feel like they’re your friends, but that’s the way it feels. It’s amazing to watch some of the most hard core committers in the project field some of the simplest questions that come up with grace and patience. (And believe me, I’ve come really close to starting a riot a few times in there.) Drop into #dojo on freenode.net if you ever want to see just how inviting the place really is.
• Clean, liberal, licensing - This post does a great job of explaining, a lot better than I could right now, so have a look.
• Pragmatic philosophy - It doesn’t try to re-invent JavaScript or build a brittle, artificial language on top of it. It embraces JavaScript for what it is, plugs holes where they need to be plugged, smooths a few things out here and there, and otherwise leaves the language alone.

Now, without starting a flame war, I’m curious about some things:

Have you tried to use Dojo in the past and gotten frustrated and dropped it for something else? If so, why (and what did you end up liking better?)

Would a solid book on Dojo have made a difference?

Have you started with another JavaScript toolkit but migrated to Dojo because the other one didn’t quite meet your needs? You don’t have to say which one you moved away from, but it would be interesting to know why you ended up moving to Dojo.

Sebastian Silva is a bit worried. Three hundred thousand XO units will arrive in Peru by August, the largest deployment in the world by far. (Uruguay started this past December and will reach one hundred thousand laptops this year.) As a volunteer on the OLPC support team, Sebastian wants to make sure the children, teachers, and technical trainers are ready.

Sebastian sees the promise of computing as in social terms. With XO, every person becomes a TV broadcaster–and XO is even more empowering, because TV broadcasts can’t be chopped up and digitally altered. Sebastian says he learned Logo before he learned to read and write, and its mastery gave him a learning attitude that he’s preserved his whole life. He wants children not just to make movies and write papers, but to appreciate the drastically open nature of the systems they’re given.

“If OLPC succeeds, there will someday be more OLPC-like systems in the world than traditional PCs,” he says. “The Internet could very well be transformed by the behavior of people using the OLPC systems.”

You have probably heard of AJAX, but have you heard of Comet? Yes, they are both household cleaners you might find under your kitchen sink, but that’s only part of the story.

In the web realm, Comet refers to an architecture that enables a server to actually push data to a client without the client explicitly requesting it. Given that web servers were built to be very good at doing exactly the opposite over a stateless protocol, you should be quite intrigued at this point, so head on over to Comet Daily, a fantastic site that includes incredibly informative articles on Comet, to get some insight on what is going to be the next big thing in web programming.

Right now, there is a series running with the theme of Colliding Comets which includes a number of presentations and rebuttals that discuss opposing philosophies on how to most effectively realize Comet. Trust me, you won’t be disappointed. This is good stuff, and it’s full of substance.

If you find Comet interesting, you may want to also take a look at Dojo, an industrial strength JavaScript toolkit for developing web applications, because it includes cometd, a client that can talk to a Comet server if the server implements the Bayeux protocol.

My upcoming book, tentatively titled Developing with Dojo, includes an in depth example of cometd. We hope to have the book available via Rough Cuts very soon, so if you’re interested in using Dojo with Comet, keep an eye out for future posts about its release.

Eugueny Kontsevoy writes in Web vs Desktop Nonsense:

Can we see past the browser? Can we accept that browser is just a runtime library that most people do not need to download to consume your application?

…

Come on, the “anywhere” part should not come at expense of losing 90% of other features.

…

Yes, future belongs to web applications, but I am not so sure that browser, with [its] weak runtime and close to non-existent programmable graphics, should remain a necessary vehicle for it.

The more I think about the subject, the more I believe that HTTP is a truly successful distributed system because it doesn’t try to solve what most distributed systems tried to solve. It doesn’t try to blur the distinction between local and remote resources; everything is a document accessible through a URI. (At least, that’s true if you’re a RESTafarian, which you should be.)

The question Eugeney raises is important. We already know how to build decent — even good — native applications. Is the zero-footprint installation (minus several hundred kilobytes, if not a few megabytes of JavaScript, or a few tens of megabytes of Flash, Air, Silverlight, Moonlight, or Java) so compelling that everything has to get crammed into the web browser model?

In my mind, that’s a mistake as big as pretending that accessing a distributed component is exactly the same as accessing a local component.

(Quick musings on my lunch break.)

When Microsoft wants to compete, it acquires. (How many successful products have escaped the $7 billion a year Microsoft Research?) Apparently Microsoft wants to compete better in the world of the Internet, so it’s trying to capture Yahoo and all of those lovely page views.

As of this writing, Google Finance on Yahoo (I like the irony of that link) says that Yahoo has a market capitalization of almost $40 billion in 1.34 billion shares and a price to earnings ratio of 63.51. Apparently a lot of people believe that big profits are around the bend.

A $40 billion acquisition is a big deal, and apparently Yahoo’s holding out for more. Everyone has a price — if the board doesn’t like $40 billion, will enough shareholders like $50 billion? $60 billion? A new board might be more pliable.

However, instead of trying to make this acquisition work, Microsoft could turn its baleful eye down the street a little bit.

Chasing page views and a new market dominance in Internet advertising may not be in Microsoft’s best interest. Of course, you know that eventually every Microhoo page will have little pockets of Silverlight here and there. If the Rich Internet Application model threatens the Windows hegemony, Microsoft has to react to preserve the attractive profits of its cash cows. Right now there are two serious competitors. One is whatever HTML 5 and Ajax may produce in the next couple of years. The other is a somewhat cheaper acquisition target.

Google Finance on Adobe reveals a much smaller, choicer target, with a $19.91 billion market capitalization and 568.96 million shares. Microsoft might not even have to borrow money to make this acquisition work.

For half the price of the hostile Yahoo acquisition, Microsoft gets control over PDF (axed), Flash (axed), and Air (axed). Talk about a destination, too — there’s a long-standing rumor that the single page with the highest PageRank anywhere is Adobe’s page to download Acrobat Reader. Microsoft also gets a fair few developers who it can retrain to port Flash to .Net while they’re on the way out the door, and there’s your backwards compatibility strategy — at least for all of those platforms which have officially blessed, sanctioned, and (here’s the important part) legal-codec-supported Silverlight ports.

(Linux users, do you have the Novell SUSE Genuine Advantage?)

What Microsoft doesn’t get is a better foothold into the currently-lucrative online advertising market… but that’s the single leg of Google’s one-legged tripod. It’s difficult to see how selling ads will prolong the life of Office, however… but maybe that will give the Office product team enough time to port a few tens of millions of lines of code to the CLR.

I watch what people like Brian Aker and Brad Fitzpatrick do very carefully. As developers go, they’re among the best at making little problems go away so as not to distract them from developing, releasing, and maintaining software.

Brian wrote recently about encouraging very easy commit access to contributors. I can’t add anything more to what he said, except to echo his concerns about the implications for making a new release of software.

I’ve seen this pattern of open repository access in the past few years, first with the Pugs project, and then with Brian and also Adam Kennedy and Michael Schwern. It’s a model that works in certain cases.

As Brian mentions, a release has certain implications. The best way to know that your software works on the platforms you care about is to test it on those platforms — and I know all of these developers believe in the value of automated testing.

The great mass of developers might just now be hearing about distributed source code systems. That’s fine. Plenty of them will realize how valuable it is to commit without network access or to branch cheaply and locally, or to have the project history instantly available, changeset by changeset, when they need it. The problem isn’t a trivial one, nor is it a completely solved problem, but we’re beginning to understand it better, and the solutions we have now are decent and robust and continue to improve.

Now it’s time to figure out the next problem. As Brian described it:

I want all open source projects being filtered into a network where users can run slaves that do regression testing for them.

We’ll likely discover that on platforms so exotic that available developer support and expertise is minimal (Windows, anything non-x86) that the next bottleneck is people who can fix problems, but being able to track which patch broke support for a platform is highly useful. Maybe for now even the minimal ability of being able to submit a patch to a farm of EC2 instances to get back “Everything passed!” or “Something failed!” results in seconds, rather than minutes, would be an improvement.

Alternately, opening up commit access might mean that we can say “If you want this to run on your favorite platform, you’re going to have to get involved.” Maybe making it even easier to contribute to a project will finally let us be clear about the true cost of free software: if you rely on it, you must help the community support its long-term health.

Adriaan de Groot, vice president of KDE e.V., wrote a short essay on Target Platforms for KDE.

His divisions of four types of target platforms is instructive; the two major axes are free/non-free and Unix-like/other.

“Unix-like” is an awfully big grab-bag, however. The joys of trying to figure out dynamic linking on Mac OS X ought to put to lie the shrill claims of the turtlenecked faithful that it’s just BSD with a shiny GUI (maybe BSD circa 1987, when everybody wanted a magnesium case). I almost dare not even imagine how much fun it is to coax compilers on non-free Unix-like platforms into interpreting modern C++ correctly.

The nice part of free Unix-like platforms is that they’re easy to obtain and install. The monster machine sitting in my other office can run multiple VMs for *BSD and OpenSolaris simultaneously, so testing a patch for portability requires a little bit of system administration and a little bit of discipline to script the process. Testing a non-free platform, Unix-like or not, is much more difficult.

POSIX and free redistribution and source code availability gives us a much better chance of figuring out and fixing those problems than we’d have otherwise. However, that’s no substitute for platform-specific experience – having an OpenBSD VM running doesn’t mean that I automatically know why OpenBSD’s handling of, for example, complex math is different from that of FreeBSD, or how to fix it. Sometimes there’s no substitute for a little elbow-grease from a passionate user of the platform. It’s nice that projects such as KDE actively support it.

It’s a sad old story, a story we’ve all gotten tired of–the patent so brainless as to be almost worth citing as a creative act of industrial sabotage, yet awakened from years of dormancy with a hungry ferocity to claw and mangle everything in its path. This particular patent is being exerted by Trend Micro Incorporated against Barracuda Networks, Inc. for a firewall product incorporating the popular open source spam filter, ClamAV. Only this court case stands in the way of a power grab that would require all open source work on virus filtering gateways to cease.

The Trend Micro patent (5,623,600) simply suggests that virus filtering be provided in a firewall. That’s all. Patents are supposed to cover things that are novel, and not obvious to a person having ordinary skill in the art. This patent meets neither criterion. Although it was filed in 1995 and granted in 1997, Barracuda has found a good deal of written evidence that filtering at the router was widespread earlier. And if lots of people are installing virus filters on their desktop computers throughout a company–any fifteen-year-old could say, “Why don’t you put it all in one place under the control of people who know what they’re doing?”

Ted Neward attempted to pull apart some of the silliness in the debate over scalability with Can Dynamic Languages Scale?. In particular, one of the most important insights is:

There’s an implicit problem with using the word “scale” here, in that we can think of a language scaling in one of two very orthogonal directions:

1. Size of project, as in lines-of-code (LOC)
2. Capacity handling, as in “it needs to scale to 100,000 requests per second”

I think it is extremely important for an organization to account for the reality of doing business (Risk based approach compared to the purist mentality of securing everything) when strategizing an information security plan. It is true that an individual who has a habit of perceiving security issues as purely a technology problem without understanding the business reality is likely to make bad security decisions.

However, I think some people in corporate security take this argument too far and end up awarding critical roles to individuals that do not have the appropriate skill-set and mind-set. More often that not, this happens when organizations responsible for information security misunderstand the argument to mean that you only need to probe for the understanding of business fundamentals and process management when recruiting for talent. Depending upon the criticality of the role awarded, this can deem disaster.

Apple did a beautiful job creating this device. Millions wanted it the moment it became known, and thousands wanted to write programs to explore its ground-breaking interface elements. Apple, however, failed to release its APIs, much less any toolkit or run-time environment.

So the community built its own.

Not only is free software development unprecedented in its size and geographic spread–hundreds of people from countries around the world collaborating on individual projects–but it brings together people who are notorious for having trouble dealing with other people. That’s really impressive when you think about it.

Of course, the stereotype of the computer programmer with Aspergers Syndrome is overblown. I used the term in my title to attract attention, but I’ve worked with enough programmers to know that many warm and socially sophisticated people take up the job.

Let’s put it more gently: many programmers have the feeling their people skills haven’t kept up with their technical mastery. That’s why they are attracted to sites such as Perl hackers Michael Schwern’s geek2geek, whose motto is “What we have here is a failure to communicate.”

How does free software development work so well, then? People often remark that the Internet made the explosion of development in the mid-1990s possible, but they focus (wouldn’t you know it!) on the Internet’s technical functions: instantaneous transmission, exact replication of content, etc. Occasionally a general “nobody knows you’re a dog” comment gets thrown in too. But we have to consider the social behavior encouraged by the tools the geeks developed.

J. David Blackstone has a pointed journal post entitled The Right Way To Do It which praises Perl’s “There’s More Than One Way To Do It” philosophy:

TMTOWTDI is anarchy. It scares people who want to keep order by force.

Allowing people the freedom to choose from many different ways of doing things is a recipe for disaster, we’re told.

… in my experience, it’s been the Perl code I’ve had that is readable, well-designed, and maintainable. It’s been the Java code I’ve seen that is ugly, poorly-designed, and unmaintainable. There are certainly exceptions to both sides of this.

Yet it’s not about Java versus Perl (and certainly not Perl versus Python).

“Perl is dead”, crows TIOBE’s January 2008 index. The world belongs to Python.

You see what you want to see in statistics though.

For example, you could compare Perl, Python, PHP, and Ruby job trends. Don’t drop those sigils yet.

Or compare Perl’s delta to C’s delta. Both lost ground in the TIOBE index, but C declined by almost twice as much.

Here’s a fun one. TIOBE’s editorial says that C# and Java will eventually be the two most popular languages. To do this, C# has to surpass Perl. That’s a problem though; it gained more than Perl lost and still slipped a position and is still more popular than Perl.

Ultimately this isn’t even good stats porn though. There’s no analysis of why languages have gained or lost in popularity. Without that, there’s no good way of deciding what these statistics mean. Without that, it seems silly to declare winners and losers and long-term trends. (One might also suspect that the actual release of Perl 5.10 and the buzz around that from the second half of December versus the “imminent” release of Python 3 may shift numbers from this point on.)

I’ve long believed that the easiest way to install software on a modern operating system is through a well-designed package manager connected to one or more carefully-maintained package repositories. Thus my brain always shudders when someone says “OH it’s so EASY to install software on MACZ just drag and drop! woo!!” (Why should I have to fire up a web browser, navigate to a website, find the download link, figure out which version works with the dependencies I have installed including the OS version, pick a mirror, and then figure out where the file actually downloaded? I suppose it’s likewise easy to get a Ph. D. in theoretical physics — just walk on stage when they call your name.)

I do remember the bad old days when installing something reasonably fresh required me to trawl through rpmfind.net looking for, if I were exceedingly lucky, an RPM built for the particular version of the particular distribution I run, or barring that an SRPM that I could coax into doing the right thing. There were still benefits to using a packaging system (mostly dependency tracking), but that’s more work than I want to suggest to my parents.

I was late to Debian (my first installation was 1999), but apt-get was a clear improvement for installing and updating the entire operating system, especially when combined with the quality and breadth of packages available for Debian.

These days I use aptitude, which is even more so.

Again, I’m not sure that I would suggest that my mother make a habit of running this on the command line by herself, but she’s perfectly capable of copying and pasting a few commands from an e-mail to keep her system up to date or install new software, and the process is much simpler than giving her a list of directions to navigate a web site. I doubt I’ll ever catch her running aptitude search, which is fine… but I use the command frequently.

The process of installing software in such a way that it does not conflict with other software, includes dependencies in a sane fashion, and receives security updates almost automatically for the whole OS (not just the kernel, GUI, bundled web browser, and DRM-laden media player) is now something I almost don’t even think about. That is the sign of a truly useful piece of software.

Thanks to the contributors to Aptitude, apt-get, dpkg, and the Debian and Ubuntu repositories.

I’m glad to see that TPF’s public relations group has spread the Perl 5.10 press release far and wide, and it’s getting some coverage. However, some of that coverage reminds me why I don’t watch television news and why I treat the newspaper as entertainment and not information. Consider eWeek’s First Release of Perl in Five Years Arrives:

Perl is a dynamic scripting language widely used in everything from Linux system utilities to Web servers to full-blown graphical enterprise applications.

What’s a “dynamic scripting language”? Is there such thing as a non-dynamic scripting language? (No one seems to know what a scripting language is anyway.) Minor nit.

During its 20-year history, it gained massive popularity by assimilating the syntax from many predecessors, making it really easy to use for anyone already versed in sed, awk, grep, csh, C/C++, Lisp, and so on.

Syntax, maybe (but Lisp? Really?). Features, sure. Easy to use? That’s debatable. Easy to start to learn, yes. I don’t know that anyone will suggest that Perl is easy to master, though I’m happy to argue that its learning curve is gentle if long.

… languages like python with rigid syntax structure have arguably gained ground in recent times over perl, for applications that are developed collaboratively.

“Arguably” is a weasel word, so you can throw out this whole sentence. I’m not aware of any statistics that show that Python is more popular than Perl. (Arguably, the Maginot Line gained a lot of ground in the southward direction. There. Now Python fans and the French can berate me in the comments.)

Additionally, scripting languages specially-made for use on the Web, like PHP and Ruby, have eroded some of perl’s once formidable share of the dynamic Web server scripting scene.

There’s that “dynamic scripting” mess again. What does that mean? What’s static Web server scripting anyway, and why would you need a programming language for that?

My favorite part however isn’t about Perl at all. Did you catch that? Apparently Matz was really busy in 1993 writing Ruby not as a general purpose language but specifically to use on the nascent Web. How prescient.

If a journalist can rephrase a press release and make this many errors in five paragraphs in a subject I know something about, how many errors are there in subjects I don’t know as well?

Oh, and the title comes from the last television news promo I ever watched, during the X-Files finale. The local Fox affiliate played a blurb for the evening news where the newsbimbo said, and I am not making this up, “Now that the X-Files is ending, let’s see what the series taught us about real aliens. Stay tuned at 10.”

I haven’t read many RSS feeds lately due to time constraints from writing a book, but I came across this post by Adam Gomaa, by way of some of the responses from Jonathan LaCour’s blog.

If there is one thing to learn from the endless, and pointless, comparisons of frameworks in Python, i.e, popularity contests, it is that the most important things for a web framework’s popularity, in order, are:

1. Documentation
2. Marketing

In a popularity contest, the “best” framework, is going to have the best documentation, and the best marketing, like tons of screencasts, etc. Every other discussion is an exercise in futility. If you want to be home coming queen, people have to know who you are, or they won’t vote for you. One of the reasons why Django is considered the most “popular” or “best” framework to many people, is that the Django people did an incredible job of documentation, and marketing.

While Adam brings up some very valid points in his criticisms of Turbogears and Pylons, he ultimately misses out on the real problem and the real solution. While I also like to find reasons to stretch my intellectual muscles and use legendary books to compare and contrast ideas against real world problems, it doesn’t work for this comparison.

The real problems doesn’t involve fancy computer science terminology, or “Conceptual Integrity”, it involves something much more mundane…..documentation. It is a known fact, that smart people, often the very smartest, don’t like to document their work, as they are too busy “inventing”, and being mad scientists. I am quite certain, that core Turbogears and Pylons developers, and power users, like Bob Ippolito, know how to do things that are unbelievable, but creating bulletproof documentation is tough work. Documentation is a job, and that is why millions of technical books are published making millions of dollars per year. If you consider documentation to be full time job, then people need to get paid to do it properly.

The reason why everyone doesn’t have killer documentation and marketing, is that it is perhaps, the most difficult part of being, “in the framework business”. Documentation is grunt work, plain and simple. To quote Bruce Lee, “I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.” In a similar sense, I fear not the framework that has 10,000 features, but the framework that documents one feature 10,000 times!

Of course, ultimately, a popularity contest is mostly meaningless for people who actually develop in a specific framework and make it their job, unless they are consultants for hire who only build a “insert framework here” websites. As anyone who has attended a 10 year high school reunion can attest, being “popular” isn’t always best, but due to human nature they will always be confused. Look at all of the “quiet”, corporations, all over the world, that just do Plone development, they are so busy making money, the framework issue must seem like background noise.

Also, it is interesting, but not surprising, that Zope 3, ZODB, Plone 3, and Grok are virtually non-existent in discussions about Python Web Development, in some circles. I wonder how anyone can feel comfortable getting into a massive diatribe on Python Web Development and completely dismiss any discussion about Zope related technology. Listen to the people who write the PEPS. If someone has gotten more than one PEP approved then their opinion is worth 1 million times the regular joe. Last time I checked there are only a handful of people who can say that, Phillip Eby, being one of them. He seems to think Zope is relevant, why don’t you?

Another troubling thing I see when Python Web Frameworks are brought up, is a lot of discussion about people writing wiki’s and blogs, etc, in the hot new framework. I suppose I should probably bring up that fact that Plone 3 is one of the most kick*** Content Management Systems on planet earth, see PyATL. If you want a blog, install Plone, if you want a wiki, install MoinMoin, or Plone. Unless you want to learn about writing a CMS or Wiki, you are essentially, self-gratifying yourself, to put it diplomatically, by recoding a solution to a problem that has already been solved.

Finally, I think WSGI is going to change everything, because it is about reusing components and products. Some of the current web framework comparisons are just silly. It would great to live in a world in which the real issues are discussed like, “My documentation kicked your documentation’s *ss!”. He who wins the documentation and marketing, wins the hearts and minds! To steal from a famous political campaign, “It is the documentation stupid!”.

Links:
Noah’s Personal Blog
OS X Automation
PyAtl

Jon Allen announced that http://perldoc.perl.org/ has been updated with Perl 5.10 documentation. This is great news as the look of the rendered PODs with syntax highlighting and many other nice effects makes me feel good.

The Peer to Patent project (which I’ve reported on before) just pointed me to a particularly broad patent that could encumber user interfaces on the web and desktops for years to come.

The idea in this patent, submitted by Yahoo!, is a clever little idea: if someone is starting to drag an icon or mail message somewhere, why not bring the mountain to Mohammed, so to speak? If she is dragging a photo, for instance, the browser or operating system can guess that she wants to open it with PhotoShop or the Gimp, and present that choice right next to the icon for the photo.

I’ve been wondering that lately. I’ve been using Subversion for … well … what seems to be a lot of years now. Looking back at the dates for Subversion’s history and coinciding them with events that were happening in my life, I’m guessing that I started using Subversion no later than the end of 2003. (So, maybe that’s not “a lot of years now”…) And I was using CVS from about 2001 until I started with Subversion. During the majority of that time, I have been either the sole commiter of the code base that I was working on, or one of very few people working on the same code.

Recently, I started a job where I’ll likely be working more mingled in with other developers on the same code at the same time. Everything is set up using Subversion. Before starting this new job, though, I began looking into distributed source control, which is the cool new kid on the block. I’ve created some personal projects using Bazaar and have glanced at Darcs, Mercurial, and Git. I like Bazaar a lot. It can be a little sluggish at times (like pushing, pulling, and merging), but not unbearably so - and I expect that it’ll get better. I keep running through my mind how moving to a distributed model would impact the work flow with my co-workers and I’m not totally convinced that distributed is always the way to go.

I just finished reading this piece of a conversation with Linus Torvalds regarding Git, and I remain unconvinced that going distributed would be the best thing for us. And I’m guessing that maybe most small teams of “closed” development probably don’t need a distributed source control system, either. It seems that the problems that have spawned this new model of source control is more of a problem for open source development, particularly of larger projects, and less of a problem for smaller proprietary development. For example, it’s really important for Linus that Linux kernel developers (or anyone, really) be able at any moment to create a new branch. It’s important for Linux that people be able to experiment with new kooky ideas and maybe come up with a cool new feature to go into the kernel. It’s also important to Linus that people be able to do so in anonymity. I can see how this would be important for a project that has potentially tens of thousands of developers interested in experimenting with the code and are doing so on their own free time. I think this is less important when a small team is being paid to work on a code base. Typically, you don’t have the leisure time to perform experiments. If you do need to work on an experimental feature set, it’s not a problem for a repository admin at work to create a branch for you. And anonymity isn’t typically necessary at work. At least, not anywhere I’ve ever worked.

I can see how having an “off-line” repository could be helpful. But from what I’ve heard, svk should address a lot of those issues. And most of these distributed systems are reputed to handle merging between branches better, which would be nice. I love Bazaar and would love to use it at work, but I’m just not convinced that it buys us enough benefits to switch from Subversion. Does anyone have convincing reasons that a small, closed source development team should consider switching to a distributed tool?

Mike Shaver has a deconstruction of false statements from Adobe about Flex’s openness. In particular:

[Adobe evangelist James Ward] has the nerve to call them “the community” and indicate that their work is a remedy for Adobe simply not being willing to remove the field-of-use restrictions on their existing documentation.

… and:

… they don’t want people to think too hard about the fact that writing to Flash is committing yourself to proprietary platform…

Maybe Monopolight will save the free software/free data/free community communities. (Why Monopolight? There’s a single vendor for licensed Silverlight codecs, Moonlight doesn’t provide the Silverlight codecs, and they’re x86 binary blobs only. Gee. Thanks.)

There’s a lot of speculation at any given time about what the future of the web is. I don’t know what it is, but I think I know what it’s not. Facebook.

Yeah, yeah, I can hearing the booing from here. Easy now, fan boys.

The truth is that the web rides waves of innovation and everyone on of those waves has a trough between it and the next wave. Facebook is currently experiencing the slide down the backside of the wave and is headed into the trough. Why?

Before you dash of a nasty reply, hear me out. There’s an excellent lesson that all companies can learn from this.

By now I suspect that many of you have already heard of Perl On Rails, an internal BBC project. Naturally, this made Slashdot, Reddit and Digg, amongst many other sites and one chap put up a very scathing post entitled Why the BBC Fails at the Internet. Most of the comments you’ll read about the “Perl on Rails” project are pretty far off the mark, but the “Fails at the Internet” post was, despite the vitriol, probably the most spot-on analysis of the problem facing the BBC.

Admittedly, I’ve only worked for the BBC for a month, but I’ve already had several friends ask me for the “inside dope” about this project and I wouldn’t say much for two reasons. One, I had only heard about it second hand and was concerned about getting technical details wrong. Two, I live in the perpetual fear that anything I say in my blog will be held against me. I’m probably right on the first count, but I’m definitely wrong on the second.

Here is a pro-Python propaganda comic. If you’re in to Perl, don’t take the alt tag (by hovering over the image) personally - it’s an xkcd comic. You know, the same comic that depicted Stallman as a ninja? Enjoy! (I know I did.)

I spent yesterday at the XML 2007 conference in Boston. It’s smaller than last year’s conference, which is a shame because I liked the sessions I attended better than last year’s. The knowledge and skills of the attendees as well as the presenters seemed impressive. Here are a few musings that resulted.

Here in the US, it’s Thanksgiving, a day of eating lots of food, watching football, and sometimes, just sometimes, expressing gratitude and giving thanks for those things that make life wonderful.

Here are the things I’m grateful for in late 2007, in no particular order after the first.

Google Code

Google’s project hosting service has been a godsend. It’s changed the way I do open source projects. It has leapfrogged SourceForge for ease of maintenance, and the bug tracker trumps RT for CPAN that we’ve been using for so long. Add that to the integration with Google Groups which makes it trivial to create mailing lists, and it’s at the tops of my list for 2007. I can’t say enough good about it.

The readers of Perlbuzz

Eleven weeks ago, Skud and I started this little website called Perlbuzz as an alternative to the “more traditional outlets” for news in the Perl world. The response has been tremendous. We get 600 RSS readers every day, and have had over 10,000 unique visitors in that time. It makes me happy that our little venture is used and appreciated by the community.

Test::Harness 3.0

It’s been over a year in the making, but the new version of the crucial Test::Harness 3.0 means more flexibility for module authors, and lots of UI improvements for people who just want to run prove and make test.

Mark Dominus

MJD is so much a fixture in Perl it’s easy to forget that he’s there. For 2007, though, never mind all the things he’s done for Perl in the past, or the hours I’ve spent being enthralled in talks of his. His Universe Of Discourse blog is the single most intelligent blog out there, and sometimes it just happens to be about Perl.

Andy Armstrong

Was Andy Armstrong always around, or did I just not notice? His time and dedication spent on climbing on board with Ovid and Schwern and the rest of the Test::Harness 3.0 crew has been invaluable in getting it out. Plus, he’s a really swell guy anyway.

Dave Hoover

When I finally despaired of the amount of time and frustration it took to organize content for Chicago.pm’s Wheaton meetings, Dave Hoover stepped up and volunteered to take it over. I’m thankful, but not as much as I hope the other Chicago.pm folks are.

Perl::Critic

I’m all about having the machine keep an eye out for the stupid things we do, and the goodness of Perl::Critic is always impressive. You won’t like everything Perl::Critic says about your code, but that’s OK. It’s an entire framework for enforcing good Perl coding practices.

The Perl Community in general

The Perl community is populated by some tremendous folks. Some names are more known than others, but these people help make daily Perl life better for me. In no particular order, I want to single out Pete Krawczyk, Kent Cowgill, Elliot Shank, Liz Cortell, Jason Crome, Yaakov Sloman, Michael Schwern, Andy Armstrong, Ricardo Signes, Julian Cash, Jim Thomason, chromatic, Chris Dolan, Adam Kennedy, Josh McAdams and of course Kirrily Robert. If you think you should be on this list, you’re probably right, and I just forgot.

My wife, Amy Lester

Because even if she doesn’t understand this part of my life, she at least understands its importance to me.

I’d love to hear back from any readers about what they’re thankful for. I’m thinking about having a regular “Love Letters to Perl” column on Perlbuzz where people write about what they love in Perl.

Here’s a provocative thought from Mark-Jason Dominus on a rainy Friday afternoon:

Programming cannot be run on the convoy system, with the program code written to address the most ignorant, uneducated programmer. I think you have to assume that the next maintenance programmer will be competent…. If an incompetent programmer has trouble understanding your code, that is not your fault; it is their fault for being incompetent. You do not have to take special steps to make your code understandable even by incompetents, and you certainly should not do so at the expense of making it harder for competent programmers to read and understand, no, not to the tiniest degree.

— Mark-Jason Dominus, Creeping featurism and the ratchet effect

Please note that I don’t believe that any one particular programming language can solve this problem; see Does Your Programming Language Have Magic Powers? This is a people problem.

Last week, the Perl Foundation announced that the Mozilla Foundation has awarded a development grant to Patrick Michaud to work on the Perl 6 on Parrot compiler. Thank you, Mozilla Foundation (especially Zak Greant and Frank Hecker.)

This grant is particularly important because it’s the largest grant of money to any Perl 6 hacker in several years. There have been a couple of other grants; the NLNet foundation made a very generous grant to Parrot a couple of years ago, which the Perl Foundation is doling out as Parrot hackers reach specific milestones. A well-known Perl-friendly development and consulting company also sponsored several microgrants for Perl 6, Pugs, and Parrot.

If you’ve read between the lines of several of my recent postings related to development, motivation, sponsorship, and scheduling (for example, Squeezing One Year of Work into Eight) you’ve probably recognized that I believe strongly that external resource constraints have slowed Perl 6 development dramatically. (Alternate phrasing: the progress of various Perl 6-related development efforts is impressive considering how few resources any of the projects actually have.)

I have confidence in the work of Patrick and a strong belief that the Mozilla Foundation’s generous grant will enable him to devote more attention to the Parrot compiler tools and Perl 6 on Parrot — and even more important, to recruit and encourage more developers, testers, documenters, and participants to participate. I do believe that we can develop and are developing high-quality, useful, inventive software for comparatively few resources, and I’m proud of what we’ve all accomplished.

Still, as Allison Randal’s Impact of Parrot Grant report to NLNet illustrates, even modest sponsorship can help us work even more effectively.

Think of the community-driven software projects you rely on and please consider how individual and company sponsorships may help us build and maintain a powerful, useful, and above all free ecosystem of high quality software.

I went down to the Cambridge, Massachusetts lab of One Laptop Per Child today to find out what they’re doing with mesh networks. This was a particularly appropriate day for a blog on OLPC, because today they’re launching a fifteen-day-long purchasing opportunity called Give One Get One. You pay them for two of their brightly colored, impressively lightweight computers, and one goes to a child in a developing nation, while the other goes to you.

But the whole point of this blog is that a One Laptop Per Child system has limited value on its own. Its most innovative and powerful features lie in its participation in a mesh network with other laptops. So get your neighbors and workmates to buy them too!

We’re used to think of system-enforced access policies as crude and coarse-grained, such as the setuid permission bit that lets a user execute a program as the file’s owner. Fine-grained access has to be enforced by individual applications, a laborious coding process that is weakened by not being able to take advantage of underlying operation system security. PolicyKit, developed by Red Hat and included in Fedora 8, ameliorates this unsatisfactory situation.

Red Hat developer David Zeuthen describes through examples the types of access problems solved by PolicyKit: “it’s fine to mount removable media; it’s not fine to mount fixed media; it’s not fine to change the timezone.” These are operating system capabilities that can be enforced by such operating system components as the HAL or the filesystem. But any application can use PolicyKit API to enforce any kind of access it chooses to, and it gets the backing of the operating system. Zeuthen compares PolicyKit to Authorization Services on Mac OS X and Group Policy in Windows.

UPDATE: If you want see an “application”, Plone 3.0, run under WSGI with a “framework”, Grok, you can check out this demo on the Repoze Site. This is one of the more exciting things I have seen all year. So what is this called, a framework, a product, WSGI…you tell me

Our local Atlanta, Python User’s Group, PyAtl, is collaborating with the local, Plone User’s Group, to build a Plone 3.0 site. I have been attending the Atlanta Plone meetings recently at ifPeople, and we had a mini-sprint last weekend where we got a demo site running behind a, newish, forward proxy server called Varnish.

On Wed, I attended an Atlanta Plone meeting, where Brandon Rhodes, gave a presentation on Grok. Brandon has kindly agreed to post his presentation on his blog, which he did in KeyJnote, which is written in Python, of course.

In Brandon’s talk, he mentioned that Grok, which is built on top of Zope 3, is less of a framework and more of a component based architecture. In looking at the Grok website, I liked two things in particular:

1. Grok offers a lot of building blocks for your web application.
2. Grok is informed by a lot of hard-earned wisdom.

One of the reasons why web application developers should look at Grok, and Zope 3, in particular, is the second point. Failure over a long period of time, leads to experience. When people learn from their failures, it then turns into wisdom. The Python world has much to learn from Zope, and that is why I am especially interested in Grok, and Zope 3. Remember Zope Corporation was founded in 1995, a lifetime in software development.

All of this background information brings us to the meat of the discussion, which is Repoze, WSGI, and the eventual death of webframeworks as they exist in their current state. I for one, welcome our new WSGI, overlords. WSGI, and products like Repoze, will make webapplication frameworks much less important, and component based architectures, much more important.

One of the massive, unaddressed until WSGI, problems of standalone, web frameworks, is that, ultimately, they could never win in the long term. If a web framework has a frenzied pace of development, then the API, and documentation will never be accurate. Additionally, web frameworks, like Django, can lock you into a specific set of “tightly coupled” components, in which your ability to use other components, like SQLAlchemy, is not possible.

With WSGI, these issues become much like relevant, as something like Repoze, can use the WSGI spec, to do things like, “allow for systems that participate in a WSGI pipeline to make use of the existing two-phase commit transaction management provided by the ZODB transaction package.” WSGI is clearly the future for Python web development, and in hindsight, it even seems silly that such an emphasis has been placed on web frameworks as a one stop solution.

Clearly the better solution, is a loose set of components that work together in a unified fashion under one specification, much like how the standard library of Python itself works. I suppose, to further illustrate the point, can you imagine how irritating it would be if you used, say, the tarfile module, and that meant that you could not use the zip module? That would clearly be unpythonic, and this is why WSGI is the new buzzword. By using the WSGI spec, and optionally, something like Repoze, combined with deliverance,it is possible to run multiple web applications, running under one URL, all with the same “skinning”, or look and feel. This smells like victory, for Python web application developers.

Note update: I want to make it clear that I am not slamming any web framework, including Django, my point is that ANY web framework can run independently inside of something like Repoze, coupled with Deliverence, making even a full framework an actual component and in fact eliminating problems like one architecture is tied to a specific ORM, or templating engine etc.. My apologizes if this point was not clear in the original post. What this does is take the emphasis, and pressure, off of a site being one specific component architecture, or framework. A site won’t be a “Django site”, or a “Turbogears” site, it will be a “website”.

(Thanks to Derek Richardson, for being such a strong advocate of Repoze, WSGI, and deliverence, as much of this post is based on things I have heard for the first time from him. On that note, it is looking more and more likely that we have the authors of Repoze speak at PyAtl in December, 2007. If you are interesting in helping to sponsor this event, please contact me or Derek Richardson.)

[1] Update: Update May 17th, 2008: I would refer anyone who is angry about my opinion to first read this essay by Paul Graham: How To Disagree. Note, I privately attempted to contact Jacob Kaplan Moss to prevent him from leaving the drunk post, but I received no response.

[2] Update: A particularly juvenile response to this post seems oddly relevant to this article from the New York Times. I don’t see any distinction between his behavior, and the protagonist in the Times article, “Weev”. I attempted to contact the poster privately via email to resolve the issues, and was confronted with a similarly juvenile response. I have the correspondence available for interested parties.

Even though I do my best to avoid using Windows, at all costs, unless it is a function of work I need to perform, I am bit impressed with how far IronPython is getting along. Even with recent news stories about ugly blue screens, and cries from a crotchety old man, about finally considering Linux or Mac, Microsoft is doing something right.

I recently starting reading Steven Holden’s blog, and in this post, he mentions some of the exciting developments in IronPython as discussed by Michael Foord. I have played around some with IronPython, and watched a presentation on IronPython at our local Python User’s group on it, recently. I wonder though, if Apple is falling a bit behind Microsoft in its support for Python as a first class development option?

I went to an iPhone Development talk today, which covered the webkit side of iPhone development, but I wondered if Apple would be forward thinking enough to beat Microsoft to the dynamic language battle, and do the iPhone SDK right. Doing it right, would be to think of the API in terms of Python and Objective C.

For example, are they going to allow pure Python code to write applications using the iPhone SDK? Apple has soundly, and routinely, beat Microsoft in the Operating System war, since OS X, but what about after that? What if the future is led by dynamic languages, and Microsoft is way ahead with IronPython?

I am not knocking the incredible hard work, and effort, that has been put into PyObjc2, and the further integration of XCode with Python. This is wonderful, and I cannot wait to start digging into PyObjc2, but does the PyObjc team have the same resources and support from Apple, that Iron Python has from Microsoft? If not, then maybe it should be a higher priority for Apple to look to the future and put more research, money, and energy into their support for dynamic languages, as this may be the next battleground for hearts and minds.

Personally, I would love to hear about Apple hiring a few Python people like Google did, and to start developing the Cocoa API with Python directly in mind, not as an afterthought. I cannot even imagine how many new, yet very experienced, developers this would bring to Cocoa, it would be mind boggling. While Apple is at it, they could abandon Applescript like they have done with carbon. If you look at GNU/Linux, it has an interesting near equivalent to Applescript Script Recorder in the form of Dogtail, where you can record and write UI events in Pure Python.

I personally think dynamic languages are an ever-growing part of the future, and I hope Apple continues their meteoric rise by taking a lead in adopting them.

One hoary truth of computing technology is that most of the pressing problems today have solutions discovered or developed, at least in part, twenty years ago. (This nicely avoids the patent problem.)

Google’s announcement of the OpenSocial API brought up yet again the persistent problem of walled gardens on the Internet, as myriad social networking sites spring up, offer to invite all of your friends if you divulge your address books, and then slowly wither as you realize that visiting half-a-dozen sites every day to read messages from your fragmented social groups is busy work.

Wouldn’t it be nice if all of these disparate messaging systems could interoperate?

Over the weekend I encountered a dusty old RFC written in 1982 that might solve this persnickety interoperability problem. Jon Postel’s Social Messaging Transport Protocol describes a system that relies on the combination of your unique identifier (username) on a social networking site with a unique identifier (domain name) for such site to produce an Internet-wide addressible identifier uniquely identifying, well, you. Given this unique identifier, any conformant messaging system can use this Messaging protocol to send you, well, a message.

Lest you fear such a system (a quarter century old!) is inextensible for the 21st century uses of zombie bites and pokes, the system builds on the tried-and-true HTTP style header/body distinction, where headers are simple key/value pairs that get ignored if unknown but offer plenty of ways to encode Gravatar or OpenID information, if necessary. (Admittedly, an extension of SMTP in RFC 2821 is only six years old, but provides further useful updates.)

The Social Messaging Transport Protocol enables the lovely store-and-forward behavior we’ve all come to expect from social networking sites, where you can send a friend a message and he or she does not have to be online to receive it. (Unfortunately, there is a flaw in the system; sending messages does not work if you are on an airplane. I can’t find a technical reason why it shouldn’t work, but I probably overlooked a MUST NOT in the RFC. One potential solution is for social networking sites to offer miniature versions of their sites that users could download and use directly while offline, but is that really easier than never traveling anywhere you don’t have broadband access?)

There are other tremendous benefits from adopting this ancient protocol for interoperability between social networking sites, but there is one drawback (besides the fact that you can never fly again): there’s a slight potential that you may receive unsolicited messages from people you don’t know. Fortunately, unsolicited commercial messages are almost unknown on all popular social networking sites thanks to the diligence and exemplary customer service shown by their operators.

A final benefit is worth noting. The Social Messaging Transport Protocol has a companion protocol revised most recently in RFC 3501, the Instant Messages, Advertisements, and Pokes protocol, which allows you again to use the unique identifier granted to you by a site with the site’s own domain name to login, view, and manage all messages held for you at the site. An IMAP widget could be embedded in any web page to allow you to check your messages from any web site.

(Imagine if we could solve the problem of local storage of data from Internet applications!)

I’ve already started writing a Javascript implementation of the protocol, but the Digg It! widget conflicts with the pulsing yellow update of my rounded corners, so it may be a week or two before I can release something usable.

Ian Bicking discusses Mozilla Prism in Prism:

Here’s a general rule I have: I don’t accept anything made by people who hate the web. If you hate the web and you want to improve the web, I don’t want anything to do with you… To me Silverlight and AIR reek of a distaste for the web.

While I think that HTTP and REST are the most important parts of web applications, not the web browser (and certainly not HTML and JavaScript), credible (that is, both free-as-in-freedom and not web hating) alternatives to AIR, Silverlight, and Flash are very welcome.

Wikipedia estimates that Facebook has 47 million active users, as of this month. If, as reported widely, its founders believe it’s worth $15 billion, each active user account is worth around $319.15. That’s old valuation math, though–bog-standard cost of acquisition. Facebook is something new. It’s a platform for hosting applications to take advantage of 47 million users who like to poke each other electronically.

As far as I can tell, the second best reason to use Facebook is the Facebook Zombie Application, which lets you bite 20 of your friends every day to turn them into zombies too. (The best reason, according to my contacts in institutions of higher learning, is to watch the friend list of your next potential significant other to see if your rival gets more attention than you do.)

If every active Facebook account signed up for the Zombie app, the value of the site becomes approximately $15.98 per daily zombie bite. This introduces new funding models for publicly traded platform-only companies in a new economy; perhaps class A shares could eschew zombie bites in favor of voting privileges, class B shares could transition from zombie bites to voting privileges over a vesting period, and class C shares could get additional zombie bites.

I wonder if they’ll have to extend EBITDA to include decapitation, though….

Bom dia!

It certainly has been a while; in fact, the last time found me in Colorado, toughing out the ridiculously beyond-cold weather of the Rockies, but managing to have a great time, and laugh about it, nonetheless. Now, my better half and I are in her hometown of Uberlandia, Minas Gerais, Brazil. I’ll stop myself from turning this into a piece on all the goodness one can experience in a Brazilian minute, but let me just say this: You owe it to your taste buds to indulge in the ambrosial bliss that açaí is. Be liberal and have two bowls. It’s Dessert 2.0. I hope they serve it in the sweet by-and-by.

Okay, so on to the topic at hand - Windows Vista’s BitLocker, which encrypts all data on the system volume. At a glance, this might not seem like the logical place to discuss matters Vista-centric, but what follows is applicable to any instance where cryptography is required. As much as it is about BitLocker, it’s about much more. First, before continuing, you might want to take a look at the original draft of an article I wrote, entitled, “On Shifting ‘Windows’ and ‘Security’ from Less Antonymous to More Synonymous.” An adaptation, “BitLocker and the Complexities of Trust,” appears in the October 2007 issue of Microsoft TechNet Magazine.

Phil Zimmermann was kind of enough to provide some commentary for the article, of which I assume you’ve read at this point. Perhaps the most resounding proverb is, “Design as if making a mistake will cost someone’s life.” This brings me to my questions for you guys and gals. What do you expect out of cryptography? Are there any general goals you think it should achieve? What is “good cryptography” to you? How do you feel about open-source versus closed-source, in regards to cryptographic implementations? Are there any examples, that stand out to you, of where cryptography is being done the right way?

What I’ve come to find, over and over again, is a perpetual state of failure within cryptographic implementation; that is, when cryptography fails in practice, it’s almost never because of the cryptography itself, but, rather, its implementation. I attribute much of this to the fact that most developers responsible for implementing cryptography aren’t, well, cryptographers. I don’t expect them to be, either. However, many of these developers haven’t the knowledge to properly define the right threat model, let alone identify which cryptographic primitives they need in order to address that threat model. Mistakes ensue. To many of you, I’m sure this isn’t news.

I’m proactively working on ways to educate developers so they can avoid the subtle mistakes that leave huge marks. There’s a long road ahead, but the dividends are grand. Besides, cryptography is usually the strongest link in any security system. Why all the lax implementations? Shouldn’t we expect the same strictness in implementation that’s put into design?

Cryptography has, arguably, the best track record out of all the other aspects of security; it’s time to do a better job at reflecting this in practice. Something’s wrong if cryptography can’t reach its fruition in practice. Developers are in dire need of something that cryptographers have, so we need to bridge what is still an uncomfortably large gap between the two.

With our systems like loose slacks, we have a tight belt, yet we can’t seem to put it on right.

I’m all ears. Well, eyes. I’m all eyes.

(Names, companies, and identities have been changed to protect the innocent.)

I recently got sucked in the dark, black vortex of linkedin, and it got me thinking about my life. Linkedin is a very interesting social networking website because, quite literally, every person I have worked with in the last 10 years of my professional life is on it.

It is quite a stroll down memory lane, as I add one person, then remember, wow, I forgot about that person, hmm, I wonder what they are up to, lets add them too…. All in all, it is fun experience that is almost like a high school reunion, but for an adult professional life.

One of the items that caught my eye recently was, a title a, link of a link, had, roughly:

Catalyst Team Organizer -
Create innovation at Joe Blow Company.
Encourage interaction between departments.
Demonstrate innovation case studies to Executive Team for approval.

I would suggest that by definition, Corporate Mandated Innovation, defines that a Corporation will never be innovative, and is currently not innovative. This reminds me of a scene straight out of Catch-22. Here is the famous quote from the book:

There was only one catch and that was Catch-22, which specified that a concern for one’s safety in the face of dangers that were real and immediate was the process of a rational mind. Orr was crazy and could be grounded. All he had to do was ask; and as soon as he did, he would no longer be crazy and would have to fly more missions. Orr would be crazy to fly more missions and sane if he didn’t, but if he was sane he had to fly them. If he flew them he was crazy and didn’t have to; but if he didn’t want to he was sane and had to. Yossarian was moved very deeply by the absolute simplicity of this clause of Catch-22 and let out a respectful whistle.
“That’s some catch, that Catch-22,” [Yossarian] observed.
“It’s the best there is,” Doc Daneeka agreed.

So here is the Corporate version of Catch-22. If you are innovative, no one needs to tell you to be innovative, you just are. If you are not innovative, you cannot say you are working to be innovative, because by definition, the very act of deciding to be innovative is not innovative. Ahh, this is also some Catch-22, maybe not the best there is, but close….

When I started using Unix seriously in 1998, there weren’t a lot of options for getting on the web. I’d happily used Opera on Windows at home, after Netscape’s rather disappointing version 4, but Netscape Navigator (and not the whole suite) was clearly the best option when I switched to GNU/Linux full-time at home later that year.

I followed the Mozilla project with interest and finally switched away from NN4 to one of the Mozilla milestones around 0.9.1. This was an improvement, and not only because Mozilla tended to be higher quality but because it supported more sites more effectively.

Firefox didn’t impress me when it first arrived. Removing useful features altogether still strikes me as deeply silly. I remember thinking that Firefox’s vaunted slimness and lower footprint would go away when I had to install a dozen extensions to get back the features I used every day.

I haven’t thought that for years. I switched to Firefox around its 1.0 release and consider it a fine piece of software. Sure, there are problems, but I remember using Netscape 2 betas. Web browsers have come a long way in reliability and utility since then.

Firefox may be my most heavily used application, if you don’t count X.org or the command line. It’s served me well, and I’ve recommended it highly to countless others. Thank you to all of the contributors to Firefox and Mozilla’s projects. Even ten years ago, I had no idea how valuable this work would be.

I have been meaning for a while to write a post about how mentors have helped me to achieve my goals and dreams, but I have been so busy recently, that I put it off. This week I came across the video of the last lecture by Carnegie Mellon Professor Randy Pausch, who is dying from pancreatic cancer, and I was inspired enough that I figured I should write this post and tie it into my response to the video.

First, I would highly recommend that everyone and anyone watch this video. It is truly an inspirational and powerful video. One of the things he mentions in the video is, “How to Get People to Help You”. He mentions five points:

1. You can’t get there alone, and I believe in Karma
2. Tell the truth
3. Be earnest
4. Apologize when you screw up
5. Focus on others, not yourself

He also mentions, “Brick walls let us show our dedication”.

I won’t focus on everything he says in final speech, but I will get straight into how this relates to finding mentors in life. My first great mentor was Dr. Bogen, who I met at Caltech. He took in an interest in me when I was starting to form into a man, and, without a doubt, changed my life forever.

I remember doubting for quite a while if I was really intelligent or not, and then thinking that here was a guy that was in school until he was 37, a neurosurgeon, and a professor at Caltech, who found me interesting enough to talk with me every Friday from 8PM until 2 or 3 in the morning for a few years. Maybe I was smart? We talked about Philosophy, Math, Psychology, Consciousness, Religion, the Stock Market, Computers, Artificial Intelligence, and Bonsai Gardening, etc.

These talks and his wisdom allowed me to dream beyond what I thought was possible. He also introduced me to other powerful mentor/guru types. He also spurred a life long interest in the brain and psychology. Even though he passed away a couple of years ago, I still remember his words and advice almost like when Luke Skywalker hears the voice of Obi-Wan Kenobi. “Noah use your brain, don’t turn to the dark side…..”. I am also very glad that I got the chance to tell him that I loved him before he died and that I got to say goodbye when he was in the hospital. It gives me great comfort that he knew that before he died. I still remember that day very clearly, I grabbed his bald head, like I now grab my son’s head now, and let him know how much he meant to me.

I have met other different types of mentors, as well, in my journey through life so far. Another interesting person who I met at Caltech was Titus Brown. He has been a much different type of a mentor, but I have been asking him questions about computers and programming since 2000. He was the voice that kept gently whispering in my email…”so why don’t you try Python?” I can honestly say that if it wasn’t for Titus, I would never have touched Python or considered it. God knows what language I would be using at this point.

In March this year, at Pycon, I had dinner with Titus, who I hadn’t physically seen in a while and I met Shannon Behrens. We ate some barbeque and started talking about Vim, LDAP, Python, etc. Before I knew it, Shannon turned into a mentor. His knowledge of Python is so vast, and his attitude is so humble, that he has made a huge impact on my knowledge of Python. As I have mentioned in many previous posts, Shannon is my hero. He is one of those rare people who will quite literally tell you anything he knows, and will not judge you for asking a stupid question, for this, he is my hero.

I have also met some incredible mentors in the various jobs I have had. What is interesting, is that each mentor has a different method to access their secrets, and different ways of teaching those secrets. In some cases I have even had “unwilling”, or “hostile” mentors. A “hostile” mentor is someone who is scared to share what they know with you because they are insecure. A “hostile” mentor is a bit like someone who has turned to the dark side of the force. They feel that they can only succeed by working really, really hard and not sharing information with people or giving them false information.

It is still possible to learn from these dark lords, by just seeing through the insecurity and focusing on what you can observe to be truth. Yes, a dark lord can still be a mentor.

I met a really impressive, good, mentor at Disney named Greg Neagle who taught me many cool OS X tricks. I met another extremely powerful, yet incredibly humble mentor named J.F. Panniset, who is now in charge of engineering at a A52, while I was working at Imageworks. JF was scary smart, in particular, because he can write code, is an expert Video Engineer, knows Film production and Post Production inside and out, is an expert sysadmin, AND is about as approachable and humble as they get. All I can say about A52 is that they lucked out big time getting JF. He is a one in a million catch for a company. I was very upset to have had to leave Imageworks, but my wife and I moved out to Atlanta to start a family.

When I got to Atlanta, I was lucky enough to take a couple of classes at the Big Nerd Ranch and I become friends with Aaron Hillegass, the owner. He has been a great help and a very kind mentor as well. I am in a book club with him and we are currently reading Information Theory, Inference, and Learning Algorithms.

So, what does all this mean and why am I bringing it up? There are mentors and gurus behind every bush. It is very easy to tell a powerful mentor. The more powerful the mentor, the more freely they give information, help and guidance. I am now under the belief that there is a direct correlation with a person’s mental power and their ability to mentor. The greater a mentor’s power and wisdom, the greater they can give. After all how much effort does it take for a flood light to light up a dark backyard, compared to flashlight? These great wizards can spare the beams.

If you want to get better at something, you need to find mentors and find out how to get them to take an interest in you. In some cases it make take quite a while for a good mentor to think your serious. I would refer back to the quote by Randy, “Brick walls let us show our dedication”. If you really want to learn from a good mentor, then if you put in the work they will eventually help you, unless they are a dark lord, but those guys are easy to catch. They say things like, “Old age and treachery will beat youth and skill, every time.”

The soon to be released “Spotlight on FOSS”, video podcast Pilot episode, features Mark Shuttleworth. (We should have a date announced for release very soon). He gives a tremendously, moving talk that is on par with the Randy Pausch talk. One of the goals that Jeremy and I are trying to achieve by doing this podcast series, is to take these mentors of the Free and Open Source Software world and to expose their thoughts, ideas and passions to a larger audience. I hope people take the time to watch our first show, and I am very excited that it turned out to be this powerful.

In closing, I am also on the lookout for a new mentor to learn new things. I am particularly interested in learning more about artificial intelligence in the next few years. If you are a guru, and are interested in taking an interest in me, I would love to hear from you. I am currently looking for another mentor like Dr. Bogen, although that may never happen again, as he was truly one of a kind, or one in a billion. Also, if you have Guru/Mentor stories, I would love to hear about them. Finally, all of the mentors mentioned here are mine, all mine, so get your hands off them!

I first used Subversion about five years ago while writing my second book. It was an early milestone, but it was easier to compile, install, and configure than CVS was, and it was much more powerful than RCS. Since then, I’ve used Subversion to host every book I’ve written or edited. (See The Making of BSD Hacks.)

Though distributed version control systems are gaining in popularity (and though I use SVK atop Subversion), Subversion is still tremendously useful. Try being a F/OSS developer without knowing how to work with a project in Subversion, for example.

The preponderance of support with free hosting providers for F/OSS projects, as well as the quality of documentation and implementation, mean that I spend a lot of time working with Subversion. I’ve rarely had any trouble (even as far back as milestones 19 and 20). My work would have been much more difficult without this project, so thanks to everyone who’s contributed in any way.

In my career, I’ve been paid to program at ten different companies. Of those companies, only two of them have taken computer related security very seriously and three have had serious security breaches. There is no overlap between these two groups.

Of the three security breaches, two of them were known security issues that had been brought to the attention of management but management chose to ignore them. One of these caused serious financial harm¹. Due to the nature of the problem and management’s reluctance to discuss it, we couldn’t determine the exact amount of damage, but between known financial losses and the cost of responding to the incident, I would conservatively estimate that we lost at least $100,000 and possibly up to a quarter million. Had we fixed this problem before it occurred, it only would have taken two or three days of developer time. Given the relatively small cost of fixing the problem, why didn’t it get fixed?

Rael Dornfest and Brian Aker invited me to an RSS meeting way back in 2001, so I’ve known about the value of syndication for a long time (at least before it was super popular). Jon Udell’s Practical Internet Groupware made me realize the value of URIs as identifiers as well as the utility of alternate views of information.

I never found a newsreader that I liked, though. I tried, really tried to get into NetNewsWire during my brief flirtation with Mac OS X. I thought Sage for Firefox was useful… but it never really stuck with me.

I despaired of keeping up on the news of the day with anything but a careful list of bookmarks in Firefox until I found Akregator. Because I already use KMail, the interface was sufficiently familiar (which bothered me about other feed readers), and it’s just configurable enough that I can update feeds once a day, then disconnect from the network (though objects embedded in feed entries need an active Internet connection).

The highest praise that I can give Akgregator is this: I never considered a feed reader worth using until I found it. Now it’s a primary application for my research and work. Thank you to all contributors to the project!

Even using the right tools, in the right way, a software project can still get into trouble. One of the most pernicious ways to fail is over-specify everything up front. As the “Lean Software Development” movement has documented, well-intentioned people often add risk to their projects when they make hard decisions too early - before any research to identify any supporting facts. The best practices are Adaptive Planning, and Just-in-Time Requirements.

Another way to fail is to allow these requirements to fall into your lap by themselves. This post explores why embracing these deceptively easy requirements still adds risk.

Scott Ambler’s The Discipline of Agile on DDJ deconstructs a myth surrounding agile development, namely the fiction that agile development or XP is for undisciplined cowboys. Perhaps my favorite part of the article is the checklist at the end which helps identify cowboy coders and actual agile teams:

• Take a test-driven approach to development.
• Work very closely with their stakeholders on a daily basis, and the stakeholders have active roles on the team.
• Produce working software on a regular basis and can show a clear track record of doing so throughout the project.
• Work in an open, collaborative, and self-organizing manner.

In Are Authors Technological Poseurs?, JRuby hacker Charles Nutter suggests that “Good authors do not have time to be good developers.”

There are plenty of counter-examples, but I think the number will end up in the range of several dozen, while the number of great developers in the world is easily hundreds. (Ohloh had over 60,000 F/OSS developers identified the last time I looked, so the top 10% could be 6000 people.)

The nature of the skills are different, though. Writing’s less unambiguous than code, where at least you have a chance of independent and repeatable verification through technical means. It’s not a skill you can practice much on your own into the dark hours, because the purpose of communication is all important (at least in technical writing).

If it takes a year to write a good technical book (and it usually takes at least a year to write a good technical book), how much brilliant code could you produce in that time? Maybe it really is the case that to master a subject, you have to work at it full time — and while you may be good at both, you can only master one. There’s just no time to do both.

Qtopia has just undergone a major new release, adding such features as the Safe Execution Environment (a kind of sandbox for native code), strong support for WiFi, and integration with the WebKit browser engine. It’s a lot to give away for free. Based on Trolltech’s Qt graphics libraries and supporting C++ development as well as a JVM, Qtopia is a graphical environment found in a huge number of devices ranging from the Motorola Razr V8 to automobile navigation systems and medical equipment–and of course, Trolltech’s own Greenphone.

I talked to CTO Benoit Schillings, who cited fairly stock reasons for going open source: development energy comes from a community of developers and users, and ultimately there’s more business in serving a thriving environment of new applications and features than in holding onto secrets.

He mentioned the ubiquitous address book as a mobile feature that has hardly changed in years and is shut off from innovation because it can’t be touched by application developers. When the whole platform is open, new blood flows to these atrophied parts of the system.

There appears to be a general culture in IT, partly, but not completely, because it is male dominated, that it is bad to say the wrong thing, reveal your weaknesses, or be an “idiot”. I have been an Engineer for about 10 years now, and I have felt it from the beginning. When I first started learning Unix/Linux at the beginning of my career, I felt stupid quite a bit, and was afraid to ask questions as there was a culture that scoffed at “stupid questions”. Why don’t you read the manual, “idiot”?

When I transitioned from working as a sysadmin to working as a Video Engineer in Film and Television, I felt the same way. By that point, I had done enough in my life that I wasn’t as deterred to ask stupid questions, but I do remember several times people telling me, you should know this, why are you asking this question?

As I transitioned from working strictly in Film as a Video Engineer/Systems Engineer, to working just as a software engineer, I have often felt the same way. Maybe I shouldn’t let someone know that I don’t know everything about Python or programming? What if people think I am an “idiot”?

In the movie, “The Edge”, there is a great quote that directly applies to any Engineer, “Most people lost in the wild die of shame. They didn’t do the one thing that could save their lives –thinking”. How many potential skills or dreams die because of our shame? How much quicker could people learn if they were able to act like an idiot at some new skill they are learning, and truly learn it the way a child learns.

Now that I have some perspective from working as a Systems Engineer, a Video Engineer, and a Software Engineer, I will tell you that in each industry, I have had someone tell me that the Engineers in my previous field were “idiots” because they didn’t do “X”. For example, when I was a Video Engineer I had a couple of people tell me, “…you see Systems Engineers aren’t real Engineers because they can’t read a line drawing.” When I was a Systems Engineer, I had people tell me that Software Engineer’s aren’t real Engineers because they don’t really understand how equipment works. Since I have been a Software Engineer I have had people tell me that Systems Engineers aren’t real Engineers because they can’t program.

This situation I have described is a classic case of the “Observer Bias”, of course. From Wikipedia, “Observer bias is error introduced into measurement when observers overemphasize behavior they expect to find and fail to notice behavior they do not expect.” When you’re an engineer in one field you notice that all the people like you are smart and know how to do what you do, but strangely everyone else is an “idiot”. This critical scientific fact is the exact reason why all engineers should do something where they feel like an “idiot”, as it gives them true perspective and allows them to grow.

My piece of motivational advice is to ignore your inner feeling of shame as an engineer. Attempt to do something new, express a controversial opinion, invent a new technique or technology, learn to program, or learn a new language or skill. Step out of your comfort zone and do some activity where you are perceived as an “idiot”. In this sense, it is really important to be an idiot. Being an idiot, means losing the sense of self-criticism that is often found in programmers, sysadmins and engineers and truly learning. Being an idiot is important!

Hear me roar. This interview has me answering why I began the Women in Technology series, my hopes and dreams for it, and how I’ve already benefited from doing this. Hope you enjoy it!
(Also, to the many who have contacted me about this series, thank you so very much.)

Software development is rarely easy. Even though some open source pragmatists suggest that Linus’s Law makes traditional schedules and planning obsolete, I remain skeptical. (I’ve worked on a few pieces of F/OSS in the past decade; I believe that I’ve earned a right to be skeptical.)

In the era of Web 2.0, it appears that Slashdot has “Jumped The Shark”. The question now, is when did this happen? I remember in the early 2000’s, Slashdot was THE geek website, but something has changed and it appears they have lost the magic. I decided, today, to take them off of my RSS Reader, as I find their stories trite, boring and dare I say, irrelevant to IT?

So the questions of the day are:

When was the exact day, and, what was the exact story, that caused Slashdot to jump the shark?
Who did you replace Slashdot with in your RSS reader?
What is your best bet to the cause of their demise?

UPDATED SPECIFIC REASONS FOR TAKING SLASHDOT OFF OF MY RSS FEED:

Why I think Slashdot has Jumped The Shark?

Please note, comments are now closed as I can’t respond to them all.

Due to an overwhelming response by a fanatical few, I have decided to post in detail, why I no longer find Slashdot interesting, or relevant, and why I have decided to take them off my RSS feed. This is just my opinion, and I am sure each and ever person that uses RSS at some point has made the same decision about another website. If you get angry easily you might not want to read the rest of this post:

1. I am 32, and I have outgrown any interest in the usual stories that appear on Slashdot. A specific example is this story, about a “Coup” attempt in an Apple Underground User Group. I have absolutely zero interesting in ever reading something like this for the rest of my life.

I felt violated reading something that stupid today, and I will admit it may be because of my age, and due to the fact that I now have a wife, a kid, and a life.

2. I never really participated in the community much, I only read stories, and as I mentioned the stories are getting very bad.

3. I am bored of the terms, “Troll”, “Trolling”, and “Dvork”, they make my skin crawl just like it would make my skin crawl to hear someone use the terms, “Your playa hatin”, or “Give me the bling, bling”, or “Far out man”. These terms are so commonly used on Slashdot that it is impossible to avoid them and the only possible alternative is to never read anything on Slashdot.

4. Slashdot played an interesting role in the early 2000’s, as it was a human funneled aggregator for news stories. As technology has progressed, a different model of story submission has started to thrive and it does not require a select group of humans to filter which stories are good and which stories are not good. I believe the model that Digg, Reddit, and DZone use are far superior to the method of story submission and approval that Slashdot uses.

I feel that the method that Slashdot uses is a dying art, and the very poor quality of stories suggests that the human element responsible for editorial content is either very young, or not very good.

5. RSS Readers have changed the way people read technology and other news, and it has caused people to stop having a “home page” anymore. People now have the ability to create their own custom filters and get their own news in anyway they see fit. The role of Slashdot as the only aggregator for IT news has ended. Given a choice, I would much rather use RSS than the editorial process at Slashdot to get news stories.

6. As evidenced by the responses today, it is quite impossible to have a discussion with some outspoken members of the Slashdot community. It would take a large amount of patience, time and energy that I, frankly, am not willing to part with, to discuss why I am “liar, hypocrite, troll, etc”. It is just not interesting to me, and even if it was I would probably need to quit my job just to respond full time to the complaints.

If you feel the urge to yell at me, please remember this is just my opinion and I am very sure other people feel very differently. I do not have the time to discuss the matter anymore due to work and personal obligations, but thank you to the people that did respond. I did learn one very important lesson today though, some opinions are probably best left unsaid, as the effort required to explain it is not worth the cost of expressing it.

In summary, I do feel Slashdot has “Jumped The Shark”, I won’t read Slashdot anymore, and I am sorry I brought it up.

-Noah

I have been reading the CEO of Sun, Jonathan Schwartz’s blog, lately and it has some great material. I just love the fact that many CEO’s blog nowadays, as some of it is quite interesting, and bold, material. I suppose I would even go so far to say, that if I read a CEO’s blog and it wasn’t good, I wouldn’t invest in their company, as I would have my doubts about the leaders intellect and authenticity. On the other hand, if I read a CEO’s blog like Jonathan’s, I would be very interested in the company.

One of the recent posts that Jonathan made was in response to a flame by Linus. In his response to accusations, by Linus, that Sun was being disingenuous about truly open sourcing its ZFS file system, he mentioned that not only was Sun going to open source everything, but that they were going to do it under GPL3.

I think this is a truly brilliant strategic move by Sun, as it raises the bar for GNU/Linux and Linus who is very adamant about his dislike for GPL3. I do see there being competition between OpenSolaris and GNU/Linux in the coming years, and I do think it is a good thing, as competition is what drives innovation. I also see an interesting dilemma for GNU/Linux as OpenSolaris will be able to use GNU/Linux code, but GNU/Linux won’t be to use OpenSolaris code. This gives OpenSolaris a huge competitive advantage and might give them a temporary head start in the head to head competition of their operating systems. Ultimately, it seems like GNU/Linux might have a very large problem on it’s hands if it stays with GPL2.

I am not making a value judgement on GPL2 vs GPL3, but I will say from a strategic standpoint it appears to be checkmate for Sun on this round. Sun has some incredible virtualization and storage technology and it if enters the open source arena, “just right”, it could capture a massive amount of market share with this strategy.

Complaints by people in the GNU/Linux camp can easily be rebutted as Sun can claim the “moral” high ground as they support what the FSF recommends. I am quite interested in what happens over the next few years. There are quite a few possibilities, some of them admittely far fetched, but interesting to discuss regardless:

* OpenSolaris comes into the FOSS market goes GPL3 and takes significant market share as it has the best GNU/Linux technology and Sun Technology. GNU/Linux tries to fight back and switches to GPL3, but the damage is enough that they become the second preferred FOSS Operating System behind OpenSolaris.

* Nothing happens. Things pretty much stay the same and OpenSolaris doesn’t really make much of a positive or negative impact.

* GNU/Linux and Linus seeing the “end game”, quickly switch to GPL3 and cancel the advantage Sun might have before it happens. It then crushes OpenSolaris as it takes all of its good technology and the huge market share it already has and makes OpenSolaris marginalized.

* GNU/Linux becomes the preferred choice for companies that don’t want to become tangled in GPL3 and OpenSolaris becomes the preferred choice for a commodity operating system that runs in a data center. This leads to a significant loss in market share for GNU/Linux.

* Other things….what am I missing?

Comments?

Recently on this blog, I wrote You Have the Right to Read Your Accuser. In this, I argued that any software with substantial risk to harm your life or liberty must be open source. I specifically mentioned some breathalyzer software that people were fighting to see the source code of. Though the Florida legislature and many regional prosecutors obviously feel that the breathalyzer company’s rights were more important the individual rights, the New Jersey Supreme Court ordered that source code to a popular breathalyzer be revealed. The results, if you’ll pardon the pun, are breathtaking.

Even though I wrote a very popular post about the Zen of Mac,
to show that I can be fair, I thought I would write about what I hate about Apple.

I don’t think Apple has the perfect Operating System or company even though I use OS X as my preferred desktop OS and I love it. They could still improve on things, so on that note these are things I hate about Apple.

1. Dump the silly DRM stuff COMPLETELY, not just for part of your library.

DRM just insults our intelligence. Apple is supposed to be ahead of the curve and creating a user experience that is better than any other desktop OS. “Authorizing” my music when I reinstall my OS is extremely obnoxious. Have some guts and say no to DRM period! No Mac users want DRM, so why are you providing a service we don’t want. Sounds like another OS we have all heard of…tread carefully!

2. Free and Open Source Software package management system doesn’t exit!

Your core OS is UNIX, yet you STILL can’t figure out how to integrate a decent package management system for FOSS? Huh, I don’t get it? Integrate Fink or Darwin Ports, or copy debian, but get with the program, it is embarrassing!

3. Regular Commercial Software Package Management doesn’t exist…i.e. the “uninstaller”?

Again, why is it so hard to uninstall or reinstall commerical software on OS X? There is at least one robust open source packagement tool Radmind, that does this. Are you telling me that figuring out a regular package management system is that hard? Often you just need to drop a bundle inside of your Applications folder or delete it from your Applications folder, but many 3rd party applications leave a trail of garbage. Make them conform to a package management system so we can get rid of their junk!

4. Locking the iPhone.

Unless this was a very shrewd marketing campaign for the iPhone, what did you think was going to happen when you released the iPhone without an SDK and locked it to ATT? Lets have less “locking” and rules with things we pay tons of money for. Just release the friggen SDK already, even Microsoft has an SDK for their phone.

I don’t want to hear all of the excuses either, like Safari is an SDK. They are all lame! Just do it already.

5. Don’t break UNIX behaviors that should work.

Now why doesn’t autofs work again? Is it because your pushing AFP? That is nice and all, but I like NFS so keep the tinkering off of autofs and make it work again! I shouldn’t have to buy an Open Directory Server to serve automounts when I could just use autofs. This was either a real poor design choice, or a somebody from Microsoft was hired to work on autofs :)

By, the way I think I might be the only person in the world who wrote a how to article on getting NFS to work with GNU/Linux and OS X via Open Directory, so I know what I am talking about: Open Directory Part 3. Everyone else uses AFP, but you don’t have to. Apple just doesn’t publicize it!

6. Now why can’t I write cross platform applications with Mac Developer Tools?

Cocoa is really great, but it is quite a daunting task if you need to write a tool that works on OS X, *nix and Windows. I can understand Cocoa not working as that framework only exists on OS X, but why don’t you have a cross platform development environment? Why not work with one of the dynamic languages like Ruby or Python and build a toolkit that is OS X like, but builds applications for all platforms?

7. Applescript

Just dump it already. Seriously, it is way past its prime. Replace it with a modern scripting language like Ruby or Python. A lot of the work has already been done for you.

8. Why does OS X Server require a running GUI?

Take a hint from Ubuntu and have the option to not install and/or run the Window Manager. Your starting to get into big leagues with cluster computing, XSans, and quad core 1U servers. Why oh why do I need the GUI running sucking up tons of CPU and memory when it is a file server, or a render node? You need to lose your server OS beer gut and get into shape!

After reading one of my favorite blogs this morning, it got me thinking about the Zen of Mac. In the article that I linked to, Shannon mentioned that one of his hangups is not using OS X because it isn’t Open Source. The topic of smart people with hangups is very interesting actually and I would love to talk even further about it, but I am going to talk specifically about the Zen of Mac in this post.

Back in 2002, I was a systems administrator for the administration building at a Caltech, and I was solely responsible for providing support for Dr. Baltimore. Dr. Baltimore is smart. He received a Nobel Prize in his 30’s, he went to MIT, and he was the President of Caltech at the time. He is a mac guy, and that is partially why I was hired to do that job, as I have been into Mac computers for quite some time.

For those of you that remember, it was a huge deal to switch from OS 9 to OS X. It was my responsibility to design and build his OS X laptop and make sure that it was an easy transition. When the time came for me to actually deliver the laptop, things worked reasonably well and were intuitive as he would expect them to be. The punchline of this story is when I finally decided to show Dr. Baltimore the terminal. I think I mentioned something like, “One of the nice things about OS X is that it has a terminal and you can do nice things like……”.

The look on his face was priceless. I don’t remember the exact specifics of the conversation, but it was roughly, “Why would I use a terminal, it is a mac, that is the whole point!”. That taught me quite a bit, as here was one of the most successful and intelligent people on planet earth and he “got” Mac. He didn’t want to think about his computer as his thinking time was spent in other areas like making sure Caltech continued to be one of the top Science Universities and doing research on curing AIDS.

That is the Zen of Mac. You don’t think, it just works. As software engineers, systems administrators, or people that are very technical, it can be difficult to just not think about your desktop computer. The reality is that you are more productive on solving your other problems when you don’t think about your computer, it just works.

So as far as I am concerned, even though I work literally all day on linux machines from a shell, my desktop experience is OS X because it just works and I don’t have to think, my thinking can be devoted to solving my problems. Finally, for those people that are switching from a Linux desktop to an OS X computer, I would give this advice. Don’t think about how you used to do it on Linux, just forget what you know and try things out. OS X is designed to be intuitive and effortless. Fighting it to make it do what Linux does is not the proper approach. You must submit your will, relax your mind and float downstream on the white glow of the macbook pro monitor. Once you surrender your desires, and realize that desire leads to suffering you will truly appreciate the mac.

The discussion turned, as it occasionally does, to licensing and philosophy. (This is what happens when you hang out with smart people who also care about the subject.) I may have surprised a couple of them by saying, fire-breathing zealot for freedom that I am, that I don’t particularly care about the mythical software as a service loophole.)

Why not?

My current camcorder is over 5 years old now and I’m looking to replace it. I’m hoping one of you can point me in the right direction. Specifically, I’m looking for a camcorder which uses flash memory, writes files in a format which iMovie can directly start manipulating, has a crisp picture (which implies 3CCD, but not necessarily HD), and has a firewire connection.

I was looking at the Panasonic HDC-SD1, but iMovie 08 looks like it transcodes the files as it imports them. The reason that I wanted to go with a flash based camcorder is the ability to bring movies onto my computer more quickly than I have been able to with my current digital 8/firewire camcorder. But if iMovie has to transcode files as it imports, them, that doesn’t really buy me much. I suppose I could get the SD1, create an image of the files on the camcorder and mount it later to allow me to free up the flash drive in the camcorder, but that feels like a pain. I’d like flash as opposed to hard drive because of the ease of replacement if it goes bad and fewer moving parts. I want firewire because sometimes it’s nice to be able to record directly in iMovie.

So, 2 questions: 1) Am I thinking correctly about AVCHD and the SD1? and 2) Do you have any specific camcorder suggestions which roughly meet my needs?

The subject of “Non-Toxic Stimulation” has been on my brain for a bit now. I remember the subject being brought up a few years ago by my mentor, who passed away a couple of years ago. The best way to describe our relationship was to say that is was very similar to tuesday’s with morrie. One night Dr. Bogen described a friend of his that was in constant search for Non-Toxic Stimulation.

The definition for it is quite simple. Non-Toxic Stimulation is something that excites you but doesn’t kill or harm you. Toxic stimulation is quite easy to define as well. It is something that excites you but could kill you literally or figuratively. Examples of toxic stimulation could be smoking crack, driving a motorcycle down the freeway at 160 mph on your back tire, running with the Bulls in Spain, rock climbing without a rope, or fighting dogs. These are all optional activities that might leave you or others dead, maimed or in jail.

There are equally exciting Non-Toxic forms of stimulation, but they for the most part involve your brain. One form could be taking a class at Big Nerd Ranch and starting a Web 2.0 company. It could be finally teaching yourself higher math because it scares you and make you feel inadequate, but you know you can do it anyway.

The Non-Toxic form of stimulation I am engaging in currently is to teach myself Ruby as a way to practice test driven development. It is too hard to do test driven development in Python as I don’t want to slow down, but in Ruby, which is very similar to Python, I am not tempted to crank out some code as I can’t. It makes it much easier to practice doing TDD with Ruby and then apply that knowledge to Python.

In a now famous article by Joel Spolsky, he argues that you should never rewrite projects from scratch. To be fair, I’ve done this, but generally on open-source projects where I’m donating my time. I’m less worried about financial constraints or competitive advantage.

Aside from that edge case, I generally agree with Joel Spolsky. If something is a tiny project, refactoring is often trivial and if you want to do a rewrite, so be it. However, large projects are often dangerous to rewrite. But how did they become large projects? The vast majority of “large” projects I have worked on started out as small projects which gradually had features and cruft added. Interestingly, this is very similar to how agile methodologies work: build something small, but useful, in the first iteration or two. Always have working code and keep building on it. So the rewrite seems like a good idea because we’ve already shown we can create the project, right?

In comments on Sci Foo 2007, Tim O’Reilly wondered if giving Larry Wall a research job may have reduced his immediate practical concerns influencing the development of Perl 6.

I interpret the line of thought as “To what degree does the lack of a single well-defined problem to solve influence the concreteness or the abstractness of the solution and its delivery date?” (I won’t claim that that’s Tim’s line of thought, but it’s the question I’ve pondered this past week.)

I rarely see my colleagues, both in my work and in my hobbies. I spend most of my time collaborating with them, and one primary communication medium is email.

My personal mail server runs Postfix. Besides one small problem with a spam filtering message loop (mail forwarding to an account which rejects spam messages and includes the spam in the response is at least as evil as challenge-response), I’ve never had a lick of trouble.

Throw in server-side filtering with procmail or Email::Filter, as well as extended addresses, and I’m thrilled.

Then I discovered how to add just a couple of lines of configuration to make temporary, expirable addresses stunningly easy to create and manage and expirable, lightweight mailing lists, both of which require MTA administration beyond one initial configuration.

I never have to think about Postfix. It silently hums along, delivering buckets of mail. Thank you to all of its developers and contributors!

This blog will talk about two organizations I met with and their members–the Linux Mobile Foundation and the Open Solutions Alliance–plus other interesting people I met at LinuxWorld Expo.

A few years ago I was at a meeting where a salesman was trying to pitch his terribly expensive closed-source software to our company. Since security of our data was very important, this topic was raised a few times. I asked about how they encrypted their data. The salesman replied that since security was so important, the company created a proprietary encryption algorithm which was secure because no one knew how it was implemented. He seemed a bit flustered when I burst out laughing.¹

If you’re reading this blog, you probably have a technical bent and know that the vast majority of software out there has bugs. The larger the project, the more bugs. In fact, I’ve never worked on any significant (you know, large) piece of corporate software without known bugs. I’m constantly talking to friends who complain bitterly about long-standing problems with their systems. So why is closed-source software allowed to take a witness stand and accuse you of crimes when you’re not allowed to cross-examine it?

I’m running Firefox 2.0.0.6 on my (relatively) new (intel) Mac. Problem is that Firefox locks up several times a day. I found a forum which suggested disabling the anti-phishing functionality. Several people seemed to have benefited from said disabling.

I also found a Bugzilla report about the same behavior. It sounds like the anti-phishing thing can cause some problems during startup, but is probably not responsible for hanging during regular browsing. My problem is not on startup, but on browsing, so disabling anti-phishing probably won’t help me. But I’ve disabled it just on the off-chance that it will. If this hanging persists, I’ll either switch to Opera or Safari.

What are your experiences with FF on Mac? Crashes, hangs? Or is your world just peachy?

A couple of disparate threads have bounced around in my head lately, which makes me think that there’s some fundamental notion at work in the world.

Mitchell Baker and Matthew Gertner have had a brief back and forth over the nature of a public good (such as the Firefox web browser in specific, or free software in general). Mitchell’s position is:

A people-centered Internet needs some way for people to interact with the Internet that isn’t all about making money for some company and its shareholders.

— Mitchell Baker, Firefox is a Public Asset

Matthew Gertner wonders if corporate backing is necessarily, in itself, inimical to the creation and community-based maintenance of such a public good:

This isn’t about a small group of people trying to get rich. It’s about putting into place the most efficient overarching structure to achieve our common goals of choice and innovation on the internet.

— Matthew Gertner, More on Mozilla and Capitalism

The other thread synchronous in time comes from a comment Tim O’Reilly made a week ago:

I will predict that virtually every open source company (including Red Hat) will eventually be acquired by a big proprietary software company.

— Tim O’Reilly at [08.02.07 11:47 AM] in Microsoft to Submit Shared Source Licenses to OSI

Sometimes I wonder if the fateful 1998 meeting which gave birth to the term “Open Source” led the world of software freedom down a dark path. If the only way to get business to adopt the idea of embracing the power of communities to build software and ecosystems larger, more powerful, and more efficient than individuals could build on their own was to focus on economic principles, rather than the notion of the public good, is it any wonder that so many businesses seem to be indifferent at best to the health of those public goods?

To switch rhetorical metaphors, do you find it more likely that any given business would invest N% of its budget in energy-saving measures because it considers the investment ethically right on its own merits, or because it saves money and provides the basis for a nice, friendly press release?

Perhaps it’s inevitable that community-driven development, maintenance, and support will reduce markets for proprietary software up and down all of the stacks. Perhaps the most successful projects will have the strong support of businesses.

Do you want to rely on their goodwill to allow you to use, study, and redistribute software as you see fit? Are you willing to take the risk that they will encourage a healthy commons which allows you to use your data as you see fit?

I’m not sure.

Updated on 2007-08-10; corrected misattribution of Matt Asay to Matthew Gertner

Piers Cawley decided that his little language embedded in Ruby is a pidgin:

Casting the problem domain as the colonial power and ruby as the native language, it’s obvious that I’ve invented a pidgin language.

The nice thing about a pidgin (or a dialect, as Ben Scofield suggests) is that all of the power and syntax and semantics of the host language are available if you need them to express concepts that your little language cannot–without modifying the little language at all.

There you go; there are two good terms for describing the use of domain-specific language within a full-fledged programming language without co-opting a term with a perfectly good, existing definition.

Many workers in human services, she told me, are reluctant to provide data that would be useful to improve the services. She’d like to track homeless people as they move from one jurisdiction to another for instance, to provide better continuity of service and find out what works and what doesn’t. The agency staff are afraid that sinister forces within government will misuse data. While we have no lack of sinister forces in government, it appears that the people needing human services are more at risk of snooping by random staff people, facilitated by the awful security practices just mentioned.

I’m not surprised that employees would treat passwords as just one of the many random impediments they have to bypass each day to do their jobs. Given how many regulations reflect political grandstanding rather than life on the street, and how many well-meaning regulations outlive their usefulness, workers have to interpret the rules in a (shall we say) creative manner. I’m sure many employees in private industry get through the day the same way; it’s not limited to government. But an even deeper issue is at work.

But I don’t want to live in a world where the only thing the Internet is useful for, or effective at, or pleasant or fun, are activities where someone is making money from me.

— Mitchell Baker, The Internet and the Public Good

Kirrily Robert dissects a meaningless job posting for a Perl programmer.

This reminds me of a story Jim Shore told me about Fit. Developers in a company wanted to use the software, but their lawyers had grave concerns about the license. Eventually, the developers appealed to Ward Cunningham, who said that they were using it in the way he intended and he had absolutely no intention of bringing suit or other legal action against anyone who used his software appropriately. Even so, the lawyers saw that as an unacceptable risk.

The punchline is Ward’s final question. “You have to ask, do you work for your lawyers or do they work for you?”

Perhaps it’s time to ask that of HR departments.

As I read reactions from people to JT Smith’s Perl is Dead. Long live Perl. I see the usual knee-jerk claims that Perl inherently leads to unmaintainable code.

In my vast experiences in dealing with difficult-to-maintain code, I’ve noticed that nearly all of the messes I’ve seen had comments, documentation, and identifiers written in English. English is not an easy language to learn. It has inconsistencies (irregular verbs! homophones! homonyms!) and quirks (idioms! punctuation styles! possessive marks!) which make writing perfectly correct English–or even succinct and direct English–difficult.

There’s one good reason it’s difficult to produce correct and successful software from a full specification written at the start of a project.

Yet somehow it’s acceptable to allow programmers, presumably smart people with the ability to juggle small details, to write terrible, horrible, incomprehensible English but not comprehensible, concise, and correct code in languages which are orders of magnitude simpler than English. Hey, if you can use the pronoun “it” appropriately in English, Perl’s topic variable $_ should give you absolutely no trouble!

If (perceived) simplicity and regularity of the language were truly an important factor toward the maintainability of software, we should all be using Esperanto or Lojban to talk about our software. It’s not as if we don’t expect programmers to be able to learn new programming languages where appropriate.

Perhaps the true source of maintenance problems lies elsewhere.

Saturday, November 9, 2002 was my first Weblog entry, right here on O’Reilly Network. In 2005 I started using my personal Weblog Copia and soon I was posting exclusively there. On Copia I post on everything from music to literature to politics to technology, and I’ve long wanted to find a way to primarily post my tech writings here on O’Reilly Network, while having this aggregated with my other personae on Copia. This, “personal aggregation” as I call it, is one of the motivations for me to move from the PyBlosxom platform of Copia to a new one I’m building from scratch, with a few other co-developers. Bright Content (I don’t yet want to link to it because i think it’s not entirely ready for any attention) is a Weblogging system that embodies much of my long-standing interests in technology architecture relating to content.

But I’ll talk more about that project later. Meanwhile I’ll start the process of moving my technological Weblogging persona back to this Weblog, while ‘ll continue with other topics on Copia. It’s actually good timing for a different reason: after a separate one-year hiatus I recently renewed my agile Web column with the new article Introducing OpenSearch, a topic I chose in large part because of the resonance of OpenSearch with my philosophy of content technology. I look forward to continuing my conversations with you, through my articles and Weblog entries on this site.

JT Smith, president of Plain Black, the creator of WebGUI, and one of the unsung successes of using Perl in business, recently sent me this essay. He gave me permission to publish it in its entirety here.

(In the interest of full disclosure, the plush WebGUI octopus I have from YAPC::NA last year is one of the coolest pieces of swag outside of Hollywood, ever.)

c|net had a story up a couple of weeks ago entitled “Kids say e-mail is, like, soooo dead”. The story was about how kids were moving away from email and using instant messaging and social networking sites for peer communication. Maybe I know why. Or maybe I know why that could be such a temptation. The reason is simple. Email sucks. All email clients suck. No one of them has the feature set that I would like. Their handling of the email protocols can be atrocious, particularly IMAP. And don’t even get me started on having to interface with Exchange with anything other than Outlook. Also, you don’t know when someone has read an email (unless you’re using X.400, but that’s another story) or when they’re online. I can see why IM, social networks, and text messaging are gaining usage. Pownce, anyone? (BTW - I have a few invites left. If you’re interested, email me.)

UPDATE: The Pownce raffle is now officially closed. Thanks for everyone who emailed me.

My wife’s business has the need to make some sales online and so naturally I want to help. I know next to nothing about E-Commerce, so I started to do a little bit of research. It appears there are three options available to people currently in the market for a shopping cart that will process credit card transactions. There may be more options as well, but this is what I have found on short notice.

Option 1:

It appears to be relatively cheap to just host a whole website with a shopping card and credit card transactions built in. Google Checkout has links to a couple of complete solution providers that offer WYSIWYG admin panels and editors for somewhere between 30-50 dollars a month.

Option 2:

Integrated Solutions that you build into your existing framework/website. They appear to offer differing levels of flexibility, ranging from an API, to a link to another website that has a cart.

Option 3:

Something like Satchmo which is a “webshop” for perfectionists with deadlines. I found out about Satchmo through a fellow member of PyAtl who had some success with it.

Why does E-Commerce have to be so hard anymore? Maybe it isn’t that hard, and I am very ignorant. It seems like 2007 might be a good time for mere mortals and Mom and Pop businesses to start taking orders online. So, what is the best solution for minimal effort? Is there a solution that works well with Python, and/or Python Web Application Frameworks?

On one level it would be great to know there is a completely Open Source solution to a relatively simple problem. After all, why pay 50 bucks a month if you can easily do it yourself. I think VOIP is a good example of small business doing Phone Service themselves, and the same model might apply for E-Commerce. On the other hand, processing credit cards is somewhat risk prone, so perhaps there is a middle ground that still serves as a good price point for small business owners.

Scott Walters recently ranted on how Perl programmers tend not to perform OO analysis and design, at least when compared to Java programmers.

Setting aside issues of language design–though they’re important, I think there are other insights available–it does seem as if there’s much more literature available for developers who want to design large systems in Java than there is for developers who want to design large systems in Perl. (I operate from the assumption that the number of people who could write a program in Perl is probably the same as the number of people who could write a program in Java, if not greater, so I set aside questions of market size as well.)

Aristotle Pagaltzis suggested I repost my analysis here. In short, it’s all about the teachers.

At OSCON I attended a handful of the sessions that were sessions were related to web application development. I’ve compiled a list of features I’d like to see in the next web application development framework:

• Give me continuations. Let my request handlers check whether the user has authenticated and, if they haven’t, prompt the user and resume execution in the handler without losing state. If the user has requested to delete something, confirm the deletion first. I want this type of logic in the controller, not as a confirm() method in the JavaScript.
• Let me attach listeners to server-side models and update the interface when the data structures have changed. For example, given a list of favorite movies that is stored in memory or in the database, I want a <div> to be modified automatically when a favorite is added or removed.
• Keep the API simple. Give me little languages, DSLs, data structures or whatever the fancy name for them is. Let me focus on the logic of what I’m doing. Don’t limit these little languages to only configuration files — Let me write my HTML templates in whatever language I’m using throughout the rest of the framework.
• Make sure testing is a breeze. Let my tests “click” on <div>s and make assertions on the contents of elements.

Fortunately, there are a few frameworks that have some of these features already.

• Google Web Toolkit lets you write applications entirely in Java. The Java that needs to be run client-side is compiled into JavaScript. Designing interfaces in GWT is similar to the popular toolkits for desktop applications, and the testing framework seems solid.
• Jifty, a Perl framework created by Best Practical, includes a lot of the aforementioned. Templates and tests are all written in Perl in a mini-syntax, and they’re working on compiling Perl to JavaScript. It even has continuations.

Above all, I never want to have to write an <a href="..."> tag again. Giles Bowkett, in his HREF Considered Harmful talk, explained that it’s the modern equivalent of GOTO. I agree.

Here’s my wrap-up of the Open Source convention. I published an earlier blog on it as well.

Andy Lester is renown for his evangelism of technical debt awareness, and his talk at OSCON was full of rich lessons on improving your code and yourself.

Managing upward is a theme in his talk, but it isn’t his primary focus. Andy wields upward management as a tool which can be used to improve code quality, but it’s a topic that deserves significant attention as well.

“Talk in dollars. This is the language that management understands.” — Andy Lester

Nobody develops for Perl anymore, CPAN is too crowded.

    Jordan Henderson

Senator Dick Durbin has announced an online forum about broadband policy that started last night. Today’s news that iPhone sales are disappointing provides one illustration of the importance of this issue. While the device is overpriced, I have no doubt that a key drag on its uptake is the slow network AT&T has devoted to it. This would have been considered ludicrous in East Asia.

US Government, as we all know, is fleeing from openness as fast as its so-called leaders can run. At Tim O’Reilly’s open source briefing yesterday, open source advocate and version control expert Karl Fogel presented a case for recording and releasing all communications that go into making laws. Apparently the New York Times picked up Tim’s blog on the subject.

The briefing also presented open web APIs, open source hardware, and other examples of how the open source movement has spilled over its origins in free software. The most popular free software packages are still infrastructure: operating systems, languages and language tools, system administration packages, and so forth. But there’s no doubt that everybody is evolving in response to this powerful model for encouraging creativity.

I just read an article at Linux.com about the OS habits of Linux users. The author of the article asked Linux Torvalds about his habits and found he exclusively used Linux. Torvalds said, “I don’t use either [Windows or Mac OS X]. OS X is kind of pointless (pretty much anything it has, Linux can do better) and Windows offers stuff that I don’t much care about (mainly games — and I’ve got games machines for those).” Before I comment on this any more than I have, let me just say that I have been a near-exclusive Linux desktop user since 2001. I love Linux and I still think there are some areas where Linux dominates. However, to Torvalds, I say, “hogwash”. He can get around careful scrutiny by his choice of words “pretty much anything it has” and “Windows offeres stuff that I don’t much care about”.

But the spirit of what he’s saying seems just dead wrong. And it’s an attitude that’s pervasive among many Linux enthusiasts. Whether Torvalds has the zealotry I’m about to discuss is irrelevant. It’s just a launch pad for me to address this attitude. The attitude goes something like this. “Linux is an awesome OS. We’ve come so far in a short amount of time. We have everything anyone would need. And bling to boot. Linux is ready for the desktop. In fact, it’s ready to take over the desktop.” There is a lot of truth in what both Linus said and my characterization of the Linux zealot.

Truth 1: Anything that Windows and Mac can do, Linux could do. Notice my choice of words. I said that Linux could do anything that Mac and Windows can do. But the sad story is that Linux is not currently doing a lot of what Mac and Windows is doing. Please, please, please someone show me wrong on this! Please show me a DVD authoring application on Linux that is as easy as iDVD is on Mac. Or a video editing application as easy as iMovie. Please! Yes, I know that there is wine and you can often get Windows apps running on Linux. But 1) it’s hit or miss and 2) most of the apps that I’ve gotten to run in wine look…..let’s just say “bad” to sound polite.

Truth 2: Linux has made considerable strides in the past few years. The desktop looks spectacular (both kde and gnome). There are tons of top notch applications available for free download. Hardware recognition and support works better than it ever has. I don’t think this one needs to be shot down, so I’ll leave it as it is.

Truth 3: Linux has the bling. In my opinion, Linux is actually winning the bling war. Just check out Beryl/Compiz/Fusion. It rocks. For bling, it is (my opinion, again) unrivaled. And there are some pretty cool productivity enhancements, too. But let’s not confuse bling and even cool productivity enhancements with a usable desktop (not that Linux has an unusable desktop). They aren’t necessarily the same thing. Just because Linux has bling doesn’t mean that its applications are well integrated with one another. Or that the applications work well on their own.

The point of this rambling is that all three of Mac, Linux, and Windows do some things well and other things not so well. Personally, I don’t like getting on Windows. It feels square and wooden. But it does some things pretty well. And I just bought a Mac a few weeks ago and I’m really enjoying it. I’m not at all ready to say that neither Windows nor Mac is really competitive with Linux. Conversely, I’d say that each of them spank the other two in some areas and don’t do so well in other areas. So, can we please let the zealotry die? Please?

Update: I’m closing the comments on this blog post because of recurring blog spam. If you want to carry the discussion on, please email me and I’ll post a new entry.

PyAtl is switching to Plone as time is short and we want a CMS that everyone can edit. A few people are are working on getting Plone configured and we are looking at using the gmail authentication plugin and the forum plugin.

All of this has got me thinking? What is the story on Plone and/or Zope? There is all this talk of Django/Turbogears/Pylons, etc., but what about Zope and specifically Zope3 and Plone? In the younger crowd you almost never here anyone talk about anything related to Zope and I wonder why?

So, what is the dope on Zope? Can you turn Plone into the next myspace? How hard is it to learn Zope3?

I’ve been listening to The Linux Action Show podcast for a few months now and really enjoy it. I’ve tried other Linux podcasts, but they seem focused on the noob level. Can anyone recommend a good Linux podcast for experienced Linux users?

I’m pretty good at Perl, so I hear a lot of comments about programming language syntax. Many of them are fluff around the old argument that “I don’t like to read punctuation.” Many of them bring up the silly idea that an ideal programming language syntax should be so intuitive that people who’ve never used the language before should be able to understand programs written in the language.

That’s a ridiculous argument.

It’s been some time since I don’t follow closely Lua development. But I try to keep updated with what’s going on. The announcement of LuaPOD 0.1 caught my attention (due to the gathering of three technologies I find quite interesting).

In the spirt of health competition, and because I selfishly want to know what blogs to fill my Google Reader account with, what are the Top Ten Python Related Blogs? Also, how many RSS