I've always thought of hacking as a *craft* rather than as an art. The innovation and experimentation that we use to produce software is no more or less profound than the inventiveness that has gone into the refinement of things such as weaving, mechanical devices, farming, food preparation, and other practical technologies. As with these other crafts, the tools used to practice them are a natural outgrowth of the raw starting materials. (Interestingly, as I know we've discussed, the techniques that programmers keep alive and the ways in which they propagate their knowledge of these techniques also follow the tradition of artisanal production. The organizational structure of most large website providers, corporate development groups, and open source projects is something close to the guild.)
For a programmer, the choice to use types, for example, is driven by the task at hand - most programmers use both typeless scripting languages and more rigorous compiled languages together. [Sometimes it makes sense to use gravel, but sometimes it makes sense to pour concrete. And of course, gravel can be used as an element of concrete; gravel also makes an excellent base on which to pour...]
Successful web apps, just like every other kind of app, will harden due to necessity. Once they are interlinked using web service techniques and driven programmatically, once they've got millions of barely attentive users who don't *want* the interface to change, or once the cost of operating the franchise becomes critical, the kinds of change that you celebrate will no longer be desired or tolerated. The good ones will harden with lots of flexibility built in. The bad ones will just seize up. And once they are locked down and expensive to run, techniques such as compilation and context-sharing will start to make lots of sense for safety and efficiency reasons. [A favorite example of a really efficient, large, distributed app is any living organism. The information that drives the process can be found in easily hackable source form in its genome, but the expression of this information is most assuredly typed once that genome is expressed!]
The rigidity of shrinkwrap PC ISV software is really an anomaly. Most corporate systems are just as dynamic as web apps, due to integration requirements, changes in exogenous data representation, pure size, or any number of other factors. Not surprisingly, the guilds that are the stewards for these apps make heavy use of databases, scripts, and data-driven integration techniques based on extensible formats like XML.
I guess that the point that I'm making is that I don't think that there is any fundamental paradigm shift involved in open source web apps. The tools and techniques that you are seeing have been used by developers for a long time, although the materials and resources (and therefore the tools) are changing as hardware and networking infrastructure evolve. I love Python as much as the next guy, but I can promise you that a really big, rotting old Python-based LAMP app will be just as smelly and hard to maintain in 10 years as the VB codebases that I've seen that won out 10 years ago and survived based on VB's ability to quickly "sketch" PC apps in a dynamic way. The guilds that used VB to sketch first, after which they channeled the dynamism of the app using structure *rather than code* are the guilds that are still happy and prosperous today. A formula that worked then and still works now is to couple databases/data formats with metadata-driven building blocks - in the long run, this is a far, far, better way to run a business than constant resketching and recoding. Web apps certainly have humans inside, but the less time those humans spend on the craft of programming and the more time they spend on the art of communicating at a higher level, the more valuable their contribution will be!
Just a random ramble - ah the pleasures of being out of work :)
And yes, as Paul Graham points out (thinking much like Dave), in large companies, these processes are likely to be constrained, "hardened" if you will. But I do think that the skills that are required to build a successful dynamic web application are different than the skills required to build a successful compiled binary application. And it's important to celebrate those differences.
I totally agree with Dave that, as one friend once said, "you pick the hat to fit the head." Scripters typically use compiled languages as well as part of their toolchest. But there is a snobbishness among some developers of compiled applications, where they don't seem to realize just how important dynamically typed languages are.
I think Dave's comments about craft guilds are also fascinating, but that's a whole other subject... Much more relevant to Paul Graham's original essay.
Tim O'Reilly is the founder and CEO of O'Reilly Media, Inc., thought by many to be the best computer book publisher in the world, and an activist for open standards. O'Reilly Media also publishes online through the O'Reilly Network and hosts conferences on technology topics, including the O'Reilly Open Source Convention, the O'Reilly Emerging Technology Conference, and the Web 2.0 Conference. Tim's blog, the O'Reilly Radar "watches the alpha geeks" to determine emerging technology trends, and serves as a platform for advocacy about issues of importance to the technical community. For everything Tim, see tim.oreilly.com.
oreillynet.com Copyright © 2006 O'Reilly Media, Inc.