LinuxWorld Expo in San Francisco next week is going to be huge!

As Editor-in-Chief of LinuxWorld Magazine, I’ll be conducting interviews all week of executives, developers and authors at the show. The interviews will be streamed out live from the show floor all week.

One of the interviews I’ve got scheduled is with Bruce Perens. I’ll be discussing with him his keynote address on the “State of Open Source”. I’ll blog a summary of this Wednesday or Thursday next week.

I’ve got too many executives and industry people signing up! I want to talk to more Linux authors and developers! If you’re a Linux author or developer in the San Fran area or if you’re going to be at LinuxWorld, drop me an e-mail or just sign yourself up at:

http://www.sys-con.com/linux/LinuxWorldExpoRadio.cfm

If you’re interested in just listening in, this link will give you the schedule of who’s on when as well.

One of the major topics at OSCON this year was the notion that software is increasingly becoming a commodity for which the customer is no longer prepared to pay a premium. So, as the market for Open Source software expands and companies rush to define new areas they can create business in - what is there to be learned from other high-premium goods that have since become a commodity?

At OSCON the software market was often compared to the construction industry with people moving to DIY in order to supply themselves as cheaply as possible.

Sitting at Portland airport waiting for my flight back to Germany it hit me that flight (or flying) has also moved from being something that only a few could afford and since become something that is available for (nearly) everybody.

In the airline market there are now many low-cost carriers who are offering cheap tickets to destinations for which other airlines still sell higher-price tickets. Some European low-cost airlines actually give away a certain number of tickets for free (all the customer actually pays is tax and a handling fee). Flying for free - who could have imagined that back in the days of Pan American? So in a way those cheap airlines can be considered the Open Source providers around flying.

So, how have the cheap airlines altered the marketplace?

Reduced vendor lock-in

There were days when you could fly with the national airline or you could walk. As a consumer, you had little choice and so you basically payed the asking price if you wanted to fly. Of course things changed before the low-cost airlines came on to the market - but the surge of cheap airlines now means that you have plenty of choices and prices to choose from when picking your flight. The airline customer has more power - because she can choose.

DIY Flying

At OSCON, Doc Searls talked about the way companies are no longer buying the software from their year-long vendors but moving into a Do-it-yourself market where they can freely pick and choose, combining Open Source components with software they may already have. In addition, languages like Python are being used to glue the bits together and allow a more rapid change of the complete application. Doc compared this to the DIY part of the construction business.

In the flying-market, customers are going to the Web as their way of doing DIY. Instead of taking pre-configured packages from tour-operators they are using the online travel web-sites to put together their own configured and personalized packages. This trend has now forced some German travel-agents to charge a refundable fee if you go into the agency in search of your ideal holiday-package. The fee is to prevent you taking their “consulting” and then using the Web to piece together the same package at a cheaper rate.

In the software business the DIY “trend” is leading to software companies becoming increasingly consulting focussed as the revenue for the actual software dwindles. Software companies will need to make sure that they can turn pre-sales consulting into a form of revenue. They will need to prevent customers from gleaning information in that phase of a project and then “doing it all themselves”. Of course, Open Source actually supports this and so software companies will need to focus a part of their business model on that area.

Flying from regional airports to attract additional customers

Cheap airlines often operate out of regional airports as opposed to national hubs, because this lowers their operational costs and attracts customers who would not normally be inclined to fly because of the distance to the airport.

Open Source businesses need to attract customers from areas that are perhaps reluctant to look at “free” software. Open Source software needs to move into areas that are perhaps as yet not really targeted. There was quite a bit of discussion at OSCON about packaging Linux for specific problem-areas or devices. This is an example.

“Traditional” airlines form their own low-cost airline

At first the more traditional airlines were sceptical about the low-cost airlines. In fact they ridiculed the chances of low-cost (and perhaps low-comfort) flights ever taking off. And yet they did and those “old-world” airlines were forced to react. And many did so by forming their own low-cost carriers. So now, the traditional airlines have discovered the new market-place of low cost flying and are competing fiercely.

I think this point is particularly interesting when we look at the Open Source market. If applied, it would mean that eventually companies like Oracle and Microsoft may be forced to produce Open Source versions of their products. “Low-comfort” versions that you can get and use for free. Oracle could release a low-cost or Open Source version of their database in order to compete in the same market as MySQL. To enter the low-end market and then provide a migration strategy to the higher-cost, supported version seems similar to buying a cheap - no comfort flight - and then being told that the same company also offers deluxe flight on their “other” airline (”and you know the tickets are really not that expensive!”)

Although knowing little about the cheap-flights market, it doesn’t seem difficult to find points that can be compared to the evolving software-market. And, as is the case with comparing the software business to the construction business, there seems to be a lot that can be learned.

Over a week ago I went back-to-back first to the
AlwaysOn Innovation Summit(AO2003) then to the yearly conference of the
World Future Society (Oddly enough, they took
down the conference web page already, which makes it really tough to link to
any particular session).

I would have preferred to have at least a month in between the two, but it
gives me the chance to compare one with the other. I wrote about the Summit
already, therefore this post is mainly about the WFS conference.

The World Future Society was founded in 1966. My guess is they were riding
the wave of the promises of NASA space travel as well as "The
Jetsons" cartoons on TV. These promises of flying cars and towns on
the ocean floor never materialized and that also deflated the field of future
study. People just lost interest.

With the acceleration of change in the last few years, my feeling is that the
interest in the future and what it has in store for us is on the rise. But my
perspective may be a bit biased; after all I am running a Futurist
Salon and am helping to organize the first Accelerating
Change Conference at Stanford in September.

Future Studies is a wide field and that reflects in the World Future Society.
Members reach from the progressives moderns, for them more technology is the
solution to all of our problems, to environmentalists that see doomsday scenarios
just around the corner. The most famous of these is "The
Limits to Growth" report of the Club
of Rome.

Matthew R. Simmons revisited
that report in 2000. His sad conclusion: "As the 20th century came
to an end, wind and solar collectively only created one-tenth of 1% of renewable
electricity in the U.S. What this means, in simple arithmetic, is that the two
"promising new energy techniques," heralded to hold such promise when
The Limits to Growth was first published, still account for only 1000th of 1%
of U.S. electricity generation! To say that no progress was made in this taxing
energy issue since The Limits to Growth first hit the bookstands is a colossal
understatement."

Looking back, I realize, that this is one of the charms of the World Future
Society Conference - the huge spectrum of ideas and viewpoints offered there.
I very much enjoyed the chance to sit in a session called "The Chaplains
of Tomorrow: Universal Quality Enhancers" Not being a native speaker I
didn’t even know what a Chaplain was, Charlie Chaplin is the closest I ever
got to this word before. After 10 minutes I understood that it is not for me
right now and moved on.

It is a big cafeteria system, where you pick and choose from a dozen parallel
sessions. You can go through the event without ever getting your viewpoint challenged
may it be doom or utopia. But I am a firm believer that the real interesting
stuff happens at the fringes, when different fields come together.

This is the strength of the World Future Society Conference and they should
put a stake into the ground and claim: "Here is the one time in the year
where all these disciplines come together". You only have to look at the
column headers of the schedule at a glance to see that this is true: Business,
Economics, Environment, Futures, Globalization, Governance, Health, Learning/Education,
Science/Technology, Society, Values.

With a few well-selected panels (and there were too many of these at the AO2003),
one could very nicely bring the different viewpoints to the forefront and create
a fruitful dialog. It would be great to enhance that dialog with additional
channels, like a conference Wiki or chat. Maybe the organizers should talk to
the Socialtext folks that were running
these things at the AO2003.

These tools would energize and rejuvenate the event. Prerequisite is of course
ubiquitous Wifi; I wasn’t the only one
missing it. Maybe within a year every respectable conference hotel will
have it in their portfolio. In some areas, hotel signs offering "Free Broadband"
are starting to outnumber the "Free HBO" ones, Wifi can’t be too far
behind. Another possibility is to get a sponsor to provide Wifi. World Future
Society conference organizers could really learn from the AO2003
where there were 17 sponsors at the bottom of their web
page. Lots of companies should be eager to show that they are tuned in and
forward-looking by being part of the next World Future Society Conference.

Will post about some of the sessions soon too, if I find the time :-)

Other suggestions on how to rejuvenate the World Future Society Conference?

I’m moving in a few weeks, so I had to ask DirecTV to mount a dish at our new place. I went to directv.com, clicked “Customer Care”, and then “Moving?”. Only two clicks to find what I wanted. Not bad.

Then, I had to dial the phone number: 1-877-616-MOVE. Argh! Why did they have to make it spell something, and only half of something at that? Now it takes two steps for me to dial on my desk phone: 1-877-616, without having to look, and then I have to look at the keypad to find the M, look to find the O, oh crap, wrong key, now start over….

I’m not knocking phone numbers that spell things. They’re useful when you have a phone number that you need to recall at a moment’s notice without looking it up. 1-800-CALL-ATT is a good example. There was a carpet chain in Chicago that was 312-CARPETS. Those make sense. In this case, it doesn’t. I don’t need to call the “move my dish” line more than once. Even if I did, how am I going to remember both the 616 part and the MOVE part?

Here’s a feature that I’ve never seen used in a valid way: That damned “Reset Form” web form button. It’s like the <BLINK> tag: Web designers put it out on forms, but without really thinking why.

How many times have you seen a form like this?

Who is going to intentionally avail himself of that Reset Form button? How long a string does it have to be to make it worth the user’s time to just click the Reset Form and start over? “Whew, saved myself 37 backspaces characters! Good thing that button was there!” (Any time discussions of a GUI turn into sounding like Jerry Seinfeld channeling Jakob Neilsen, something’s wrong with the GUI…)

An even worse place for the Reset button is the very long form. Ever sign up for one of those free computer rags magazines where you truly are just a couple dozen points of demographics?

A mere 7 months after I wrote about last-minute business RSS here, Amazon gets it. Nearly. Amazon now provides RSS feeds embedded inside the HTML pages. To actually subscribe to the RSS you will need to take a look at the source of the page and then find the link to manually add it to your RSS newsreader (at least my version of NetNewsWire can’t “auto-discover” the RSS feed). Here is an example RSS feed.

Related link: http://news.com.com/2010-1071_3-1026988.html

A C|Net article asks the question begged by the recent SCO v. IBM lawsuit: Who should be liable in the event that customers wind up using software that was created from misappropriated intellectual property? Could a non-profit like the FSF or the OSF stand forth as a lightning rod for liability over Free and Open Source licensed software? Or is the indemnification issue going to prove a fundamental limitation on the commercial uptake of Free and Open Source software?

What’s the answer to the intellectual property liability issue with respect to Free and Open Source software?

Related link: http://www.nytimes.com/2003/07/24/technology/circuits/24font.html

The winner of a University of Minnesota Design Institute commission is a typeface called “Twin” which can adjust its appearance based on external factors. One demo available on the design.umn.edu site renders text of your choosing with the typeface appearance based on the current weather in the Twin Cities.

The typeface was created by the Dutch company LettError. Aside from being keen in its own right, the typeface gets bonus points since one of its designers is Just van Rossum, brother of Python BDFL Guido van Rossum and programmer in his own right.

Related link: http://www.smh.com.au/articles/2003/07/23/1058853121030.html

It’s about time somebody made SCO face up to the consequences of its grand-standing and howlling, which have gotten immeasurably worse since I wrote my weblog on

Irresponsible SCO. Their lawyers are essentially telling every Linux user that we’re engaging in illegal activities, and that by the way we can save ourselves by buying them off.

How about a similar parry to SCO in the United States? Are we a country that lets powerful forces go on wildly destructive campaigns on the basis of inflated or even untruthful claims?

I’ve been watching more and more people use href="http://www.makeashorterlink.com">make a shorter link — a
web-form service that you cut-and-paste a long link into and then
returns a 38 character URL that is easy to e-mail around. On the back
end, the Pants Collective’s service presumably records the URL in a
database and then returns another URL that can be used to key to the
longer one. While this service seems great for e-mailing URLs around
(even then, I’m not sure — most e-mail clients can handle long URLs
these days), I hesistate embedding these shorter URLs into published
web pages.

Think of this like giving your friend somebody to get more directions
to your friend’s house. What happens when your friend arrives at the
first location, and there are no more instructions there? What does
he do? On the web this is of a bigger problem — which link will web
crawlers index? Does this level of indirection break programs (or
more likely, does it exacerbate stupid web programs that don’t handle
meta refresh tags)?

What I would love to see is this service aquired by href="http://www.google.com">Google or the href="http://www.archive.org">Internet Archive. What they could
provide is a system with more value — this key could key into a
cached copy or an updated URL when it changes, or it could key not
only to the page but its entire history. aAkey for the sake of
shortening to the URL (and not even shortening it to something that
people can remember) seems mostly pointless, but a key that provides
more value might be interesting.

Related link: http://ao2003.com

Just came back from the AlwaysOn Innovation Summit
organized by a team around Tony
Perkins of Red Herring fame. He set the tone in his opening remarks: "Consumers
and businesses have an insatiable need and drive to be always on." (One
commented that the real money will be made in 10 years, once the promise of
AlwaysOn is fulfilled via getting people off, giving them back anonymity :-)

Consistent with the always on theme they used an interesting mix of cutting edge
conference technology:

The whole event got webcasted and all sessions will be archived.
Kudos to AlwaysOn and their sponsors
for offering that service for free to the online world. (Registration required).

Wifi throughout the conference, which after a rocky start enabled the people
on the floor to have an eye on and participate in the webcast chat, as well as
being distracted by their inbox and the Internet. Once in a while the chat was
also projected to one of the big screen on the floor.

Online polls, where the goal is to bring in the opinion of the online users as well
as the audience via the Wifi connection. Not sure whether it was the questions,
but it felt a bit forced and wasn’t really enhancing the panels.

Conference Wiki, (What is a Wiki)
was a bit hidden in the webcast window. I personally love the possibility of
enhancing a conference by changing an area of its web page on the fly. I think
Wikis will blossom in the next years, as soon as more people understand and
then use them. Check the list of other conference bloggers, I discovered Tim
Oren’s Due Diligence there.

Maybe I was still jet lagged, having just returned from Germany, or other
things were on my mind, but after the webcast was in full swing I had problems
taking notes while following the panels, the chat and vote in the polls.

Everyone has to find his own comfort level of how many things can happen simultaneously
while they still get the most out of an event. You need the discipline
to shut certain things off that go beyond your threshold.

All that said, I wouldn’t want to live without any of these technologies at a
conference, only some fine tuning is needed.

Larry Wall’s State Of The Onion,
where he
announced Ponie, which will be Perl 5 running on top of Parrot, renewed my faith in the Perl 6 effort. I’ve never been comfortable with the idea of Perl 6 being two, two, two projects in one: Perl 6 and Parrot. Now we can have two separate and distinct goals.

Having Ponie and Parrot as two deliverables means we can have twice the faith in the correctness of the code. Without this setup, we might not know if a problem lies with Parrot or Perl 6. Now, we’ll be able to isolate some of our problems.

Once I started thinking about the project (and talking with Leon Brocard over gin & tonics), I realized how much automated testing would be required to verify that Ponie works as advertised. I put on my Testing Evangelist hat and started hatching plans for how to make use of the existing Perl 5 tests, as well as the tests for the top 20% of CPAN (following the 80/20 rule), to help verify the correctness of Ponie. Not all modules have great test suites, so I’m going to work with those authors to see how we can beef them up, and increase the coverage, both for the module and for Ponie.

Automated testing is a low-cost, high-payback benefit. Cycles are cheap, and it makes sense to let the machine be watching over your shoulder as much as possible. I’d like to see daily, or even hourly, builds of Ponie to make sure that nothing’s been broken.

(And please, allow us a month or two of bad Ponie puns. The Java guys have never stopped with theirs….)

(Mirrored at http://use.perl.org/~petdance/journal/13544)

How else can we help ensure the correctness of Ponie?

Related link: http://www.twingalaxies.com/

Steven Wiebe is the new King of the Jungle among Donkey Kong gamers.

With a score of 947,200, he demolished the old record of 879,200. Now that’s something to beat your chest about!

Think of the tricks he must know. It takes real ingenuity and creativity to think all those moves up. The tricks he knows must’ve taken years to figure out.

“Donkey Kong Hacks” would be the perfect vehicle to capture all that expert knowledge. It’s perfectly in line with the rest of the series - tips and tricks from seasoned experts.

Can I preorder mine? Wonder if I can get it autographed?

Related link: http://conferences.oreillynet.com/pub/w/23/oscon_grid.html#friday

I arrived a couple of minutes into href="http://www.onlamp.com/pub/a/onlamp/2003/06/17/dyson.html">George
Dyson’s talk about the pioneering computing work done at the IAS. I’d been
looking forward to this talk for months and it did not disappoint —
suffice it to say that I’m trying to get the content online very soon.

Two things struck me. First, the spirit of hacking I enjoy with software is
very similar to the spirit of hacking that went on building the big machines of
the 40s and 50s. (It also helps that Ward Cunningham demonstrated the tiny IC
hacking he’s been doing for a hobby right after the talk.) Second, Dyson
exhorted the audience to keep alive the spirit of that age by continuing to
explore the world in an open and free fashion. I’d love to hear from Dyson
again next year.

Next up, Miguel and Nat from Ximian
talked about their successes with Mono. It was quite interesting to see
Eclipse running (well, launching with painful slowness, though anything on
stage with Nat will look amazingly slow) on Mono.

I have mixed feeling about Mono. On one hand, I’m glad to see more
attention given to a language that’s more appropriate for systems programming
than C or C++. (Given the long history of Java being all-but-ignored on Linux,
I’m not sure what to say.) On the other hand, I have to agree with Dan when he
points out that href="http://www.sidhe.org/~dan/blog/archives/000218.html">the .Net folks are
missing a lot of useful language features. Then again, there were a lot of
people asking why you’d ever want garbage collection five or six years ago.

Next up, I caught most of the session about why Ticketmaster uses the
patronage system with open source developers. I’d love to hear this message
repeated further and wider — if you have an expert in-house, you can
support an open source software project yourselves. That’s not to say that
Ticketmaster won’t ever call someone from Apache, MySQL, Linux, or Perl, but
that they have control over their destiny. (It’s also nice to hear that Geoff
Young has a job. Good work!)

Ask (from Ticketmaster) and Robert then presented an overview of href="http://www.perl.org/">Perl.org’s single-sign on process. SSO has
been a bit of a project on the O’Reilly sites lately and it’s on a lot of
people’s minds. There are ways to make it easy, and if I ever end up working
with lots of sites and lots of subdomains again, I know where to look. By the
way, Robert and Ask are two of the hardest working and least appreciated people
in the Perl community. Thank you.

Finally, the official content wrapped up with a talk from Nat’s old college
buddy Milton Ngan, of Weta. Milton had
lots of wonderful photos, including their massive (and continually growing)
server room as well as video explanations of the more impressive shots from
The Two Towers. Unfortunately, there were no sneak peeks at The
Return of the King. The Gollum footage was quite nice, though.

After that, the party broke up into smaller parties. Lots of people headed
off to the Zoo and the final Stonehenge party. I went home for lunch, a nap,
and some quality time with my e-mail. Yeah, that was naive.

Saturday and Sunday were Perl 6 design meetings, with lots of edge cases of
Perl 6 OO worked out. A few still remain, but we’ll get there.

What worked well at OSCON this year? The same things that always work
well: good talks, good speakers, good attendees. Portland was quite nice,
though I’m less enthused about the light rail system than I was before I tried
it every day. The OSCON wiki was a
tremendous success and there’s a nice buzz inside O’Reilly about using it at
other conferences. Oh, and wasn’t the weather nice?

The Portland user groups were very nice. Thanks to you all. Our href="http://conferences.oreillynet.com/pub/w/23/sponsors.html">sponsors
also deserve a round of applause. Active
State, Apple Developer
Connection, Hewlett-Packard, href="http://www.ibm.com/">IBM, href="http://www.sunsource.net/">Sun, href="http://www.ticketmaster.com/">Ticketmaster, href="http://www.realnetworks.com">RealNetworks, href="http://www.redhat.com/">Red Hat, href="http://www.dyndns.org/">DynDNS, href="http://www.jabber.org/">Jabber Software Foundation, href="http://www.microsoft.com/">Microsoft, href="http://www.pogolinux.com/">Pogo Linux, href="http://www.stonehenge.com/">Stonehenge Consulting, and the media
sponsors all helped bring OSCON to Portland. They’re worth considering.

Some things weren’t perfect. The wireless network was flaky early in the
week, but with some troubleshooting (thanks, Schuyler and Rob!) and some
hardware (thanks, Portland Personal
Telco!), things had much improved by the end of the week.

Several of the rooms were too small for their talks, though that’s the best
kind of problem to have. In particular, subjects like testing, Ruby,
Subversion, and Fit (presumably also Kwiki, though I’d seen that presentation
before) were more popular than anticipated.

All in all, I’m quite satisfied with OSCON 2003 and am happy to have played
a part in making it happen. Here’s to an even better OSCON in Portland
(hopefully!) next year!

Software process improvement doesn’t have to come only with the approval of the Pointy-Haired Bosses. There’s plenty you can do on your own without telling anyone… until you show how well it works.

I spend a lot of time in the pulpit of automated testing, talking about the benefits of writing tests for your code, in effect creating your own guardian angels. At a
Chicago Perl Mongers meeting where I preached the gospel of testing, someone said “That’s great, Andy, but I can’t do this. How do I convince my boss that this is a good idea?” The short answer: you often don’t need to.

The number of tools available to us in our toolboxes is amazing, and you can usually make great use of them, even if you’re the only one using them. You can set up macros in vim or emacs to make life easier for your fingers. Create shell scripts in your own ~/bin directory to speed repetitive tasks. Use CVS to keep track of changes to your files, even if you’re the only one in the shop. Check your web pages, especially those dynamically generated, for syntactic errors with
weblint or any of the online checkers. Set up automated tests on your code, even if you’re the only one on the project who’s doing so.

Acting as the one-person prototype also means that you’ve got real-world experience with the process that you’re pushing. Maybe you find that CVS doesn’t do all your department needs, and you need subversion instead.
Better to find out after your own experimenting, rather than after it’s implemented across the department.

Improvements under the radar mean immediate results, whether good or bad, without the endless meetings and discussions of how things should be. Far better to show your boss or departmental peers something that actually works than to talk endlessly about what might work.

Whatever your idea, try it. Play around. Maybe it’s not the best practice, but you won’t know until you try. Once you have some success, whether it’s taken the drudgery out of a tedious build process, or had a bug found by an automated test, show it off. In the words of the great Bobbie Flekman, “Money talks, and BS walks,” and we PHBs always like the sound of money.

What have you done to improve the quality of your coding life? Was it under the radar? How did you approach it?

A look back at OSCON. I’ll do my best to weave together some impressions under my
three themes of PHP, Productivity, and Trends. I’ll also mention a couple of things (collaborative editing, and
wireless video) that will undoubtedly come into play in future
conventions.

Any wrap-up
after such a wide-ranging conference is bound to be a sliver
of a sliver - there were lots of Perl, Python, Ruby sessions
I never went to; not to mention many non-programming sessions,
such as legal issues (SCO vs IBM, Intellectual Property Law Basics for Open Source Developers). Indeed, there were often 11 things going on at once!

PHP

Sterling kicked off the week with a great tutorial
on Advanced PHP. I wrote about this in my Monday ORN blog.
A key area of interest in Sterling’s talk is performance. I’ll be checking
out the PEAR package APC - Alternative PHP Cache very soon. PHP can take longer to compile
classes on the fly than to execute them, so a cache can really
speed things up.

I got two looks at doing web services with PHP. The more
theoretical talk was Sterling’s “Web Services with PHP”. Adam Trachtenberg did a more applied version
with his “Web Services in PHP” (yes, titles are similar). Want to
come up to speed on Web Services with real data? A first step
is to be a client that consumes a web service. One possibility
is to head over Amazon, sign up ( ) and experiment
with product searches and how you can output the data.
(such as smallest shop)…

Some places to explore PHP & SOAP:

• PHP + SOAP + Amazon - onlamp.com/pub/a/php/2003/07/03/php_amazon_soap.html
• Shane Carvaeo’s SOAP Presentation - talks.php.net/show/soap-phpcon-ny2003
• XML.com’s Web services area: xml.com/webservices/
• http://www.soapware.org
• http://www.soapbuilders.org

Use PHP? Check out PEAR (pear.php.net), which provides libraries
much in the same spirit as CPAN does for Perl. I wrote about
Shane Caraveo’s talk on PEAR in my Wednesday blog entry.
A flip side to this may be how PHP is used at Yahoo. Michael Radwin
of Yahoo gave a talk (also mentioned in my Wednesday entry) where
a key point was: “use abstraction sparingly. Don’t add a bunch of layers, as it can be costly”. It is good to know that PHP has
matured to the point where a wide range of developers can
approach it from different angles. Want ease of getting
started? Go to go-pear.org, follow
the instructions, and then try some Pear packages (like PEAR::DB). Want performance? Use PHP calls
directly, or better yet, write a C/C++ extension using PECL.
Yahoo has written about 70 of them.

Shane also gave a talk on PHP5, with slides available at talks.php.net/show/php5-intro-oscon-2003. I mentioned this on Thursday. There
are lots of improvements in PHP5, and if I had to pick one
that will really help me out, it would be exception handing
(try/catch/throw). I am eager to get away from a myriad
of “if” statements! The “Objects are automatically references”
aspect is also great. SimpleXML and SQLite are also crazy sexy cool.

Productivity

There are two main projects that I will mention that can
do a lot for productivity. Jabber is useful today (but also
falls easily into my notion of trends). Eclipse is
intruiging, and I managed to keep it mostly away
from my PHP writeups.

Jabber

Jabber is so much more than an IM and group chat client.
In the context of productivity, though, I’ll point out the
following:

• Multiple transports - need to talk to someone on AIM, MSN. Yahoo Messenger? Jabber talks to all of them
• Multiple platforms - yes.
• Encrypted? Compressed? - yes, can do
• An application can effectively have YOU on its contact roster, and send notifications when it needs attention (errors, jobs that have finished, status updates, and so on) - this frees you up so that you don’t have to keep watch over a process

Eclipse

Eclipse is a highly extensible IDE for whatever you would like to customize it for. An angle that
interested me was Christopher Judd’s talk “Eclipse PHP for Developers”,
which took a look at how to construct an environment with
debugging, navigation, and other traditional IDE features.
I’m about to try out the TruStudio tools, which sit
on top of Eclipse. It will be great to have a true debugger,
as opposed to watching statements flow into an Apache error log.

Another site for finding eclipse plugins is eclipse-plugins.2y.net/eclipse/index.jsp

Three Trends

Crossing over from Productivity to Trends, I still have
things to say about Jabber. But first, I will mention
once again a tidbit from Tim O’Reilly’s talk: networking enables collaboration.

I can easily remember Usenet days in the 1980’s, and
how software development projects were posted to newsgroups
(shar archives, anyone?) Patches to those projects (as in, Larry Wall’s
Patch program) would be posted to newsgroups as well. The next
leap was the web, where every project seemed to have its
own .org domain. Networking enables collaboration.

Does Jabber have the potential to take this a step
forward? It’s two XML streams per connection, one
in either direction. Apple’s iChat uses Jabber.
File Transfer is there, and the potential to ask
a program for a custom file is there. The potential for a Hydra
clone (for multiuser collaborative editing) is there.
The potential for videoconferencing (perhaps Out Of Band?)
is there. I think we are going to see interesting
applications where Jabber is used for a transport
to enable web services (it’s much more efficient
than HTTP). Keep an eye on jabberstudio.org/

Another big shift revolves around the idea “I like my
custom web site, but I want to use your data”. Amazon
Web Services are one of many examples of this.
Jeff Barr, Amazon Tech Program Manager, stated
a simple goal: Enable discovering and buying from anywhere.
A few examples of it in use:

• store.nba.com
• greeklandscapes.com - (see the travel guides)
• simplest-shop.com/camera - (compare cameras)

For more information on Amazon Web Services: http://amazon.com/webservices

Amazon is just the tip of the Web Services iceberg. Google
is also leading in this area with their offerings: http://www.google.com/apis/
See webservices.oreilly.com/ for a lot more information.

I’ll ignore all things blogging, so far as trends go. As Clay
Shirky has put it, we could have been doing this 8 years
ago with web forms. It just took some easy to use front
ends (like Moveable Type) to get it going. What’s more
interesting to me is the wonderful world of Wiki.

A Wiki that got lots of use before and during OSCON
was Brian “ingy” Ingerson’s Kwiki powered oscon.kwiki.org
Wikis allow for a very diverse group of users
(grannies running www.quiltzilla.com/,
geeks running londongeek.org/) to
evolve a website that can capture, organize, and make searchable
a lot of information. The emphasis is on the information,
more than pretty presentation. Get Kwiki at http://kwiki.org/.
What happens when Wikis start calling web services?
What happens when Wiki links can fire off a jabber session,
possibly leading to the automatic update of other Wiki pages?
I can visualize a lot of intersections, and while it is
true that some of them will be skipped, it is also true
that it is getting easier to glue functionality together
in any of the three P’s: Perl, Python, and PHP. Stay Tuned!

Future Conventions

Note Bene: These are my personal opinions, which may
or may not reflect what the O’Reilly staff is thinking.
I haven’t talked to them about things to do with
video from laptops. I do see it as an issue on
the near horizon that they, and indeed any tech
convention organizer, will need to ponder.

Hydra was used quite a bit at the Emerging Tech Conference,
but was much less in evidence at OSCON. I found this
puzzling, and wrote an entry, “Hydra Missing”, in my personal blog. The gist of it is: we need a
Hydra clone for all platforms, and we need people
to trust the process of collaborative editing. My selfish
interest in it is that I want a more complete set of
notes from a talk (even ones where I am not in the room).
No one person can capture a talk like a group.

The capabilities of iChat AV and iSight should be
considered by conference organizers going forward.
There are a couple of reasons for this: bandwidth,
and the notion of broadcasting a talk (permissions).

Imagine a conference like OSCON, with up to 11 simultaneous
sessions. Now, 2 or 3 people in each session decide
to open up iChat AV or its equivalent, so that the
talk can be seen in other rooms, captured, etc.
That’s about 20-30 decent quality video streams.
What happens to the network?

I suppose one can say “capture it, and make it
available via BitTorrent later”. There’s
IM and IRC, Jabber and Hydra… all mostly text,
and consuming pretty
low bandwidth. Cheap laptop video has the potential
to consume the majority of available convention wireless
bandwidth, and it will start happening within a year.
Heads up, conference planners!

Which leads to the whole permission aspect. There
will be a need for clarity as to whether it’s cool (in increasing
levels of permissiveness) to a) video capture a talk for yourself, b) share
it later, and c) share it live

Did you go to OSCON 2003? What were your impressions? If you did not go, was the coverage helpful?

Related link: http://www.oreillynet.com/oscon2003/

The tone at the

2003 O’Reilly Open Source Convention
was one of business-like confidence. I sensed much less of the
demonstrative posturing that I noticed at earlier conferences, where
many people felt that they had to prove themselves as well as their
projects’ viability. The free software movement has made it by
now. Just look how the trade and mainstream press has covered the
story of the SCO lawsuit. A few years ago, all the buzz would have
asked what the suit meant for IBM, but now everybody’s more concerned
about Linux.

And with this mainstreaming comes responsibility. Developers, system
integrators, and other professionals are acutely conscious of the
millions of people who depend directly on Open Software. The time has
come not for manifestos but for the unflagging work of keeping the
systems at peak operation.

One corollary of this sense, I believe, is a silent acceptance by now
that the world of proprietary software is deeply entwined with Open
Source. This entwinement has been going on from the beginning, whether
one considers the popularity of Cygwin tools or the many free software
bindings for proprietary databases.

Consider SAP, one of the most ensconced providers of heavyweight
software in the industry. Its financial tools are considered
indispensable in large corporations and its consultants get some of
the highest salaries in computing. But SAP realizes its products need to
interact with outside scripts and applications. It further realizes
that free software tools and languages are key to its usability.
Piers Harding has developed Perl, Python, and Ruby bindings to SAP’s
binary API, and along with DJ Adams he presented the interface in a
conference session.

So free software has become a key element in the strategies of large
proprietary corporations. I am not referring to the cynical
exploitation of Linux by SCO (see my early weblog on the issue,

Irresponsible SCO),
but to a deliberate embrace of free software to enhance proprietary
offerings. Oracle supports Linux in order to reduce customers’ total
cost of ownership and free it from dependence on proprietary operating
systems. Meanwhile, SAP partners with MySQL to reduce its
customers’ total cost of ownership and free it from
dependence on Oracle.

Will SAP’s turn come in a decade? And is
Mono
a bid for compatibility or an implementation of a platform that is
inherently superior to non-Microsoft offerings? Miguel de Icaza,
who delivered

one of the last keynotes
today, claims the latter. He puts all his time into Mono because he
believes the .NET approach to components makes software develop faster
and easier, period.

Meanwhile, Open Source opens opportunities. At the

kernel hacker panel
today, attended by six generous Linux developers, it was pointed out
that a student can show potential employers some work done on real
projects that are actually in production, not just toy projects done
for class. Furthermore, the employer can check mail archives and see
whether this student can get along with others while working on a
project.

Some predict that we are in a historic transition to a world of
completely Open Source software (the references to “commoditization of
software” everywhere one turns), but we can’t really be sure that the
hybrid, symbiotic situation we are in will ever end. All we know is
that maturity brings responsibility.

Related link: http://www.osafoundation.org/desktop-linux-overview.pdf

If you’ve ever played the classic arcade game Punch-Out chances are you’ve heard the words, “Body blow” echo in your head when you’ve seen someone take a painful hit.

I sometimes wonder if Microsoft feels like the computerized boxers of this game, defending itself from the ever increasing number of coders who decide they want to take a shot at the champ. Open Source makes the cost of entry low, you don’t even have to pump in a quarter to play.

Here is the commentary from the lastest two minutes of the fight:

• 80,000 Linux desktops deployed in Spain–Body Blow!

• Open Source bills in Oregon and Texas–Body Blow!
• 2 Councils in England, representing 10,000 desktops, considering switch to Linux–Body Blow!
• The city of Munich, third largest in Germany, to switch 14,000 desktops to Open Source solutions–Uppercut!
• Ximian working with various governments in Europe to deploy 250,000 desktop Linux systems–Body Blow!
• Opengroupware.org releases an Exchange replacement–Body Blow!
• Thailand sponsers the sale of 1 million Linux laptops–Uppercut!
• HP to sell a corporate desktop model running Linux–Body Blow!
• OpenOffice.org to release 1.1 with speed improvements, better document conversion, and a macro language–Body Blow! Body Blow! Body Blow!

We may never hear the fight announcer shout out, “Go for the knockout!” There may never be a killer app from the Open Source world that delivers the final punch that puts Microsoft to the mat. It may be that after the final round the judges award a decision victory to a battered and bloody, but still standing, Microsoft. But, even if that were to happen, Open Source would still be the winner in the eyes of the people. A rematch would be in the works before the fighters had finished their showers.

Remember, Rocky was the true winner in the first movie, Rocky, even though the decision went to Apollo. And we all know the rematch did not go Apollo’s way.

Related link: http://conferences.oreillynet.com/pub/w/23/oscon_grid.html#thursday

Thursday was a long time ago. The keynotes have been covered in detail
elsewhere, so I’ll just give a general impression. It was really nice to hear
Mitch Kapor’s enthusiasm for Linux on the Desktop. He also mentioned that his
first brush with free software was in the mid 80s, when Richard Stallman was
protesting outside the Lotus offices. Kapor took a strong stand against
software patents, which endeared him to the audience.

Mark-Jason Dominus is well-known as an entertaining speaker as well as the
capable host of the Perl Lightning Talks. He’s so good at what he does, he was
able to give the conference committee only the title of a talk and
receive 45 minutes to do whatever he wanted. The result was “Nine Views of
Mark-Jason Dominus”, nine lightning talks about cool hacks and good ideas.

One of the most useful talks was “How to Progress”. Dominus gave some of
his secrets of being smart — read the books no one else is reading, take
good notes, and read one sentence and really think about it before you read the
next sentence. The final talk explored a message to aliens, encoded in a radio
signal. It’s really interesting to think about how you would explain human
concepts and science starting from the number system and working up to ideas
such as gravity and pressure.

Andy Lester then presented “Automated Testing of Large Projects with Perl”.
The big secret is how his company handles their smoke tests. In short, they
run the complete smoke test suite automatically every hour. Any failures are
e-mailed to all developers. (There aren’t many failures.) Andy’s learned
quite a few things and most of his experiences and conclusions match mine. It
was nice to see forty or fifty people picking up good habits. Andy suggested
daily, weekly, and monthly goals to improve testing. (Good suggestion one:
read our testing materials! Good suggestion two: everyone writes at least one
test every day! Good suggestion three: keep records on how much testing helps
you.)

Microsoft provided lunch again. Version 2.0 was better than version 1.0.
It’s unfortunate that they won’t have the chance to impress us with version
3.0.

During lunch, I helped James Duncan man the href="http://perl-foundation.org/">Perl Foundation booth. We explained the
goals and principles of the foundation to a few hardy souls wandering the
exhibition booth during lunchtime. If you’re looking for a way to support your
favorite language and community in a volunteer fashion (and possibly a non-code
fashion), there are foundations for other languages.

Next up was an O’Reilly Network meeting, with Daniel Steinberg, Derrick
Story, and Nancy Abila. We discussed href="http://today.java.net/">Java.net and href="http://www.onjava.com/">ONJava, how they complement each other and
how our editorial strategy needs to evolve. Since both sites address different
audiences in different ways, we’re just looking for the right way to handle
things editorially. Enough work, though, there are still talks to attend!

Just about everything Perrin Harkins says impresses me. I rushed to his
“Object-Relational Mapping Tools for Perl” talk. I’ll admit that I don’t ever
want to write a serialization layer again, so I was interested to see his
survey. His conclusion was about what I expected — I really need to look
into Class::DBI. Fair enough.

The next talk was a little disappointing. While David Fetter’s “How
Database Projects Fail” had some good content (it’s always amusing to hear
horror stories of when things went wrong, and it’s usually easy to learn a
lesson or two), he spent a lot of time bad-mouthing MySQL. I understand that
PostgreSQL fans are disappointed that MySQL gets a lot of press for adding
important features that Postgres has had for years, but they’re both fine
projects. Fetter’s information about MySQL features and flaws was a couple of
years out of date in places and just plain wrong in others. Hopefully he’ll
revisit his research and improve his comparisons for next time.

I ducked in for the tail end of the “Perl Certification Panel”. There are
few subjects more contentious in the Perl world. Things were winding down, but
I heard two good points that really changed my mind on the issue. First, Andy
Lester, who manages more than he codes these days, asked the question, “Why
would I need to look at a certification if I’m going to interview prospective
hires from a technical perspective, anyway?” Brian Aker responded that just
having a certification for Perl allows Human Resources people to filter CVs
before dumping a huge pile on a developer or a technical manager.

Andy’s right, in that a company that wants to write good code would be
crazy not to interview people in detail. (Of course, there are good
arguments that the software industry is crazy, and you don’t
have to look very hard to see why I think that.) Brian’s also right, in that
it would be useful to be able to say “Hey, this guy knows a few things while
this guy doesn’t seem to.”

I wonder if there’s a certification that could be built that doesn’t imply
that one person is a great developer, only that she demonstrates a basic
proficiency with the language. You wouldn’t hire her based on just having a
certification, but you could consider her for a more detailed interview.

It’s a thought, and there’s a Perl
Certification Wiki in place now to discuss these ideas. Perhaps we’ll see
more progress along these lines in the near future.

Fotango’s James Duncan (yes, the guy
from lunch) next gave his “One Year, 93,210 Lines of Code, Ten Staff and Some
Lessons Learned” talk. He gave this at YAPC to good reviews. Fotango grew
from a startup with a website thrown together by consultants into an effective
development group with a large product written in Perl that’s spawning
generically usable open source libraries.

The real secret of Fotango’s success is good teamwork. Part of that is
their strict hiring process. Another part is their agility. It’s possible to
build really good, really useful, really elegant large applications in Perl (or
just about any other language). They don’t talk a lot about doing Extreme
Programming, but their development process is definitely agile. Good talk.

The Testing Cabal had a BOF partway through the talk, but as Schwern was
listening to the ever-clever Dave Thomas and I was entranced by the wacky
antics of orange-loving Londoners, I showed up late. Unbeknownst to me, Dale
Dougherty and Steve Mallett were meeting in the same room so we discussed some
of our upcoming publishing plans. There are lots of interesting ways to start
conversations, and we’ll be exploring those in the future.

The DynDNS/ href="http://www.onyxneon.com/">Onyx Neon open bar party came next, so I
was a good little citizen in my free Onyx Neon shirt. My sometime co-worker
Curtis Poe had come up with an interesting idea for training that we might
explore; see the Onyx Neon site for further details.

The rest of the party is a little fuzzy. The TPF auction had the usual
assortment of ridiculous items, including Andy Lester eating pages from his
book, Karen Pauley donating Dominus’ boxer shorts (clean) and then buying them
back (and you should have seen Marty Pauley’s face), and the right to determine
the color of CPAN Search for the next
year (and you really should have seen Leon Brocard’s face when
London.pm LOST!). I wish I had one of those nice wizard hats, though….

As I started fading, I was in a heated discussion with Curtis Poe and Piers
Cawley about the best way to test to make sure that your individual code units,
well tested with programmer tests, fit together. I think we finally convinced
him, but I really wanted my bed at that point.

Then it was midnight and we missed the last train home and my brother and
sister-in-law are absolute saints for driving downtown to pick us up. Maybe
next year they’ll build a hotel within walking distance of my house so I can
cut the commute from an hour to about two minutes. Ah, a man can dream. See
you tomorrow!

There is fountain near the Willamette River. A circle of
high velocity water jets converge in the middle. There are signs
that warn you about the intense water there …

This convention is a little like that. You are in the middle,
the conference tracks surround you, and you get hit with
a firehose of information. There are no warning signs
here - you just learn at some point that the schedule,
with all of its “kid in a candy store” appeal, can clobber
you if you try to take on too much in such a short period of time.

Therefore, I move that next year’s convention be expanded to three weeks!

At the end of the week, I will look back at some other
Thursday sessions (JabberJazz, IngyOnKwiki, and Amazon Web Services).
I was particularly interested in the new capabilities of
PHP5, and will highlight those today.

PHP5

The lone PHP session I attended on Thursday was “Introduction to
PHP5″, presented by Shane Caraveo (Shane at ActiveState.com).
The presentation is available at talks.php.net/show/php5-intro-oscon-2003.

There are tons of improvements in PHP5, such as access (public,
protected, private), unified constructors and destructors, object
dereferencing ( $f->bar()->barabarina(); ), abstract classes,
and interfaces. I will pick a
couple to mention, because there’s too much to write about on a deadline.

PHP5: Exceptions

Hooray! We get try/catch/throw. The Exception class is the base
you extend. See the slide on Page 27 for an example. A nice
thing about this for me is that it is something I liked about
C++, and it gets rid of the need for many “if” statements.

PHP5: Bundled SQLite

SQLite has excellent SQL
standards support. It is an embedded flat file SQL DB (you
pretend it’s MySQL) You can call it via a function or OO
interface. It is good for developing something where you are not
sure if customers will have access to MySQL, etc. A very cool
thing about it is that it can exist as an in-memory db. Having
said all that, my gut tells me that this will be useful for
small projects, but it remains to be seen how it will stack
up against MySQL or other relational DBs for anything big. Still,
it definitely has a place.

PHP5: Stream Servers

Easy servers! Stream servers allow you to open up a socket on a port,
creating a server. Basic example:

$s = stream_socket_server('tcp://localhost:8080');
while ($client = stream_socket_accept($s))
$line = fgets($client)
echo $line;
}


PHP5: Timelines

Beta 1 has been out for about a week. Don’t use it for production.
Shane says a release in 4-5 months is optimistic, and that
his conservative guess is 6-9 months.

A Thank You, and a note

Getting together a conference like this is never easy. I think
the O’Reilly folks have done a great job. Yes, I personally
want a “Power Strip In Every Pot”, but I also note that
most things ran very well. Thanks to chromatic for
giving me a chance to write, and thanks to the people
behind the scenes that figured out enough of the
networking stuff to keep most of us online, most
of the time.

If I can refer to a Slashdotism for a second,
which starts with “In Soviet Russia…”, I’ll leave
off with this point: most people with tech ambitions
in some quite undeveloped parts of the world
have never experienced what we live everyday:
Consider the hotel lobby, full of folks lounging around with
wireless 5 pound devices
that can reach out to any Internet spot on the
planet. That is a powerful capability. Having
lived long enough to remember the very first
Compaq “luggables”, I marvel, and am thankful
for the progress.

Related link: http://www.oreillynet.com/oscon2003/

Total registration has reached 1,650 at the
O’Reilly Open Source Convention,
and it showed in the overflow crowds spilling out from some of the
sessions.

By the way, six of the kernel hackers I mentioned in
Monday’s weblog
will be here for a panel at 10:30 tomorrow.

I can’t say much about this morning’s keynote speeches, even
though I heard them twice. Relying more on the slides than the jittery
sound system, I can report that the
keynote
by Stormy Peters of Hewlett-Packard offered a no-nonsense guide to
bringing free software into a large business based on her company’s
extensive experience. Companies are used to monitoring the spread and
use of proprietary applications, but she recommended that they also
provide policies and controls about what Open Source software can be
run and who is allowed to run it.

Industry leader Mitch Kapor offered in this
keynote
a bit of a personal odyssey concerning Open Source, a bit more of a
social history, and something of the role played by his
Open Source Applications Foundation.
He put up a rather dismal report card
(which a
report
expands upon)
concerning Linux’s readiness in various areas ranging from peripheral
support to applications. His talk got down to quite practical levels
as well, even to the question of font compatibility. He predicted that
Linux would start appearing in call centers and other restricted work
environments in 2004, but no on standard consumer or office desktops
until at least 2007. Deployment outside the U.S. has progressed much
farther than within. I guess Microsoft employees still have plenty of
time to cash in their new company stock.

My coverage of the conference sessions is also sparser today than href="http://oreillynet.com/pub/wlg/3472">yesterday. I had
meetings to attend, and because I stayed out late with some O’Reilly
authors last night I took off a bit of time this afternoon to get a
horizontal perspective on things.

In
OpenSSI (Single System Image) Linux Cluster Project,
Bruce Walker of Hewlett Packard laid out to some 35 attendees the
clever and ambitious steps with which this project solves the problems
of transparent, self-managing clustering. In a brief hour he uncovered
some of the magic behind processes hat migrated between nodes in a
self-balancing manner that depended on self-repairing networks.

Take for instance the carefully planned way in which processes can
continue using interprocess communication mechanisms (semaphores,
pipes, and so forth) even when bouncing from one system to another. A
single IPC nameserver keeps track of all such IPC mechanisms
throughout the cluster. If it goes down, a new node is chosen and
built up dynamically with the same information. To send something
through a pipe to a process, one sends it to the nameserver. (There
are unified namespaces for processes, devices, files, and so on.)

Emerging Open Source Business Strategies,
moderated by Tim O’Reilly in the last session of the day, rounded out
the theme with which he opened the conference in yesterday’s
keynote. He brought in managers from three companies that somehow
market Open Source software to discuss motels for making money. Ian
Murdock of Progeny pointed out that commoditization is not necessarily
bad for business: “You can build a great business around a commodity,
but there are only a few slots there.” But an attendee told me later
that the casual application of the term “commodity” is misleading. A
commodity is characterized by very low marginal manufacturing
cost. But there is still some manufacturing cost, and that
provides the profit businesses can fight over. This marginal cost is
totally absent in software and information, so they’re a whole new
ballgame.

I ended the day with a trip to Powell’s Books (a store I mentioned in
my
first weblog from this conference),
where I met some of the wonderful people who staff O’Reilly’s
competition, a party, and an auction to benefit the Perl Foundation.
The Perl community knows how to have fun. Among the items auctioned
were a promise by Guido van Rossum to implement conditional
expressions in Python, and a “Twenty-Five Years of Animal Magnetism”
O’Reilly mug that the audience voted instead to throw against the wall
in a re-enactment of the stunt Jon Orwant employed a few years ago to
indicate his insistence on need for the Perl 6 project.

Come back tomorrow for my wrap-up and observations on the conference
as a whole.

Related link: http://conferences.oreillynet.com/pub/w/23/wednesday.html

Conferences need naptimes.

Tim O’Reilly started the day with a keynote that explored the relationship
between open source and web services. We spoke later that night, at the
author’s signing, about the curious gift some people have for developing new
abstractions. Once you realize that two existing projects or ideas can be put
together in a new way, you wonder why you never noticed it before. I’ll talk
more about that in a moment.

Paul Buck followed with a demonstration of IBM’s Eclipse IDE. What struck
me is that his short demo looked exactly the same whether stepping through Java
or C++ code. It sounds like the future of Eclipse development will be in
plugins. I’m reminded of Mozilla… maybe there’ll be a framework unification
sometime.

Next I escorted Sun’s Danese Cooper to the Speaker’s Lounge. She’s been
back and forth to Europe several times since I last saw her in Santa Clara a
few months ago — the EC is making decisions about open source adoption,
apparently. She’ll probably have more to say on this. href="http://www.dibona.com/">Chris DiBona, ex-VA and Slashdot leader, was
upstairs, and we chatted about his new game company. The game industry has
always seemed to be five years behind the times for what works in software
development, but it sounds like he’s got his head on straight. I’m hopeful
that his technical and business acumen will lead to a successful company.

Ward Cunningham and Brian Ingerson demonstrated href="http://fit.c2.com/">Fit and href="http://search.cpan.org/search?query=Test::Fit">Test::Fit before a
packed room. Fit is a framework for customer testing — just define a set
of inputs and expected outputs in a simple HTML table and link to a Fit runner
that will run the tests. If you make this available, for example, on a Wiki,
it’s very easy for customers to start writing tests.

It was interesting to watch Mark-Jason Dominus, who arrived a couple of
minutes late. He’s very clever, and the wheels in his head are always turning.
When Ingy explained that you could use Fit tests as baselines for the behavior
of any language, just linking to any other website that actually contained the
Fit runners, Dominus’ hand shot up. “How does that work?”

The Fit runner just looks at the HTTP Referer variable to determine the
calling page (the one that defines the tests), grabs the HTML, parses the
tables, and returns a slightly modified HTML page with the test results.
That’s the kind of simplicity and interconnection Ward’s really good at.
That’s the kind of thing Tim’s thinking about when he talks about data, not
code, reuse.

Brian Aker, fresh on his new job as Senior Architect at MySQL, shocked the
world (or, at least, me) when he announced that he’d embedded Perl in MySQL and
was using it for stored procedures a couple of years ago. Of course, it did
segfault rather often. Fortunately, it’s highly mature now. In his talk on
“Making MySQL Do More”, Brian showed the embedded function API. You can write
new functions for MySQL in Perl, Python, PHP, and Java. (Keep asking him about
Ruby.) You can link to C libraries; he’s used Image Magick and zlib. I’m
excited about how easily you can modify queries — SELECT DIFF(foo)
... anyone?

Lunch was decent. My brother met Dominus for the first time and, I think,
was a little intimidated.

The Perl Lightning talks are always worth watching. Though a few were
repeats from YAPC, it’s always nice to see such a wide spectrum of interests
and ideas all crammed into short, sweet talks. Andy Lester spoke on getting a
job, though he refused to tell me how to give up a job when you have too many.
Allison Randal sang “Allison’s Restaurant” again, about Perl 6 Development.
Autrijus Tang brought down the house by rapping, in Chinese, about 1% of his
favorite CPAN modules. (There’s no truth to the rumor that he learned Chinese
in two hours just for the talk. He’s that smart, though.)

Phil Tomson, Portland native, introduced SWIG. I had hoped for more
details about how it worked, but he gave a good overview and I learned a few
things. Phil’s loves are electronics (chip design), Ruby, and SWIG, and he’d
put them to use on a recent job on a large integrated circuit manufacturer in
the Beaverton area. Chipzilla? My last serious attempt to use SWIG was
several years ago; it’s picked up a lot of momentum and polish since. In
particular, I’m impressed that you can override and seamlessly go back and
forth between Ruby and C++ objects — that’s a goal of Parrot.

Next, Greg Stein packed dozens of people into a much-too-small room to
discuss Subversion. Apparently CVS has quite a few warts. (Yes, that’s irony.
I’ve used Subversion very happily for almost a year. I was convinced the first
time I used svn move.) Lots of interesting features are planned,
though lots of interesting features are scheduled for “post 1.0″. It’s nice to
see a project with well-defined goals that sticks to those goals. If you want
to see how well an open source project can be organized and run, you’d do well
to examine Subversion.

The Author signing was next. Brian Aker showed up partway through and it
was very interesting to sign books with a co-author present. He lives in
Seattle but we never seem to show up in the same place at the same time. Oh,
and we gave away lots of mugs and lots of t-shirts. Surprisingly, we were all
out of calendars. Finally.

After the signing, I had a chance to speak to David Axmark from MySQL.
He’s as good natured and smiley in person as he was in his talk. He has lots
of (good-natured) plans about taking over the world, though he’s not sure the M
in LAMP can stand for any other open source databases. (Hey, I’m generous. I
don’t even call it MostgreSQL!) We also talked slightly about the challenges
of working with remote developers. MySQL has several Russian programmers,
including one in Siberia. He has to travel as far through Russia as he would
in Europe just to get to Sweden.

Karen Pauley of Kasei came downstairs
at that point and had a conversation with Allison Randal and myself. Karen and
Tony Bowden had reviewed href="http://www.oreilly.com/catalog/extprogpg/">my book. We had a nice
talk about dealing with customers and some of her programmers. I’m not sure
that insulating developers from customers is the right decision; sometimes a
little conversation can make everyone more productive.

Randal Schwartz threw a party to celebrate the release of his href="http://www.oreilly.com/catalog/lrnperlorm/">Alpaca book. As usual,
Stonehenge Consulting rented a bar, invited everyone in the area, and let
loose. I’m not sure anyone remembers the party.

Nat Torkington also had a few people over to the presidential suite.
There’s a great view of the Willamette River and several bridges. (I’d not
seen OMSI’s submarine since OMSI moved from by the zoo to down on the river.)
Lots of Perl people were there and I spent more time talking than I intended.
Good fun, but I got home way too late and got up way too late. Again, naptimes
would be wonderful.

I’m beginning this entry in a full-up Rendezvous area, where a plethora
of power strips and decent wireless coverage happen to be in
the same place. My thought for the day, outside of all things
Open Source, is that we need wireless power. Sure, it might
be dangerous, but I’ve seen a couple of tripping over power cord
incidents that could have resulted in some serious amounts
of embarassment, and perhaps a hangnail. On a more serious
note, I really look forward to the day when laptops have
the battery life of (at least) a mobile phone, are quieter
than my cats in some tuna-induced stupor, and do not
double as portable heaters.

There are a lot of PHP sessions today, plus two keynotes (Tim O’Reilly
going over Open Source paradigm shifts, and Paul Buck of IBM
talking about Eclipse (an IDE on steroids). I’ll come back
to Eclipse tomorrow, but will say I have a good first impression
of it.

A quick note/reminder for some: a lot of the talks are available off of the OnlineSlides page of the Wiki

Trends

Robert Kaye has a nice take on Tim’s speech in his weblog. I won’t repeat that here, but will instead share a couple of thoughts on the notion of “adhocracy”.

Tim notes the term “adhocracy” from Cory Doctorow’s book “Down and Out in the Magic Kingdom”, defined as:



“Like-minded developers can find each other and work in ever-shifting groups”

He thinks of Usenet as “the mother of Open Source”. Another big
influence has been a program by Larry Wall. Nope, not Perl, but Patch.

It used to be pretty common to download a multipart shell archive
of some package from a Usenet newsgroup, and to keep it up
to date by downloading patches from yet another newsgroup (and
applying them to the source with Patch). Even with all of this
clunkyness, sometimes having to wait for a daily UUCP connection
to get the latest newsfeed over the “company modem”, there was
still a sort of thrill to see the new stuff in certain newsgroups.
We knew others were out there, somewhere, on the other end of
their modems…

Getting back to Tim’s keynote: the users started to talk to each
other over Usenet and mailing lists, and started to help build the
apps. Major apps commonly have key contributers from different
countries. Tim made the point: “networking is a key driver for
collaboration”.

I think of the leaps made from the late 80’s till
now, where WiFi is really starting to take hold, and wonder how that
will affect things. The mechanism for the collaborative process just
keeps speeding up, and has become ubiqitious in some places (how
will it change things when high speed access is available
everywhere?)

PHP

I’ve been listening to how PHP can talk to Java and COM, how
to do web services in PHP, what Yahoo does with PHP, and
an intro to PEAR. It’s easy after a few hours to enter
a sort of PHP Reality Distortion Field. Fact is though,
you Really Can do a lot with it!

PHP: Java/Com

John Coggeshall spoke about how to use the Java and COM
extensions to PHP to get at those environments from within
a PHP script. A large caveat, especially in the case of
Java, is that while this is a handy thing to do, it is
more of a “use an existing component” idea than “we
are doing this because we need blazing speed”.

The ext/java extension allows PHP developers to get at
properties of Java classes. One example would be
to use a Java class to create a .zip file. A couple
of links:

• Java in PHP
• John Coggeshall’s site

PHP: Web Services

Adam Trachtenberg’s talk on Web Services in PHP was a less theoretical version of Sterling’s versionon Tuesday (it was good for me, as a newbie to this, to hear some things
again). Adam’s definition of a web service is:

“A web service is a network access interface to an application,
built with XML, and usually HTTP”.

…or another way to think of it: Standardize the way you get data back and forth.

• Shane’s SOAP talk
• XML.Com Webservices area
• Getting data from Amazon via PHP and SOAP

PHP: The Yahoo Performance Angle

Michael Radwin presented “One Year of PHP at Yahoo!” Yahoo has
been using PHP for a year, and in no small fashion. They get
232 million users per month, have 112 million active registered
users, and get 1.9 billion hits a day. They must trust PHP!

Michael had ideas in several areas as to how to use PHP on a very
large site. I will focus on the performance aspects:

• scaling: Install an accelerator. With no code mods, you can get a dramatic increase in performance by having parsed pages cached in memory.
• profiling: Write for functionality first. Use APD (”gprof for PHP”). Use faster PHP calls, such as strncmp instead of substr.
• include files: try to minimize include files. Each one
  results in several system calls (on the other hand, using an
  accelerator can reduce the number of calls to one. Balance
  performance and reusability. Keep your include_path
  short.
• abstraction: use abstraction sparingly. Don’t add a bunch
  layers, as it can be costly. OO is slow in PHP4, faster in PHP5.
  Pear::DB, in particular, is aimed at a sort of lowest common denominator.
  It adds a layer that gets hit with every call to your database. (remember,
  I am just writing about Yahoo and their quest to maximize performance,
  so do not send me comments suggesting I am an anti-OO heretic)
• C/C++: Yahoo has about 70 extensions in C/C++ internally.
  TRe code is highly specific for their own use, so they won’t be
  releasing it as Open Source. It’s a big win for speed, and
  interfacing with legacy systems. The downsides are all of the
  usual (my quote) “why you would use scripting instead of
  C/C++” concerns, such as memory management, speed of prototyping, etc.

PHP: Pear, PECL, and Open Licensing

Shane Caraveo of ActiveState
talked about the PHP Extension and Application Repository (PEAR). PEAR provides an OO class system for PHP packages,
and coding and versioning standards. It makes it easy to
install and update packages. It has a large developer community,
and the licensing is very open (a goal of PEAR is to make
it easy to use packages in the commercial environment).

What PEAR is to PHP classes, PECL is to C/C++ extensions.
Current PHP extensions that are part of the PHP4 distribution
are being moved to PECL. The sheer number of them has
complicated the process of getting PHP releases out (getting
all of the extensions synced up), so off they go to their
own package management universe.

Another acronym to mention is PFC. PFC is the PHP Foundation Classes.
They are held to a higher standard of QA and API design than
other PEAR classes. They are meant to be a core subset
that are used to bootstrap other classes.

Having a large collection of PHP classes is great, but what is
it like to install them on your system? The PEAR installer
has multiple UIs (CLI, Gtk, and a web front-end). It is
easy to do an inventory of clasaes you have, to grab
new packages, to update everything, and to uninstall them.
Of particular interest is a script you can get just by visiting go-pear.org. Go to the site, save the script, and
run php on it. It automates the process of installing PEAR.

To get started on the PEAR web installer, run from a command line:


% pear install PEAR_Frontend_Web

Related link: http://www.oreillynet.com/oscon2003/

I experienced déjà vu twice during Tim
O’Reilly’s
keynote
this morning. I remembered that many years ago he was asked, along
with a number of other leaders in computing, whether Unix had produced
a “killer app.” Tim replied with aplomb that the Internet was Unix’s
killer app, but most people reacted with shrugs or giggles. I think
this was an earlier stage of the same paradigm shift Tim is talking
about these days.

At this point, the big “applications” changing people’s lives are
services such as Amazon.com, Google, and Yahoo. These are data-heavy;
the way they work is less important than the data.they offer. Because
of the centrality of the data, they require constant updating, which
in turn requires people. The application does not stand on its own.

I will leave a summary of Tim’s keynote to others, but I must spend a
moment on his dismissive attitude toward open source licenses. Tim
built his case carefully, pointing out that Usenet created the
original far-flung community of collaborating programmers, followed on
by the Internet. Infrastructure, therefore did more to create an
environment conducive to free software development than licenses. In
fact, he said, Larry Wall’s patch program did as much as any license.

Tim wrapped up his argument by pointing out that ASP.NET grew within
Microsoft through a process very much like Open Source development,
and summarized, “You put networking in place and you put developers
together, and you get open-source-like behavior, regardless of the
licenses in effect.”

Well, that was my second déjà vu.
Counterposing technological environment versus licenses recalled Larry
Lessig’s two types of code. The code developed by programmers creates
or restricts opportunities as much as legal code–but legal code is
not obsolete. For instance, digital rights management and the DMCA
mutually support each other.

And free software has not become quite popular enough for us to know
what the effects of different licenses may be. I imagine that a lot of
companies are salivating over Linux. In the absence of the GPL, what
protects it from being embraced and extended? Branding would be a
strong factor predisposing users to choose “real Linux” (branding
protects Apache somewhat from being exploited, even though it has a
loose BSD-style license) but we can’t be sure it’s enough, especially
with the pressures in embedded system space.

Tim’s talk was the start of another long day. With the start of the
keynotes and sessions, OSCon was in full force. Noise and bustle from
the loaded exhibit hall spilled out, and streams of people moved in
every direction. The excitement was contagious. Some other events I
attended included:

• The
  
  An Open Source Tool Framework for the Enterprise
  keynote by Paul Buck of IBM, which was a history and promotional talk
  on Eclipse. One strength of Eclipse, which has made Open Source
  history by emerging as an immensely popular IDE, is that it was
  designed to have a small core and offer most features as
  extensions. This provides a framework that the community can easily
  plug into to produce more extensions.

• 
  Applied Ant
  by Erik Hatcher. He showed the elegant integration between Ant and
  several other environments, such as:

  • Code generation for EJB.
  • XML validation (which permits one to identify invalid XML before
    running the application), XSL transforms, and the retrieval of text
    and attributes from XML files.
  • WSDL integration using Axis.

• 
  A Guided Tour of the MySQL Source Code
  by Zak Greant and Monty Widenius. Mostly a discussion of how to find
  one’s way around the directories and files of code, a subtext of this
  presentation could be titled “Reasonable ways to code a large
  program.” Among the very balanced decisions made by the team were:

  • Using C++ for server, but avoiding heavyweight, non-C features such as
    exceptions and the STL.
  • Using virtual layers so that all platform-specific code is hidden in
    low layers. This removes the need for ugly and hard-to-maintain ifdefs.
  • Use a robust source control system. MySQL AB, like the Linux
    developers, uses Bitkeeper, which Zak said is much easier than CVS
    when large numbers of files change at once.
  • Keep an ever-evolving internals document.

  I didn’t stay for the whole talk, but I was amazed that after all
  these years each large project is still creating its own string
  functions.

• 
  Microsoft Office Files Exposed
  by Simon St.Laurent. The upcoming release of MS Office will use XML as
  its storage format, but it’s a rough “1.0 release.” Probably none of
  the important details will remain the same by the time Microsoft
  settles down. (And, according to Simon, they figure out what to use it
  for. I suppose allowing free software programmers to extract data from
  Office is not a Microsoft marketing goal.)

• 
  How to use XML Security Standards in Real World
  by Aleksey Sanin. He explained why the standards are useful (for
  instance, SSL does not let you encrypt arbitrary portions of documents
  or attach multiple signatures from different people), introduced the
  arcane XML Canonicalization, Digital Signature, and Encryption specs,
  summarized some toolkits with actual code examples, and warned about
  basic precautions to take. Surprisingly, many implementations don’t do
  such basic things as checking whether the person is signing something
  reasonable or that the right person is offering the right signature
  and is the right source for a key.

Related link: http://www.oreillynet.com/pub/h/360

Google’s international front ends are an example of something I like to call ‘language skins’. They are quite easy to change (see this Hack for help when you’re away and don’t know the language Google is talking to you in) and make your life easier when you are on the move in a country where the local language is one you don’t know.

Every desktop application’s window ought to have the ability to change language skins seamlessly and without rebooting the whole system or restarting applications. Changing window and menu language, or keyboard layout for each window and dialog ought to be only a single mouse click away.

It will not only make software localization easier and faster, but will make book publisher’s and user’s life easier. The classic problem with today’s software and computer books is their localization. Suppose a German-speaking professional is assigned to a Moscow office, where all versions of Word are localized for the Russian user. If he has a problem with some options, his German Word manual will be useless, because the messages he sees in print and on screen don’t match; similarly, he won’t get much help from a Russian Word manual if he doesn’t speak Russian. With language skins he could change the descriptions of windows and menus to German and keep on working in the way he is used to.

Related link: http://conferences.oreillynet.com/pub/w/23/tuesday.html

Tuesday was overcast and cool, though the west looked much clearer. I again rode the train in to town and hiked a few blocks to the hotel. Fortunately, I was on the train earlier and was able to grab a seat to write Monday’s weblog.

The Speaker’s Lounge was filled with people taking advantage of wired connections and a critical mass of people. The Fotango guys were all wearing hand-made Ponie shirts, though very few people actually knew what Ponie was then. (The Perl community knew Larry was announcing something, but more on that later.)

I sneaked in to Pragmatic Dave Thomas’s “Ruby in a Day” talk. Ward Cunningham was there, writing three Ruby programs for every code snippet Dave demonstrated. I was highly impressed; though Ruby is a consistent and easy-to-learn language, Dave was speeding through things. In an hour, he’d gone from “Hello, World” with arrays to iterators, blocks, and advanced block use.

Lunch came next, so it was back to the Speaker’s Lounge to talk again to Damian Conway and Allison Randal. I swept through the Exhibitor’s Hall as it was being setup. The Perl Foundation has a booth, as does the O’Reilly Network (and I don’t have to be there). The hall opens today (Wednesday), so go down, see all of the booths (I saw a couple of huge model rockets being wheeled in), and grab some swag.

The full-day tutorials (”Advanced OOP”, by Damian Conway; “A Day of Extreme Programming”, with Marty and Karen Pauley; “Ruby in a Day”, by Dave Thomas). That afternoon, though, the Perl 5 Porters meeting was moved up to 3:30 pm, so I ducked out early and played Zendo with gamemaster Tom Phoenix for a little while.

The big news in the p5p meeting was the upcoming Ponie announcement. We discussed the idea for a little while and then discussed the public reaction. It’s important to note that Ponie is not a complete rewrite of Perl 5. It’s a port of as much of the Perl 5 internals as possible.

The really important bit is making Perl 5 use Parrot’s PMCs. Perl 5’s fundamental internal data structures are SVs (scalars), AVs (arrays), HVs (hashes), CVs (code/subroutines), GVs (globs), and so on. They’re each defined in a header and a code file, as in sv.h and sv.c. These data structures are very C-centric; an SV is basically a struct. Given an SV, you can call functions defined in sv.c, such as sv_upgrade().

Parrot’s PMCs are similar in a way but different in a way. It’s easier to think of them in terms of actual objects, from an object-oriented standpoint. Each PMC holds a vtable of function pointers that can be thought of as methods. Instead of calling sv_do_something( sv ), you’d say something more like pmc->do_something().

The really nice part of this is that it removes a lot of special cases, especially switch statements within the functions themselves. If you need to do something a little different, just replace the function in the PMC’s vtable. This gets rid of the need for Perl 5’s “magic” that enables things such as tie(). Yes, PMCs are subclassible and inheritable.

I’m glossing over a lot of details here, but the important part is that by “simply” replacing SVs, AVs, HVs and their friends with the appropriate PMCs (which have mostly already been written), Ponie will reduce a lot of unnecessary complexity while reusing the important complexity — Perl’s existing and working parser and lexer. (It was either Kurt Starsinic or Graham Barr who pointed out that that’s at least half of the complexity of Perl 5.)

PMCs are also tremendously faster, as is Parrot’s opcode dispatch — the basic code that handles the underlying Parrot operations. Because Parrot has a blazingly fast JIT compiler (that turns platform-independent opcodes into very fast, platform-specific machine code), most operations should be very quick.

The other really nifty part of the Ponie announcement is the plan to write an XS compatibility layer for Ponie. Several of the most important and complex Perl modules, including mod_perl and the DBI, have C and C++ components. XS is used as an interface between C and Perl. Because the internals of Ponie will be slightly different, there needs to be a way to do the same things with the new internals without having to rewrite lots of custom XS code. Since XS is pre-processed, this isn’t impossible.

Perl 5.8.0 has a tremendous test suite that’s only improved for 5.8.1 and the upcoming 5.10.0. It’s written at the Perl level, so Ponie can use these tests to judge their progress. It’s likely that Ponie will find bugs in Perl 5 and in Parrot that can be fixed for all three projects.

As the meeting wound down, Ziggy and Ask started filming their TPC video. Expect a lot of strange revelations from noted Perl hackers.

Tuesday night featured the First Annual States of the Unions addresses. Daniel Steinberg has written a nice feature about them for the OSCON coverage page — rather than duplicate what he wrote, just go read that.

ActiveState also hosted their annual party at a nice bar on the waterfront. I ducked out early to try to get to bed before midnight. Hey, it almost happened. See you tomorrow!

I gotta tip my hat to Andy Oram, who did a heck of
a job today with his coverage
of OSCON Great job Andy! It is not easy to go to
session after session, keep a laptop going when power strips
are few and far between, and to be able to recall everything
separately after a long day. If you have ever bought a home,
it’s a bit like seeing 10 houses in a day…

After Monday’s writeup of the Advanced PHP talk, I was
exhausted. Tomorrow (Wednesday) is very wall-to-wall with
talks, so I will add a few tidbits on top of what Andy
mentioned from the “Managing Web Services with PHP” session,
presented by Sterling Hughes.

PHP/SOAP Links

Sterling’s talk on web services can be
found at: talks.php.net/show/webservices. He’s a very enthusiastic speaker about things that he
really digs. SimpleXML? “best thing since sliced bread!”
If he doesn’t like something, you’ll hear that too. DOM?
“DOM is bloat” (because it requires a lot more object creation
than SimpleXML (which is “sexy efficient fast cool”).

A couple of links that Sterling mentioned for XML-RPC
and SOAP:

• xmlrpc-epi.sourceforge.net
• intertwingly.net - Sam Ruby’s site has some good SOAP info

Related link: http://www.oreillynet.com/oscon2003/

The second day of the

2003 O’Reilly Open Source Convention
featured a couple dozen tutorials and no less than six keynotes on the
progress of major free software projects: Perl, Python, PHP, MySQL,
Apache, and Linux. I ran down two batteries on my laptop before the
end of the day, and the number of attendees present slid downward
inexorably as well.

I’ll say a bit of each project in this weblog, along with summaries of
four tutorials that I attended in part today and a note about Apple at
the end.

Tutorials

Here are some impressions I got during a long day of running from one
tutorial to another.

Who Watches the Watchers: Developing “Translucent” Applications that
Protect Sensitive Data

Delivered by MySQL AB employee Zak Greant, this talk was about
security rather than databases, and as much about policy as
technology. It attracted about 35 attendees.

Zak began with an observation that has been popping up more and more
in recent years: that the major source of sensitive data leaks comes
from trusted people within the system, and that one therefore can not
guard against leaks just using traditional authentication checks
within applications or operating systems.

The solutions he presented included:

Reducing the amount of information stored.

A common example is to verify passwords by storing hashes that can
be used to validate passwords, but not used to reconstruct them.

Break data into chunks that have little value on their own.

For instance, instead of storing all the information about a person in
tables of one database, encrypt it with different keys and store it in
different databases.

Send partners only the data they need.

Reduce the chance that organizations will leak data because their
goals are different from yours, or simply because they don’t
appreciate how sensitive it is.

Zak went on to talk about the process for assessing and tightening up
areas of vulnerability.

Building Data Warehouses with MySQL

This talk was a bit more about technology and less about process than
the previous talk on “translucent” applications, but still more about
modeling than coding. I have to admit being somewhat out of my depth
here. The terminology and concerns of data warehousing (assuming that
presenter John Ashenfelter was reflecting standard practice) are
notably different from what I’m used to hearing about in databases. In
addition, Ashenfelter focused on such practical issues as extracting
data from one database to another, cutting down on dump time, and
checking for corruption, The talk was sold out quite early, with 50
attendees.

Managing Web Services with PHP

Sterling Hughes, whom I heard presenting advanced PHP topics
yesterday, graciously and expertly stepped in at the last minute to
take over this tutorial from a presenter who fell ill.

Sterling’s enthusiasm today was driven by two extensions he wrote:
cURL for retrieving URLs, and SimpleXML for parsing XML. Faster for
PHP than a SAX or DOM parser, SimpleXML puts the whole XML file into a
hash that you can access with hash dereferences.

For communications, Sterling recommended XML-RPC in preference to
SOAP, because SOAP support in PHP is not yet mature and XML-RPC is
also more efficient.

Tricks of the [Perl] Wizards

How can I possibly describe this presentation by Mark-Jason Dominus,
one of the most accomplished of Perl’s developers and trainers? As he
has done for many years since the Open Source conference started as
the Perl conference, Dominus grabbed the audience by the ears and
whirled us through a virtuoso display of code that takes obscure Perl
features to their limits, or pretty close. It’s no use trying to
explain the tricks he showed–I was present for discussions of tied
variables, filters, and autoloading–but the effect was like that of a
circus where an impossibly high pyramid of acrobats keeps growing as
one after another keeps emerging from the bleachers and jumping on
top.

I did wonder about the utility of these techniques (which Dominus
occasionally doubted himself), and this certainly is not a talk to be
attended by Perl’s detractors, who already have enough fodder for
their claims that it is tortuous and inscrutable. I have a feeling
that few among the large audience for the talk (about 75 or 80 people)
would use such techniques in their own code. To some extent, the
example code helps to give users a mental model of some of Perl’s
basic features. It also helps explain how the minority of developers
who really master these techniques can produce some of the
jaw-dropping extensions that do prove useful.

Keynotes

The long day was followed by quite a long evening, where we heard six
keynotes interspersed with White Camel awards (for Perl), the Frank
Willison award (for Python), Active awards (for several languages),
and pictures of developers’ children or favorite mascots.

Perl

Larry Wall delivered the seventh of his notorious State of the Onion
talks. Like his others, in my view, this keynote was ultimately about
the miracle of Perl’s very existence. He was not shy about pointing
out that he and other developers are making major sacrifices to
continue with the project. He also entertained us with what he called
a universal architecture diagram, which was particularly ironic
because he also invoked the concept of post-modernism, which (as I
understand it) claims that nothing can be universal.

As for hard content, I will leave that up to many other bloggers and
journalists to report.

Python

Inventor Guido van Rossum presented the state of Python. He said
downloads have doubled in two years. I noticed that Python is mature
enough to have backward compatibility issues (an unwillingness to
eliminate old features when offering better replacements). He finished
by announcing that, “Extreme Programming works extremely well with
Python.”

PHP

Shane Caraveo estimated that PHP use has grown 30% in past year, and
cited Netcraft as reporting that it is used on 31% of all domains.
PEAR 1.0 has over 100 modules and extensions. Currently the team is
working on improved OS X support and improved object orientation
through formal enhancements such as exceptions, constructors, and
destructors.

MySQL

David Axmark, with some additions by Monty Widenius, summarized the
history and current projects of MySQL AB. Several practical
extensions to SQL include one that makes it easier for Web servers to
show subsets of results as one commonly sees in Web searches. GIS
support has been added. The company is growing quite a bit and
benefiting from investments from SAP and elsewhere

Axmark won two rounds of applause for announcing that MySQL AB is
officially against software patents.

Apache

Greg Stein introduced the structure of the Apache Foundation and
offered a brief history. The foundation’s goal is to allow innovators
to do what they want–not to impose deadline and market pressures–but
to provide a long-term framework and community that ensures someone
will pick up a task when earlier contributors leave. Apache has
spawned innumerable projects and is generating enormous amounts of new
code.

Linux

This presentation by Ted Ts’o–his first attendance at an
OScon–focused heavily on the technical. He pointed out at the end
that the most exciting work on Linux is happening in user space, where
languages and desktop advances are increasing its usefulness. Some of
the details in the 2.5 and 2.6 kernels include:

• Major thread-related improvements that are particularly good news for
  Java users, including fast userspace mutexes (a kind of locking
  available to applications).
• IPSEC support.
• Support for uClinux, which allows embedded system development on
  low-cost chips that lack memory management units.
• Easier and more intuitive kernel builds.
• Support for bigger filesystems (16 terabytes on 32-bit systems and 8
  exabytes–count ‘em!– on 64-bit systems).
• The sysfs filesystem, which will finally remove ancient Unix
  limits on device numbers.

Many 2.5/2.6 changes were put to the test recently when Linus Torvalds
announced his move to the Open Source Development Lab and the OSDL
site was slashdotted. Thanks to improved efficiency in the kernel
(especially the virtual memory management system), the system running
their Web site stayed up through the onslaught.

A correction on
Sunday’s weblog

One of the flightier paragraphs in that article, as someone pointed
out to me, could be seen as disparaging toward Apple. Now that my jet
lag has been replaced with conventional conference-engendered
exhaustion, I realize that the criticism is accurate. But read
carefully, the posting should be seen as not disparaging at all. My
concern was to recognize the opinions of free software advocates who
oppose the intrusion of any proprietary software into the free
software space. This is a hotly ranging debate (whose importance to a
large set of software developers is often underestimated) and requires
some mention.

But there is no doubt that Apple computers play a critical role at
this conference. In fact, it’s loaded with Apple lovers. I haven’t
tried to do an official count, but the silver laptops are gleaming out
all over the place.

Related link: http://conferences.oreillynet.com/pub/w/23/tuesday.html

I arrived too late on Monday to go to any of the OSCon tutorials, so I was eagerly looking forward to Tuesday’s events. In fact, I ended up getting a little carried away and going to four different tutorials.

I started the day listening to Zak Greant talk on creating “Translucent” databases. Zak works for MySQL, AB and he gave a great talk touching on a variety of issues from encryption to security. (It doesn’t matter how secure your network is if somebody steals your backup tapes.) However, despite enjoying the first half and Zak’s Canadian accent, I felt the call of Mark-Jason Dominus.

So, at the break, I switched over to Dominus’s tutorial on “Regular Expression Mastery.” He covered some nice material on Perl and Regexes, particularly how simple modifications can drastically improve performance and how Perl optimizes a set of special cases. Additionally, he outlined the Perl 6 plan to drastically change regex syntax to make it cleaner and more understandable. I don’t yet know my opinion on this. On one hand, all the new weird (?= and (?: options are confusing as hell to understand. On the other hand, changing a 30-year-old syntax isn’t necessarily a great thing to do, IMHO. (As one example, character classes will no longer use []; instead, [] will be changed to act as non-capturing parenthesis and <[]> is used for character classes.)

After running out to Chinatown with Dave for a quick dim sum lunch, I made it back in time to hear Sterling Hughes give his presentation on Managing Web Services with PHP. Sterling did a nice job, especially for someone who jumped in at the last minute because the original presenter got the chicken pox! (That’s the problem with having second graders give talks.) I took down as much as I could because I’m giving the PHP: SOAP vs. REST talk tomorrow afternoon, so I needed to have something to talk about. Sterling’s talk had a whole bunch of theory, all of which I’m going to nicely ignore during my 45 minutes.

Finally, Sterling finished up 15 minutes early, so I snuck into the Q & A period with Monty and David at their MySQL future directions talk. Lots of good questions. Those guys are working hard at adding a whole new suite of features to MySQL — I think they have something like 4 different versions out at the same time.

Now I’m sitting in the main auditorium listing to the State of the Union speeches. Larry Wall literally and figuratively has fireworks in his slides! I have to say, he definitely puts together a nice show.

Here’s a look at sliver of Monday’s OSCON activity. I went
to an Advanced PHP talk, a Jabber “Bootcamp”, and a BOF
where we talked about a range of Open Source issues (acceptance
in corporations seeming to be a point we kept coming back to).

PHP

Sterling Hughes gave a talk on Advanced PHP today.
The slides are online.
Sterling loves performance tweaking, and let us know that up front.
He’s a very informative and entertaining speaker.

The major areas of PHP he went over were: Testing & Debugging
Documentation, Error Handling, Security, and Performance. I will
touch on some of the highlights. Some aspects of this
tutorial (and other PHP sessions) will be weaved into a roundup
of PHP impressions at the end of the week.

PHP Testing

Sterling is a proponent of writing unit tests before writing code.
He believes that writing code, and then writing tests afterward,
is a slower way to develop in PHP. In order to do unit testing,
he suggests using PHPUnit,
which can be installed via:


% pear install PHPUnit


Another angle is stress testing. Sterling is working on a tool
named “Cherry”, which should be ready in about a week (watch the
php.net site for details)

PHP Documentation

There are many forms of documentation, ranging from comments
in the source, pretty architecture diagrams that don’t
map well to reality, bug reporting tools (BugZilla), wikis,
and so on. A few tools that Sterling recommends:

• CVSTrac (CVSTrac.org) - a wiki, bug reporting tool
• FIT (fit.c2.com) - a wiki with integrated unit testing
• He also recommends running LXR (see lxr.php.net/ and lxr.linux.no/) and Bonsai (see www.mozilla.org/projects/bonsai/ and bonsai.php.net) on your CVS tree daily

Another area is API docs. Sterling advocates using a JavaDoc
style of documenting the API, and running phpDocumentor on it. This can be installed via:


% pear install PHPDoc


PHP Error Handling

Any sufficiently advanced bug is indistinguishable from a feature.


- Rich Kulawiec

There are four main areas of error handling: receiving errors (lots of “if”
statements are inefficient), exceptions (use try / catch), shielding (set the PHP variable display_errors=off, use a custom error handler), and logging (watch out for tabs! Be sure to escape your data that goes to a log).
There’s a lot more to say on this, so please check over his slides. He has some very good tips.

PHP Security

Sterling showed some of the security pitfalls that can easily trip
up a developer. A big problem area is command and SQL statement injection,
where an evil end-user is able to add commands and/or database
statements to a script. Problems are made worse by running
Apache as root (don’t do that!), since a command injection
attack can do anything at that point.

Some other security highlights:

• always turn off magic_quotes_gpc
• always turn off register globals
• escape shell commands with EscapeShellArg() and
  EscapeShellCommand()

• encrypt cookies that have sensitive data
• run as an unpriviledged user

Another big one is infrastructure. It is one thing to have
Apache & PHP configured to be secure, but do not forget the
machine they are running on! The machine must be locked
down as well. (my emphasis: think in terms of “what is
the weakest point in the system?”)

PHP Performance Optimization

Premature optimization is the root of all evil


–Donald Knuth

I think Sterling likes to talk about performance optimization more
than any other aspect of PHP. A recommended tool is APC - Alternative PHP Cache. It can be installed via:



% pear install apc


This is very important, because it can often take
PHP longer to compile classes than to execute them.

Know how to set Cache-Control headers, in order to cut down
(dramatically, in some cases) the number of hits being passed
through to PHP.

Another area is benchmarking and profiling. The Xdebug extension is
very helpful: http://xdebug.derickrethans.nl/

Two pear packages to use in benchmarking may be installed via:


% pear install Benchmark_Timer
% pear install Benchmark_Iterate


A not so obvious costly operation in PHP4 is hash table lookups.
Every time you reference a hash index, a check is done to see if
the index is numeric. This can really slow things down in
a loop. Sterling says this is much better (faster) in PHP5.

Again, these are highlights. See the slides. If you ever get a chance to see Sterling talk
about PHP, go see him!

Productivity

The Jabber Bootcamp was very technically oriented, but
there’s a huge productivity angle to it as well. This
may not be so obvious to those of us who know it
just from instant messaging functionality. My only exposure
to Jabber before this conference
was the Exodus client
on Windows, and Gaim
on Linux.

Some tech notes: A couple of preliminary things to know about Jabber, aside from the
bridges it provides to many IM systems, is that it is an open
protocol, and that it uses a pair of XML streams to transfer data. The complexity
resides mainly on the server side, making clients easy to write.
Jabber is great at passing structured data around. It is also
good at detecting the presence of people AND machines (think
of a buddy list for an app that can open a connection to
a Jabber server, or to you personally)

Getting back to productivity, I can see Jabber coming into
further use in these areas:

• group chat. Jabber has a better control hierarchy
  than IRC, which prevents abuse/takeovers.

• videoconferencing: there is a draft for a method of doing
  stp/ng videoconferencing.
• running commands remotely: Jabber can handle multiple logins
  from a person or machine, with each one implementing different
  functionality (called a resource in Jabber). A resource
  that parses incoming info queries can easily map them
  to commands, run them, and return the output.

I won’t include the notion of Web Services directly into
productivity at this point of the week (is WS a trend,
productivity enhancer, or dessert topping?) I will say
that Jabber, being a pair of opened XML streams (one connection
from each side), has great potential in this area. It
is faster than HTTP, and also handles a wide range of
encryption options. It can also compress the streams
to save on bandwidth (though, I am told, one or more
of the forms of encryption it uses also handles compression)

More info on Jabber for developers can be found at jabberstudio.org. Users can go to jabber.org to see a list of clients.

Trends

Open Source, in my mind, is beyond my idea of Trend. It’s
a permanent reality. An old project of mine was organizing
a huge collection of what we used to call “public domain
software” at Autodesk in the early 1990’s. I made a lot
of packages available from a local network mount point of
/usr/local/pubware (public, beer…) So for me, the idea
of getting a large company going on non-commercial
software packages (in this case, for in-house use) goes
way back. A lot of the talk at the “Daily Debriefing”
BOF, presented by Steve Mallett (the webmaster of opensource.org) and Matthew Langham, revolved around
the idea of how Open Source is making inroads into large companies.
The trend here is simple: lots of decision makers are starting
to “get it”.

“No one should buy their first copy of Linux, but they should buy their second”


– Steve Mallett

Matthew answered some questions about how German companies
respond to Open Source. He states that some leading German
Telcos are using Apache/Cocoon. Munich will save a lot
of money when the local government switches from Windows to Linux,

There is a feeling that small companies are more in touch with OS.
Large companies often don’t see the value of using
OS software, or of making changes available back to the community.
They want their proprietary systems, etc. The idea of
“free” scares managers, because they think “free” denotes
“bad quality” (one of the factors in how “Open Source” got
its name - Open Source is inclusive of Free Software, but
not the other way around)

I asked about the possible “splintering effect” of so many
Linux distributions out there. How does a company choose?
They have one provider of Windows, one of Mac OS X, and a
multitude of Linux offerings. One reply from an attendee
was that his large company went with Red Hat, in part
so that they have a responsible contact to fix problems,
but also so that their money would be spent to develop
software which would go back to the community.
Companies want someone they can call (and perhaps blame).
The trend is that many organizations seem to be in
an evaluation stage. Decision makers in those places
will be swayed by financial impact, but will want to cover
their legal bases (”who’s responsible for developing this?”),
and will want to be assured that Open Source will around for
some time to come.

If you are at OSCON, what are your impressions so far?

Related link: http://conferences.oreillynet.com/pub/w/23/oscon_grid.html#monday

Since I live a few miles west of Portland, I took the href="http://www.tri-met.org/">MAX light rail downtown. The eastbound stop
is several blocks north of the hotel, so we hiked to the hotel. I’d never been
to this Mariott before — it’s a great location. It’s right by the
riverfront, with lots of good restaurants and shops within walking
distance.

Speaker registration was upstairs in a huge, comfortable room. Last year’s
speaker room was tiny and cramped. This year is much better.

Unfortunately, the network was having trouble and the ever-pleasant Vee
couldn’t register us. I still had my badge for last year, so I was fine.
Allison Randal was behind me, so Vee made her a temporary badge.

Most of the talks were downstairs, one floor down from the lobby. They’d
already started. As we stepped off the elevator, Suzanne Axtell cornered us.
She pointed me to the press room, where I (finally) met Timothy Lord, from
Slashdot. (I used to work with Timothy and Jeff Bates when I wrote for
Slashdot.)

After dashing out that day’s Linux newsletter, I sneaked into Damian’s
“Inside ~damian/bin/” talk. I’d caught the very tail end at YAPC::NA and I
wanted to see the beginning. I’d missed the first part, Damian’s
vi customizations. He was just responding to a comment that
vim could do this or that.

As I reported in my YAPC::NA entry, the point of the talk was “Find
something you can automate to save you time and do it!” I picked up three
great tips. First, Damian recommends not setting your PATH on one
line in your shell customization file. Instead, he recommends something like
this:

PATH=$PATH:/usr/local/bin
PATH=$PATH:/sw/bin
export PATH=$PATH:~/bin

(Excuse the bash-isms; Damian uses tcsh.) The benefits are twofold.
First, it’s easier to read. Second, it’s easier to modify. If I want to
change the order of path items, I can just swap the appropriate lines.

The next trick was putting my shell aliases in a separate file,
.aliases. My .bashrc just sources this file. This
allows me to use the following alias to edit or to add aliases:

alias realias="vim ~/.aliases; source ~/.aliases"

This is really handy, allowing me to tweak my aliases in the same process.
Sure, I’m only saving two commands, but saving two commands makes me work that
much more quickly.

The final trick is cdd, an alias that changes to a directory
and immediately lists the files in the directory:

alias cdd='cd $l; ls'

It only saves a couple of characters, but it saves a couple of characters.
Damian went through some much larger customizations; it’s a talk well-worth
seeing.

During a break, I caught a few minutes of Michael G. Schwern’s and Norman
Nunley’s “What Works in Software Development”. The point of this talk is that
it’s possible to improve your development environment even if you’re the only
person interested in making a change. These guys have loads of good ideas,
from eating lunch with the vegetarian or vegan on your team to being the person
who solves little problems. There are a lot of really impractical books on
software development, but I suspect these guys have a useful book in them.
We’ll see.

I was pleased to note that several of my co-workers were in this talk.
There’s been a lot of discussion lately in my group about improving our
software and software development. Within the O’Reilly Network, we’re trying
to do ambitiuos things without huge amounts of resources. It’s important to
know how to work effectively and efficiently. More on that later.

I had lunch with Allison, Damian, and Nat Torkington. Damian was
demonstrating a super-secret module he’ll explain Friday. It’s pretty amazing.
Schwern showed up and I demonstrated my slide presentation software. On the
train, I realized that if it worked as planned, I’d probably have concurrent
access issues, so I revised the annotation-saving strategy over lunch. That
took about five minutes — test-driven development is very nice.

After lunch, Schwern and I presented the three-hour, 130 slide
“Introduction to Perl Testing”. This is basically the same talk we gave last
year, with a little more theory, a little more wisdom, and some revisions of
confusing parts. The room was packed and the audience was smart. (Really
— everyone followed along the explanation of mocking functions. That’s a
very smart audience.)

I’m happy to report that my presentation software ( href="http://wgz.org/chromatic/jellybean/">Jellybean WikiAnnotate) worked
flawlessly. I ran it from my laptop, published my IP address, and let everyone
follow along with the slides. Unfortunately, the network went crazy about
two-thirds of the way through the talk.

If you’re giving a presentation, make sure you have at least one local copy
of your slides. If I’d served these off of a remote site, I’d have been in
trouble. (Jellybean worked very well, but I had raw HTML slides to use if
necessary.)

After the talk, the Perl 6 design team met up and went out for supper. (I
finally had the chance to meet Dan Sugalski’s wife and kids. They do exist. I
think that means I’ve met the spouses and children of everyone on the Perl 6
team that has a spouse and children.) It turns out that Mrs. Wall is also a
fan of Neal Stephenson, Terry Pratchett, and Neil Gaiman. We briefly talked
about Larry’s impending State of the Onion talk. London.pm will be happy.

Someone suggested ice cream at that point, so we wandered around downtown
for a while, picking up small groups of people, until we’d taken over the
waterfront. Since it was edging past dark, I took my leave and went home.
Ahh, network access again. The OSCON
Kwiki was jumping, though. What a useful tool!

Day two is another day of tutorials, with OSCAMP and a couple of BOFs that need my attention. See you tomorrow!

Portland is a particularly appropriate setting for the

2003 O’Reilly Open Source Convention,
for deeper reasons than being a fun city that knows how to host
conferences. Portland just happens to be a hotbed of Linux
development. I attended a casual gathering of kernel developers this
evening at a bar, and nearly a dozen showed up–besides which, a few
more couldn’t make it. These people’s work ranges from drivers and
advanced filesystems to security and power management. I’m convinced
that if Linux and the BSDs vanished tomorrow, the people at that table
could create an operating system from scratch.

There are many reasons Portland sports such a concentration of kernel
hackers. The
Open Source Development Lab,
of course, is headquartered here, but that’s only a small part of it.
Several other companies in the area contribute developers to Linux.
Not much was said about Linux at the bar this evening, actually (they
spent almost as much time talking about Linus Torvalds’s car) which
was good because I was suffering from jet lag and had spent a full day
at the conference which I am about to describe to you, so after two
beers I couldn’t have told a pid from a pilsener. But this was an
everyday kind of guy and gal crowd, an unusual group of hackers who
are just as comfortable talking about houses and kids as about
computers. Maybe we’ll get a few of them to a Meet the Developer
session here at the conference; we’ll let you know if we do.

The conference started off quietly as usual with some tutorials; the
one I attended was Sterling Hughes’s
Advanced PHP.
Sterling is a developer on the PHP team, and a natural draw because
everybody knows he has the goods to offer. Fifty attendees packed the
room. Sterling, who seems to be predilected toward enthusiasm, waxed
particularly enthusiastic about SQLite, upcoming XML support, and some
briefly illustrated .NET integration. (These were by no means the
central themes of his talk. Although he just touched on them,
something made them stand out to me as interesting.)

SQLite

SQLite is a database founded on flat files. It initially sounds
primitive, but actually possesses some features, such as triggers and
subqueries, that aren’t supported by MySQL yet. Sterling described
SQLite as ideal for logging, which is something every PHP script
should do in handling errors.

Sterling mentioned that starting with PHP 5.5, SQLite will be bundled
instead of MySQL. This will not diminish the use of MySQL with PHP in
any way, of course; people can still install MySQL and uses all the
familiar PHP calls. But there are two reasons the PHP team decided to
include SQLite. First, the PHP license is more permissive than the
MySQL license toward people basing a commercial application on it. The
mismatch was proving difficult. Second, they occasionally
experienced build problems as new MySQL versions came out, until
things settled down.

Upcoming XML support

Sterling is working on an XML parser for PHP 5.5. It looks like it
will quietly handle all the XML syntax oddities in the background, as
XML support does in other popular languages such as Perl, and will let
PHP developers manipulate XML files with a few simple statements.

Upcoming .NET integration

Sterling said even less about this than he did about the others, but
clearly there’s some integration with
Mono.

Well, that’s enough for the rumors. What was the actual content of
Sterling’s talk? In brief, he discussed robust ways to:

• Do unit and stress testing on PHP, with PHPUnit and other tools.
• Document code, partly through a Javadoc-like system called PHPDocumentor.
• Handle errors (an area where you definitely don’t want to let PHP do
  what it does by default).
• Tighten security (a standard laundry list of good practices).
• Improve performance through profiling, benchmarking, and various forms
  of caching.

A part of each topic was Mom and apple pie, but Sterling knew what was
unique about PHP in each case and concisely pointed attendees to what
they needed to know. In some cases, PHP is distinguished by how its
called within the overall Web server architecture. While traditional
programs tend to be monolithic and create a hierarchy of function
calls, PHP is broken into Web pages that run individually and
cooperate only loosely. It’s like doing all your coding as callbacks
(my analogy, not Sterling’s). And it clearly imposes some difficult
constraints on a type of coding that for a while looked like child’s
play.

Well, that was just one day at OSCon. When it really gets going, with
keynotes, multiple sessions, and birds-of-a-feather, this conference
promises to be hot.

I looking forward to this week in Portland! I will be
doing my best to write every day about PHP, Day To Day Productivity, and
Some Trends To Think About.

About this week’s subject “PortLAMP Courses”:


Portland. Linux/Apache/MySQL/{P*}. A course from the OSCON buffet every day.
Just rhyme “Open Sources”. Stretched pun.

Multi-Distracted?

Mobile phones. WiFi laptops. Anything with a current. Cyberspace,
phonespace, and reality collide (causing some to be less, not more productive,
as a result)

On my way up to Portland, I read an excellent article in the Sunday New York Times (July 6), entitled “The Lure Of Data: Is It Addictive?”.
It gives a great look at the culture of the Always-On, where
some take a warped pride in the sheer number of devices they can juggle
(as opposed to, say, the main task at hand, such as a business meeting)

What’s that fine balance between sucessfully taking in a lot of data (in a conference, or at work, etc.), and pursuing it in so many ways, that it becomes counterproductive?

Related link: http://conferences.oreillynet.com/os2003/

The coolest thing about the start of the

2003 O’Reilly Open Source Convention
is the free T-shirt. It’s provided by MySQL AB and bears on the back a
box with a check-mark next to the word TRANSACTIONS. I’ve got to
take my hat off to Scandanavian humor. A single word (along with a
simple icon) here packs an elegant and ironic message combining
history and philosophy; a statement about the importance and place of
transactions in database applications; a defiant challenge to unnamed
competitors.

A different statement is made by the conference’s terminal room, now
dubbed the Rendezvous Room and furnished completely with Apple
computers. These were at the conference last year, too (and in fact
are offered by Apple to several O’Reilly conferences) but I shuddered
upon entering the room as if I had been transported into the
affectless cyborg dungeon of a sci-fi horror flick. In the lobby I had
just seen at least one person wearing a Free Software Foundation
T-shirt, and I could well imagine someone using the presence of the
Macs to bolster their arguments–”We always knew that Open Source was
a plot to adulterate and compromise our freedom!”

But I set up my Thinkpad running Linux in the foyer and recorded my
various grievances and satisfactions for the day. Six hours of flying
left me plenty of time to think up lofty issues for my weblog, such
as: why are the only clocks in Dallas/Forth Worth International
Airport in a store that sells clocks?
By two-thirty I was in the unDallas of the United States:
Portland, Oregon.
I immediately met half a dozen people I knew, and heard a tip about
why the current trend toward dynamic construction and execution of
code would lead to a resurgence in popularity for Lisp. (Don’t expect
a book about it from O’Reilly soon, in any case.)

For an hour and a half I walked about downtown Portland, whose main industry seems to be parking lots. But this plenitude
does not do justice to the city. It has one of the best public
transportation systems in the country, which serve largely to entice
you own to the historic district where you can get anywhere you want
easily on foot. Powell’s Books–the bookstore that outdoes almost any
other in coolness, with a special Small Press section that collectively constitutes a
complete history of every fringe or avant-garde trend in literary
history, where the meandering aisles make you feel literally lost in
thought–provides both free parking and a heavily used bike rack
area. Add to the city’s walkability the observation that the
down-and-out sections seem much less down-and-out than ghettos of the
East, and that even the down-and-out people try to preserve a bit of
panache.

I’ll be weblogging after tomorrow’s tutorials, so some actual content
will be forthcoming.

My recent niche publishing efforts made me think about the future of books in general and electronic books (eBooks) in particular. I was looking for good arguments to support going down the DocBook path and, as I was trying to convince myself that it is the right thing to do, I found very little munition to support that decision.

The main reason was the lack of incentive to justify the extra effort that goes into writing a well-structured DocBook document, accompanying XSLT sheets, and XSL-FO constructs just to end up with plain old troff source that must be manually tweaked before it is sent off to the printers. If I can do it in troff, TeX, LaTeX, or Lout in the first place, why bother with all of these extra steps? So I stayed with the tools I trusted.

One could say that I’m missing one of the key selling points of XML, which is the relative ease of conversion from XML to another markup. I wouldn’t agree. I just don’t need tat property of XML for that particular project. The importance of XML depends on the choice of the final delivery medium. If you plan to publish in print and PDF formats, then you need much more control over the final look than XML, XSLT, and XSL-FO can give you at the moment, and you stay with whatever tools you are using.

Things are different when you plan to deliver content primarily online, but even then, a lot of stuff can be delivered in PDF or HTML. And it is likely to remain that way in the near future.

What XML needs to spread like wildfire is a business model that will convince publishers that it pays to publish in that format. That new business model will quite likely require authors and publishers to abandon their current ways of thinking. Books will no longer follow the linear path of information delivery, but will instead be broken up into hundreds or thousands of tiny little tips, much like Windows Help files.

(On a side note, it is interesting that the market for alternative Help files for end user applications like Microsoft Word has not caught on. Could this be attributed to a lack of vision among publishers or the closed software that does not allow to plug alternative help?)

Another difference would be in the way these publications are distributed in. They must be delivered online and they must incorporate an easy to use and inexpensive micropayment model. How would this work in practice?

Imagine that you have a problem with document formating in Word. You press F1 and your Help browser connects to a Napster-like site and presents you with a list of tips on topics related to document formating, each with their usefulness rating (like Amazon.com review usefulness rating). These ratings will help you decide which publisher’s tip you want to download and pay for.

If the price is tiny, say $0.01, you will not mind if the tips was not helpful, and will dig deeper for additional tips, rating those that you downloaded so far as helpful or not according to their value to you (you could even add your own comments to the tip). Your opinion will be automatically transferred to the tip server and published on-line. If you fail to find information you are looking for, you could publish your own tip after you solve that problem yourself, and receive money from people who download your tips. A rating system similar to /. post rating will help sort useful information from the bad.

To make this system work seamlessly, we’d need software developers to provide a way to transmit to the tips server the GUI object identifiers that are now used to display correct Help pages when we press F1 or click on the ? icon and then click on a button, window, or an icon. This would be very easy to implement.

Such system would be a great incentive to switch to XML and to publish quality information. The tools are already there, and the only bit missing are micropayments.

This, not the portable eBook readers looks like the future of eBooks and computer books. Today’s eBook efforts are missing the point. They make reading books harder, not easier. A book that cannot be read once the batteries run out or when you forget your password is pretty useless to me. Technology ought to make our lives easier, after all.

As for my tiny book, I decided to stay with troff. My book it is not designed to be delivered in small bite-size pieces. If I ever need to publish it online, I will use PDF, and if I see an incentive to convert to XML, I will whip up a script that will do most of the hard work for me.

Could this be the future of computer books?

Related link: http://www.jboss.org/index.html?module=html&op=userdisplay&id=developers/project…

I got a chance to listen to Bill Burke, Chief Architect of The JBoss Group, speak at TheServerSide Symposium on the new Aspect-Oriented Programming (AOP) features that are in the upcoming JBoss 4.0 release.

The AOP features of JBoss 4.0 really are in two parts. First, there is the AOP Framework that enables AOP features to be implmented. Second, there are a series of AOP Services (or system-level aspects) that have been developed and are provided with JBoss.

What is AOP?

AOP is an approach to application development that is based on developing functionality that can be applied, or layered, on any of the Java objects you develop.

If you think of Object-Oriented Design (OOD) as designing interfaces and classes that other classes can then implement or inherit behavior from, then think of AOP as defining functionality that can be applied to ALL your classes — declaratively, not programmatically.

For example, you can literally add a method declaratively (without modifying the Java source code) to any class (or to all classes) in your application using what’s called an ‘Introduction’.

The ability to add ‘Introductions’ to your classes is based on the ability to use what are called ‘Interceptors’. An interceptor works by intercepting program flow just at the point between when you invoke a method and when that method is actually executed.

These ‘interception points’ are called ‘pointcuts’ because of how you are adding functionality just at the ‘point’ between when the object access is called and when the object is actually accessed.

The JBoss AOP Framework and AOP Features in JBoss 4.0

JBoss 4.0 comes with an AOP Framework that can be used to develop applicatins with AOP features. A number of AOP features, called system-level aspects, have been built by the JBoss group and are supplied in JBoss 4.0.

Among these features is the ability to make plain Java objects transactional. Using this capability, you can declare plain Java objects to be transactional and then have the JBoss container roll-back or commit changes to the Java objects based on how your transaction is progressing.

Another feature this allows is the application of the J2EE security model on top of plain Java objects as well. Using this capability, you can define security on top of plain Java objects.

Another feature is ‘remoting’. This allows you to declaratively specify that an object exists in a remote container (specified by a URL) and then access that object as if it were a normal object.

I’m really just scratching the surface. For more information, refer to the JBoss page on JBoss Aspect Oriented Programming

If you read TheServerSide regularly, then you know they hang out on the leading edge of the Java world. They preview chapters of upcoming books, have timely discussion threads and provide a pretty leading edge view of Java technology.

Knowing all this I was looking forward to the TheServerSide Symposium that run last weekend outside Boston.

I wasn’t let down. There were three days of sessions which were deep in technology and code and were delivered by some great people.

Some of the cool sessions were:

• Bruce Tate Author, Bitter EJB / Common EJB programming traps
• John Crupi Author, Core J2EE Patterns / Next Generation of J2EE Patterns
• Bill Burke Chief Architect, JBoss Group, LLC. / Aspect Oriented Java Development

My favorite session was the keynote by Bill Burke on the new aspect-oriented programming capabilities and features coming out in JBoss 4.0. It definitely looks like industry changing stuff.

It was refreshing for me as a Java developer/architect to attend a conference that was so deep tech that even the keynotes had Java code in their powerpoint slides. I spoke to a two people that had just recently attended Java One and they said that this conference was more deep tech focused. That was fine with me!

I got a chance to talk with Floyd Marinescu, Director of TheServerSide.com, and he told me that no call for papers went out. He just picked all the cool people he thought should be there and lined them up.

To sum it up, I thought TheServerSide Symposium was an excellent conference. Deep in tech talk, great presenters and leading edge technologies all in an intimate atmosphere. I’m already looking forward to next year.