A lot of people are asking what’s happening to media and
entertainment–especially people whose jobs hang on the answer. These
industries are changing fast under the hammer blows of Internet
marketing and sales, blogs and podcasts, and the unstoppable flow of
the larger economy. I see four possible directions the field could
turn.

Stagnation

In this scenario, nothing much new happens and current trends
continue.

In particular, the trend toward spending less time and money on
entertainment and news continues, as people turn to new pursuits such
as email and web surfing. But no new media develop online, partly due
to timidity in the larger society and partly due to a copyright regime
that places brakes on the distribution of underpromoted artists. An
economic downturn could contribute to the starvation of the
media. People could also turn from mass media to local pursuits:
community theater, dancing to live music, and so forth.

Television continues to try oddball novelties, such as cartoon shows
for adults and reality TV, but they don’t lead to a real
revitalization of the medium.

Musicians, who have been able to create fresh sounds for a few decades
by mixing genres from around the world, continue recycling old styles
and produce stunningly excellent performers but little memorable
content.

Film studios continue the trend toward making movies into little more
than static demos for games or advertisements suitable for viewing in
shopping malls.

News viewing continues to decline as news gets more and more
pessimistic and people get too anxious to view it because the facts
don’t fit their notions of the world. Literacy in general declines,
and tolerance for all the longer forms of publications with it.

Rallying

In this scenario, the dominant players–major studios and
publishers–find ways to regain some of their former success, or are
replaced by upstarts in the same industries.

New music sensations arise, along the lines of Stevie Wonder or The
Beatles. who create a mass audience across all demographics and erase
boundaries of genres.

People get interested in the world again, perhaps driven by a desire
to understand the inexplicable, such as wars in far-away places
involving cultures that view life in fundamentally different
ways. This drives new books, TV shows, and other offerings in
conventional media.

So the world of media as we have known it lasts at least a little
longer.

Amateurization

In this scenario, bloggers, podcasters, and other indie producers
working on a shoestring gain more and more of the audience. This
causes the professional stalwarts–despite their Johnny-Come-Lately
attempts to co-opt the amateur movement–to decline to the point where
they simply disappear because their income has sunk below their costs.

Performers get aggressive in generating buzz. With no sumptuously funded
organization to carry on large marketing campaigns, each performer may
depend more on a geographically local following. While this can
reduce revenue, it may also turn out to be a boon to local
performances. Perhaps it reverses the trend toward monoculture. On the
other hand, without professional marketing organizations to define
channels, perhaps genres disappear; culture becomes a soup in which
everybody tries to do a bit of everything.

News is reported by whomever is on the scene–often anonymously–so
that rumors circulate as wildly as they did in the age before mass
literacy. Occasionally a government or non-profit funds an
investigation to discover what really happened; the official account
becomes grist for yet another whirlwind of speculation.

Revitalization

In this scenario, new media grow up suited to the current age, just as
radio, film, and television grew in the twentieth century. The media
is online and highly interactive. Passive consumption is joined to a
new participation. Structures emerge for building an accurate view of
breaking news from the clashing viewpoints of multiple observers. New
opportunities for money-making are found in news and culture, perhaps
totally different from the advertising, subscription, and other models
of the past. People are re-engaged and resensitized.

Related link: http://www.navicasoft.com/Newsletters/August%202005%20Newsletter.htm

Bernard Golden, a deep thinker I recommend reading, has just published his August 2005 newsletter. The big insight for me is something I’d always sort of known but wish I’d said or even thought directly — “… the viability of commercial open source vendors depends upon the existence of a vibrant community.” Go read — he has much more to say.

The other day, someone forged my e-mail address and sent mail to a bunch of other people. (It happens every day.) This time, it hit a network that checked my domain’s SPF records and realized that the message came from an unauthorized server. Then it “helpfully” sent me a message delivery failure notice. I’m sorry someone sent unwanted mail, but we both know that it wasn’t me. What possible use is there in sending me the bounce message? (If you absolutely must send the bounce message, why not take a trick from the spammer, virus, and worm playbook and choose someone else on the Internet randomly? You have a non-zero chance of finding someone who cares about that message — but I don’t.)

Why haven’t all mail server administrators passed the One Question Certifcation Test for E-mail Filter Authors yet?

Related link: http://www.spikesource.com/contest

Have your heard? SpikeSource is currently sponsoring an Open Testing Contest. The contest aims to promote the concept of participatory testing while at the same time help projects improve their testing process. They are even offering prizes to the projects that improve their testing the most. First, if you haven’t nominated your project, you better do it fast. The deadline is August 31st. You want to sign up right now? Ok I’ll wait.

Still not quite sure if your project is right for the competition? Ok, first, your project should be written in C/C++, Java or PHP. Second, your project must have at least 20,000 lines of code. I know what your thinking….No! You may not use that trick we all did in college. 18,000 lines of comments will not get you qualified for the competition. There are some minor details, like you should actually be a contributor to the project, and it can not be commercially backed, but the other important requirement is that your project should have less than 75% initial code coverage.

Just to be clear, I checked in Wikipedia for what code coverage means. In management speak (see my others posts here, here, here, and here), all this means is the percentage of code that is tested. Why is this important? It’s important because code coverage is the metric that SpikeSource will use to choose the winners. Winners, as in plural, you say? Oh yeah, there will be first and second prize winners in each of the three categories: Lightweight (20,000 - 50,000 lines of code), Middleweight (50,001 - 100,000 lines of code), and Heavyweight (over 100,000 lines of code). There will also be a prize for the gnarliest test. I’d definitely like to hear about that one when the prizes are awarded.

The contest will run from September through the end of the year. During that time contestants will have to submit new tests for their project. All the while, regardless of the results of the contest, you will be improving the quality of your testing and reaping the rewards of automating the process with the SpikeSource testing service. How sweet is that?

So if you are looking to start off the new year with tight code, a better quality project, and pick a cool $2,500 to boot, then head on over to the SpikeSource Open Contest Registration site and get your project nominated. If you happen to be one of the first three committers to nominate a project in your weight class, you’ll walk away with $500. I’d say that’s enough to buy some shiny new gadgets. Well what are you doing still reading? Head on over and nominate your project.

Need $2,500?

Introduction
Suppose we have a table called ‘Tree’ with only one column called ‘lemons’ (unsigned int) for a very simple application to document the productivity of our trees. Typical data might be five rows with lemons numbering 0, 1, 2, 3 and 7. Of course, there can be any number of trees and the number of lemons can span the full range of an int.

The Puzzle
Write a pure SQL program that computes the smallest non-negative integer, such that no tree has that number of lemons. In the typical data above, the answer would be 4. Just to standardize, your solution must run in MySQL 4.1.12.

Guidelines

• Obviously, you could chain an huge tower of subqueries together

  select
  if(
      (select * from min_absent where value = 0 limit 1) is null, 0,
      if(
          (select * from min_absent where value = 1 limit 1) is null, 1,
          if(
              (select * from min_absent where value = 2 limit 1) is null, 2,
              if(
                  (select * from min_absent where value = 3 limit 1) is null, 3,
                  if(
                      (select * from min_absent where value = 4 limit 1) is null, 4,
  ...
                    )
                )
            )
        )
    ) 

  up to the full range of an int, but this is not elegant.

• There are also tricks involving creating new (non-temporary) tables. This requires both more permissions, that there be no table name conflicts and it is less ‘lightweight’ and clean than selects. You don’t need to do this, either.

Solution
To come in a few days if no one finds an answer.

Related link: http:www.yubnub.org

I love yubnub.org! Here are three commands I created today to make my job easier:

obsd — list matching OpenBSD man pages
nbsd — list matching NetBSD man pages
amarank — display Amazon.com Sales Rank for the given ASIN/ISBN

Go to yubnub.org and try them.

Example: obsd device
Example: nbsd device
Example: amarank 0596000278

What’s your favorite yubnub.org command?

Related link: http://www.flickr.com/groups/notestonetflix/

Too much free time on your hands? Write out a message on a sticky note and send it to a Netflix sorter. Take a picture first so you can upload it to the notesonnetflix Flickr stream.

If you don’t have enough time to do that, but still have some free time, just look at what everyone else wrote.

It’s so funny when I hear people being so protective of ideas. (People who want me to sign an NDA to tell me the simplest idea.)

To me, ideas are worth nothing unless executed. They are just a multiplier. Execution is worth millions.

Explanation:

AWFUL IDEA = -1
WEAK IDEA = 1
SO-SO IDEA = 5
GOOD IDEA = 10
GREAT IDEA = 15
BRILLIANT IDEA = 20

NO EXECUTION = $1
WEAK EXECUTION = $1000
SO-SO- EXECUTION = $10,000
GOOD EXECUTION = $100,000
GREAT EXECUTION = $1,000,000
BRILLIANT EXECUTION = $10,000,000

To make a business, you need to multiply the two.

The most brilliant idea, with no execution, is worth $20.
The most brilliant idea takes great execution to be worth $20,000,000.

That’s why I don’t want to hear people’s ideas.
I’m not interested until I see their execution.

Related link: http://hardware.slashdot.org/hardware/05/08/16/197222.shtml?tid=180&tid=98&tid=3

Now this Slashdot thread is classic!

This thread title: Henrico County iBook Sale Creates iRiot

Blurb: What do you get when you combine 1000 used iBooks being sold for $50 and 1000 people desperately wanting to buy them? You get an iStampede of course! Add into the mix one guy who watches too much wrestling and one gal who re-lived her first Backstreet Boys concert by wetting herself and you’ll being looking for video of the whole thing.

But seriously, 1000 used iBooks being sold for $50? I don’t think the amount raised will cover the cost of the entire event, let alone the fracas. And what about the planning of this event? I have seen, and been a part of, plenty of events like this that have gone smoothly. What a shame!

I posted this to python-dev last week, and since I’ve not heard any screaming, I’ll publicise it a bit more widely now.

So I’m currently planning for a 2.4.2 sometime around mid September. I figure
we cut a release candidate either on the 7th or 14th, and a final a week
later.

In addition, I’d like to suggest we think about a first alpha of 2.5 sometime
during March 2006, with a final release sometime around May-June. This would
mean (assuming people are happy with this) we need to make a list of what’s
still outstanding for 2.5. There’s a bunch of accepted PEPs that are waiting
for code. Once that’s done, there will be a final 2.4.3 sometime after or
close to the 2.5 final release.

I’m working on the Python 2.5 Release PEP at the moment. This will list all the outstanding major work that remains to be done.

Anyone who’s seen any of my talks beginning with PyCON 2004’s
“Scripting Language My Arse” would have seen I’ve switched to the Larry
Lessig style of lots of short, punchy slides (typically 6 per minute).
I was pleasantly surprised at OSCON to hear from a number of people who
were inspired by this to also try this format for slides for their own
technical talks.

The people who talked to me about this found that this format worked
really well for them, but that it was a bit trickier to put the talks
together this way. I thought I’d talk about a couple of ways I’ve found
to do this.

The way I did this the first time was to start with traditional (boring,
awful) slides - multiple bullet points, the works. The day before my
talk I was staring at the slides feeling a sense of impending doom.
The slides were so dull I was sure the talk was going to die, and die
very badly. This was extremely worrying, as the title of the talk (Scripting
Language, My Arse) meant that I could expect a large crowd, and I wanted
to make a good impression.

Glyph Lefkowitz and I got into a discussion about talks, presentation
formats and the like, and I recalled seeing a flash version of Lessig’s
“Free Culture” talk (available on the web
here).
This really inspired me,
and I sat down and rewrote the slides from scratch in a 2 hour frenzy,
starting with the original slides and using these for the structure.
The new format just rocked. It really freed up the talk, and the audience
seemed to enjoy it a great deal.

So that’s method #1: Write a complete set of slides in a traditional way,
where it’s easier to see what you’re doing. If you’re like me (lazy), this
is probably more work than you’d like. Writing one set of slides is enough
work as it is.

While I was putting together my tutorial and talk for OSCON, I tried a
different approach. For both of these, I had a lot of material. I also
didn’t really have a plan for how I was going to slot these together.

I tried a different approach this time - I went to the local shops and
bought about 6 packs of 5×3 inch index cards. I then wrote down every
single idea onto a different index card, and then spread them out over
the kitchen table. I then found it remarkably easy to see all the material
in a convenient-to-manipulate format. It was very easy to group the slides
that went together, string them into a coherent story and figure out ways
to link the pieces together.

There were a few other advantages to this approach. One was that I always
had a small stack of the blank cards with me - so that if I was drifting
off to sleep and had a sudden idea, I could scribble it on a card and
then figure out how to slot it in the next day. If you’re like me, you
find you’re always having random ideas and then forgetting them before
you get to put them into your talk. This fixed that mental problem with
a (very low-tech) technical workaround.

Another was that it was very easy to reorder the presentation. As good as
OpenOffice is getting, it’s still fairly painful to do massive slide
reorganisation.

This isn’t an instant solution to the problem of putting together a talk,
but I found it helped me a great deal, and index cards are remarkably cheap.

Finally, I should thank Lessig for his magnificent example - it truly inspired me to try something outside the normal style of talk, and for that I’m very grateful.

What tricks have you found for preparing presentations that work for you?

I finally got a Motorola Razr. I know I’m behind the times, but I was waiting until I wouldn’t have to pay for the phone. Amazon had a deal (and has had in the past) where I pay for the phone then get that money back as a rebate after my second month of service. T-Mobile can now officially offer the phone, and since I’ve been happy with them I went with them again.

The Razr is replacing my Nokia 3650, an ex-sexy phone. People were drooling on it when I got it, but I never really grew to like it. It’s a bulky candy bar design (remember when i said last year I liked the bulky design? Yeah, that lasted like two weeks). The Razr is everything I wanted last year: it’s a flip phone, it has bluetooth, it works with iSync, and it has a caller-id/time window. This is the phone I was describing while I looking at the Nokia. I could use it with Verizon, but right now it’s not to long after the Tour de France where Telekom T-Mobile got a place on the podium and T-Mobile hasn’t really annoyed me in the past year. It’s thinner and lighter (3.4 ounces to Nokia’s 4.6 ounces). It charges with a regular USB cable, where the 3650 needed a weird booster thingy I never figured out. It came with a 312 area code, so people won’t have to ask where 773 is.

It took me a while to start playing with the Razr though. I needed iSync 2.1 (included with Tiger, which I still needed to put on some computers) to sync over bluetooth. I updated one of my bluetooth-capable Powerbooks to Tiger. I needed to get all my address book data to it since I did a clean install. I set up a free 60-day trial of .Mac and sync from another computer to .Mac, then .Mac to my new Tiger machine. Once I had the Powerbook sorted, I paired (which Motorola calls “bonding”) the Powerbook and Razr. No problems there. I sync-ed. No problems there. Everything looks like it just worked.

Things aren’t so sexy once I opened the Razr to look at the address book. I should have known something was up when I saw Motorola called it the “phone book”. In the phone book there was a single entry for every phone number. Not a single entry per person, but one for every one of their phone numbers. Some entries showed up on five different lines in the phone book. That’s annoying (and I later fixed it by going into the Phonebook settings and changing the view to only show the primary contact. I can cycle through the numbers by right clicking the joy-wheel thing). Each entry has a little icon on the right hand side: a little house, mobile phone, or office building. That doesn’t really help me that much since some entries have multiple numbers in any of those categories. Just which cell phone entry does this little icon represent?

The horror doesn’t end there. iSync didn’t send over any of the email addresses. What the heck’s going on? Oh, and where are the postal addresses? Why is the list sorted by first name (because there is only a name field)? Why do I have to edit multiple records just to fix a person’s name? Curiously, while I’m doing this I’m listening to David Pogue rant about bad user interfaces. I want to cry.

The Nokia address book is actually pretty good. I didn’t always like it, but at least it could show me all my data. Indeed, last weekend a friend called me while he was filling out some paperwork, and I was able to scroll through my Nokia address book to give him phone number, addresses, zip codes, and all sorts of other things. The Nokia address book is good enough that I didn’t have to carry a PDA with all that other stuff.

Okay, so the Motorola phone book sucks. No biggie. I’ll just buy some Razr app that’s good. I start googling around. Not only do I not find anything to buy, but I find very few people complaining about the sucky software. Even after I figured out it is “Razr” and not “Rzor”, I didn’t have much luck. Is it like lots of other consumer technology things where people just don’t realize how bad they have it?

I’m not really a power user. I don’t care to much about listening to music, doing fancy multimedia messaging, or playing games. I just want a useable address book. I’m sure to get that sorted, and then I’ll be happy. All the other software can suck, but give me a decent address book.

I thought I’d start this blog by presenting a few random highlights
from OSCON 2005. This is not meant to be an exhaustive list, and please
don’t feel that I’m slighting you if I don’t mention your talk - this is
written a week later after a long plane flight home, so my memory is
not going to be fantastic.

I didn’t get to any tutorials on Monday - in the morning I was presenting
my Effective Python Programming tute, and the afternoon was mostly spent
recovering. The response from attendees at the tutorial was pretty positive - I got a lot of good feedback over the course of the week.

Tuesday morning I saw Jim Huginin and Martin Maly’s IronPython tutorial.
This was an excellent tutorial, and covered a lot of the things I wanted
to know about FePy. I will note that seeing the demonstration of cross-language
debugging in Visual Studio (stepping through Python and C# code, being able to
set breakpoints in both, the whole ball of wax) gave me a serious case of
debugger envy. A fancy .gdbinit that lets you dump out a nice representation of Python objects really isn’t even close.

(It might seem strange that Micosoft folks were presenting at the open source conference - but IronPython is actually open source, and not in a bogus “Shared Source” kind of way, but real open source, with a decent license.)

Tuesday afternoon was Brian Capouch’s Asterisk tutorial. I already knew a
lot of the material covered, but I still got a lot out of the tutorial. My
only complaint would be that the handouts were of a bunch of configuration
type things, and didn’t include the tutorial slides. I hope the slides end
up on the web somewhere.

On to the conference proper.

Wednesday morning’s keynotes were a bit disappointing. Tim and Nat’s
Open Source Trends was fun, but it would be nice if one of them posted a
blog entry with a list of the various things that they mentioned in the
talk. The Linux talk and the Schwartz interview didn’t do much for me,
although points to Nat for really hitting Schwartz with some tough questions.

Following that, I presented my VoIP Is The New Black. Luckily, I was in
one of the E rooms downstairs. These were small enough that I could do the
talk without a mike. This was important, because for some reason, OSCON didn’t
provide wireless mikes, and my style of speaking would have resulted in me
either ripping the mike out of the desk, or throttling myself as I bounced
in a random direction and forgot about the cable.
The talk was well attended, and the material seemed to go down well. One slight positive of the less than great keynotes in the morning was that I had time to ruthlessly hack into my slides - as of the night before, I was up to around 280 slides. I know from experience that around 6 per minute is a good number to aim for, so I took the axe to the slides and got it down to 240.

Other highlights from Wednesday included r0ml’s Semasiology of Open Source
talk (lots of fun) and Conway’s latest “my solution to the huge number of
CPAN modules available for task X is to write a new module for task X” talk.
Lots of pointless evil in there. I’m a big fan of pointless evil.

I was good and refrained from heckling during Guido’s State of the Python
Union talk.

Thursday morning’s keynotes were a better lot. The comparison of TCP/IP
to Shipping Containers was a novel take on the power of standardisation.
The Origami talk was fascinating, and massive points to the conference
organisers for scheduling it. Wrapping up the keynotes we had Dick Hardt’s
Identity 2.0. While this didn’t contain a lot of material that was new to
me, the format and the presentation of the talk blew me away. This is a man
who really knows how to present. I was left slightly in awe - and well done
for acknowledging Lessig for the style pointers at the end. More people need
to do talks this way - it produces a far better talk all round.

Randal Schwartz and Tom Phoenix gave a talk on various spam-filtering
technologies that they’ve used at Stonehenge. I got a bunch of ideas from
it - I’m currently doing a lot of thinking about the pending problem of
VoIP spam (I’ll post some ideas in a future post).

Sam Ruby gave an update on the parrotbench on Parrot project from last year.
The material was good, although it was slightly let down by his presentation
format. He went into some of the evil that Guido put into Parrotbench. Never
let it be said that Guido doesn’t have a twisted and strange sense of humour.

why the lucky stiff presented his Ruby rock opera during one of the afternoon
breaks. It was initially scheduled for earlier in the day in a smaller room,
but was moved after that room became dangerously crowded. What we saw of it,
between technical snafus, was a lot of fun. I’m still not planning on switching
to Ruby. “It’s better than Perl” really doesn’t cut a lot of ice with me.

Jeff Waugh gave a survey of a bunch of interesting new toys in Gnome and
Ubuntu in his Running with Scissors talk. Again with the “post a list of
links to the software in your blog”. I’d also suggest that Jeff invests in
a watch, or at least a small travel alarm clock that he can put in front of
himself to be aware of time. I was amused to notice just how many times
Jeff swore during his talk - mostly because I’d tried very hard not to swear
at all during my talk the previous day. (I think I succeeded, but I have no
real idea - my brain tends to think in swearing. I blame Deadwood, it made
me a potty-mouth. No, really, it did. Honest.) No-one seemed upset at
the swearing, which really does show that Australians can get away with
almost anything at a US conference.

Friday morning’s keynotes were a mixed bag. The CA presentation on Ingres
was intensely crap - I don’t go to this sort of conference for a marketing
talk. The only question from the audience was (paraphrased) “Aren’t you just
dumping your old rubbish software out there as open source because no-one
will buy it?” I’m sure I wasn’t the only other person thinking that.

Danny O’Brien’s On Evil talk was extremely entertaining - but anyone who’s
read his ‘To Evil’
series at OSDir.com would have expected that.

Asa Dotzler presented his “Linux suxx0rs” blog entry as a keynote. The
entertaining bit about this talk was that I was sitting next to Jeff
Waugh at the time, and it was fun to watch him seethe and seethe and
seethe. I thought he was going to explode by the end of the talk.
Of course, there’s a major difference between Linux and Firefox. Linux
applications, in general, understand how to free X resources. It would
be nice one day if Firefox gained the same ability, rather than requiring
regular restarts to free up the hundreds of megabytes of memory that it
consumes.

Other highlights of the morning keynotes (really, they were all good on Friday)
included the Howtoons talk (really, Make magazine for kids) and the Open Source Biology talk from a guy at MIT. I want my grey goo!

Friday was a half-day. I can understand why O’Reilly does this (a lot of
people leave during the afternoon) but given the utterly insane number of
tracks during the conference, I would have probably included one more session
on the Friday and cut down the number of tracks a little.

The conference finished with Miguel de Icaza demonstrating a bunch of the
eye candy and useful bits and pieces that Novell and Gnome have been working
on. I’m not convinced that I’d use much of the eye candy, but it was still
pretty, and X getting a decent compositing system is something to be praised.
I’m also really hanging out for Beagle.

And that wrapped up the conference. All in all, it was a lot of fun. I’m
not covering the drinking or random conversations in hallways that also made
the conference useful and fun. I’m definitely considering returning next
year - particularly if it’s in Portland again. What a fun town.

Related link: http://www.perlcast.com/audio/Perlcast_Interview_002_perl_testing.mp3

At YAPC::NA, Josh McAdams from Perlcast.com interviewed Ian Langworth and me about our just-released book, Perl Testing: A Developer’s Notebook. He’s released the interview as a podcast — here’s your chance to hear what it’s like to write a book and why we think that Perl is a great language for software development and testing. (There are a lot of other great interviews on the site too.)

Want more interviews? We can possibly do that…

Related link: http://linuxworldexpo.com/live/12/events/12SFO05A

A different sort of evidence of free software’s forward march came
today at LinuxWorld. I’ll report on that here, along with some other
interest events of the past two days:

• The march to open source

• A firm hand on anarchy: Eben Moglen unfolds (a couple creases of) the GPL 3.0 roadmap

• Perl over Java

• LinuxWorld Expo wrapup

My earlier blogs from this show are:

• LinuxWorld San Francisco 2005: Kick-Off

• Two countries, two systems for promoting free software

The march to open source

Two sales people (the kind of staff most evident at LinuxWorld) from
two different companies approached me within a 24-hour period to say
they want to talk to me about making part or all of their product open
source. I was flattered to be consulted, being neither a system
designer nor a marketing expert. But before I directed them to others
I thought would have more direct experience, I had some interesting
conversations that laid out some of the issues.

The determination of a company to open-source its product is at least
as significant as the decision of a customer to use open-source
software. When a company entrusts its core value to the community, and
declares that it can make a living by doing so, the movement has a
major gain. And it’s much harder to open a proprietary product after
building a company around a proprietary model than to start out open
source. These two companies–Radiant
Data Corporation
and
Qlusters–are brave folks.

Morality versus practicality

I don’t for a moment believe that, when leaders of the free software
community met at a
summit organized by Tim
O’Reilly in 1998
and chose to promote their work as the “open source” movement, they
were abandoning ethical or moral beliefs about the value of free
software. Nor do I think anyone who supports free software believes it
lacks practical application. But somehow an unwholesome view has
arisen that there are two different positions on free software, one
saying, “We wouldn’t care if free software had no practical value,
we’d insist on it anyway” and the other saying, “Go open source and
we’ll make you rich.”

But there are certainly times when one must decide what arguments to
stress when promoting free software. I chose the moral one in my 2002
article

Why Human Rights Requires Free Software. In talking to Radiant
Data and Qlusters, I chose a more practical approach.

These companies have to deal with investors, partners, VARs, and
customers with critical needs. Few of these will find it persuasive if
a company says they made their code base free because “We like open
source” or “We want people to cheer us.” The people on the other side
of the bargaining table want to see tangible benefits. Placed suddenly
on the front lines, therefore–asked point-blank to help a company
justify going free–I found the open source movement’s approach to be
crucial.

Law professor and FSF advisor Eben Moglen (one of whose talks I’ll discuss
later in this article) said that free software
advocates should let the “gravity of free software” attract more
people to use it. Practical arguments can bring both software
companies and their users further along.

Radiant Data Corporation

Radiant Data are the makers of a distributed, highly available
filesystem called PeerFS. Because it supports Linux, according to VP
of Sales Robert Peverley, the company would like to consider
open-sourcing the system “over beer on a Friday afternoon.” I
responded that they’ll get somewhere when they start talking about it
Wednesday mornings. But I wasn’t rushing him by any means: I
recognized that moving from a proprietary to a partial of full open
source model is a risky decision that has to be supported by a strong
business case.

First of all, does anybody want the software? Would anybody take a
second look if it were opened up? There must already be a baker’s
dozen free distributed filesystems. Well, Radiant Data thinks they
have something pretty compelling, and others have told them so
too. They are particularly proud of their locking mechanism, which
greatly reduces contention compared to other systems out there.

Second, what tangible benefits would they get by opening part or all
of it? For this question there’s a good answer too: they know there
are features they lack that the community might provide (or at least
test and comment on). For instance, all data is currently replicated
in full across all nodes in the system. It might be more efficient to
implement a RAID-5 sort of scheme; they would like to be suitable for
a Wide Area File Services solution.

So we tossed around all the models we knew of and speculated about
whether they’d work for Radiant Data: dual-licensing (a distinct
possibility, because a filesystem is tailor-made for embedded in other
applications), proprietary add-ons, support (not a good option for
them–the system works too well out of the box), parallel free and
commercial offerings, and so on. I’ll be staying in touch with
Peverley and report if the company gets to discussing it on Wednesday
mornings.

Qlusters

Qlusters’s SEMPRE product makes it easy for companies to deploy
software on multiple systems for load balancing and high availability;
they claim impressively fast failover times of a few seconds. SEMPRE
controls clusters of Linux systems.

According to Fred Gallagher of Qlusters, the idea of open-sourcing
part of their product came to them when potential customers told them
they had something valuable to offer, but that the customers already
had some part of the solution in place–for instance, using Nagios for
network monitoring. The customers didn’t want to throw out the system
that already was working and around which they had built up their
operations. They wanted Qluster to offer an open system they could
plug their existing operations into.

So the first step Qlusters (and Radiant Data, too) need to do is
modularize a fairly monolithic system. Qlusters plans on creating a
lower layer containing such things as their Virtual Environment Domain
(a way to build easy-to-deploy operating systems) and their
provisioner (which lets administrators add and remove systems from the
cluster as needed). They’ll provide a next higher layer of hooks, and a
set of useful tools on top. The lowest layer is the one they’re
thinking of open-sourcing. In theory, they could provide the hooks
without opening the code, but in practice they think users will find
it much easier and more appealing to adopt their system if it’s open
source. They can then offer a proprietary layer of products on top.

A note on clustering

Before I continue with the issue of open sourcing, I’ll point out that
Radiant Data and Qlusters both offer types of clustering, and that a
look around the show floor at LinuxWorld suggests it’s the most highly
contested market in Linux. It seems that open sourcing may be seen by
companies as a necessity in the fierce race for dominance.

At the Ottawa Linux Symposium three weeks ago (which I reported on
in other blogs),
a proposal was raised to add a field to the task data structure in the
Linux kernel to support clustering. I asked Donna Jeker of
Emic Networks
what she thought of this initiative. (The basic idea is to indicate
whether a task is local or remote, because tasks hosted on remote
systems must currently be handled through intricate user-space logic.)
Jeker was noncommittal, saying that they had certainly found their own
solution in user space but would be open to seeing what a kernel
enhancement could do for them.

Dual licensing and MySQL

In my conversations with Radiant Data and Qlusters, the possibility
came up of dual licensing (one open source, one proprietary, for the
same product). The companies best known for their
dual-licensing approach are
Trolltech
(makers of the Qt programming framework)
and MySQL AB.

Dual licensing is a natural for Trolltech, because a programming
library by its very nature is incorporated into the customers code, so
a licensing requirement must be obeyed to derive any use from the
product.

MySQL is different. Millions of sites can deploy it without linking to
its code. The base of paying customers is therefore much smaller,
although many will support the company out of self interest or will
purchase the MySQL Network support package.

The case of MySQL is hard to judge because a couple years ago they
received a massive infusion of both money and programming talent from
SAP, a one-time windfall that alters their funding needs.

MySQL also places tremendous pressure on itself by trying to hire just
about all the talented programmers who know it well and contribute to
it. This is important to their business model because they want to
remain the key innovators for MySQL; a substantial stream of useful
innovations from outside the company would alter their business model.

A firm hand on anarchy: Eben Moglen unfolds (a couple creases of) the GPL 3.0 roadmap

Many of use are wondering what the new version of the GNU General
Public License will address. After Eben Moglen’s talk at Linuxworld
yesterday, we’re still wondering. But Moglen laid out why Richard
Stallman is proceeding in such a perplexingly closed manner.

What would you do if you had a document that would affect the
computing needs of millions of people in a couple hundred different
countries, many of whom with scaldingly hot opinions of both what you
have now and what you’re about come up with? How would you advise
handling an estimated 150,000 individual commentators, and 8,000
shareholder organizations?

Thus the goal of providing a clear beginning and end to the discussion
progress. Moglen could not tell us exactly what is being proposed, but
he did lay out the outline of a process for the proposal:

• In October or November of this year, a process document will be
  released to “close down uncertainty,” so everybody knows how Stallman
  will proceed.

• The draft will be released at the end of 2005 or beginning of
  2006. Discussion will proceed for one year.

• In the summer of 2006, a conference will be held in Europe about the
  question of translating the GPL. Another such conference may be held
  in Asia.

• After the year of discussion is finished, Stallman will make the
  decision what to put in the GPL.

If this doesn’t seem like a particular open-source way of doing
things, it highlights that openness needs structure. There must be a
center somewhere; a place where the buck stops. Given that fixed
point, everything else can be pretty free.

I should mention here that to call Moglen a persuasive speaker is a
tremendous understatement. His passion for what he’s doing (he called
himself the “chief facilitator” of the GPL 3.0) is profoundly evident
throughout. The gravity and eloquence of his utterances makes one
feel, even when sitting in a sterile conference room, as if one is
present at the General Assembly of the United Nations. I joked to some
colleagues later that the U.N. should stop fumbling around with ICANN
and try to take over the GPL.

Despite Moglen’s’ refusal to discuss details, he was willing to mention
some of the general new social and technical trends that have to be
addressed:

• Software patents. While the GPL “can’t solve the problem,” it may
  contribute to an answer. Moglen presented a balanced assessment of the
  idea of an open-source patent repository, leaning (in my subjective
  opinion) against it. It’s worth noting that the OSDL has just
  announced
  a project to form such as repository.

• Web Services (by which he means, I think, application service providers).

• Trusted computing.

Moglen made many other statements about the background of the problem
and their philosophy in handling it, but I will leave those for others
to report on.

Perl over Java

I spent a good deal of time talking to my pals in
Osoft,
makers of an open-source ebook reader called ThoutReader. Writing it
in Java has made it easy to deploy, because apparently the “write
once, run everywhere” philosophy works in this case. But Mark Carey,
CEO, told me that their designer Gary Varnell, together with another
leading programmer in the Perl community, plan to redo the reader in
Perl, achieving a substantial reduction in size and speed-up in
execution. How can this be, I cried.

Well, the main explanation that comes to my mind is that Gary is a
devoted Perl programmer from way back, and that he may do a better job
at coding in Perl. Perhaps he just doesn’t grok Java in the same way.
The true coder, like the true revolutionary, is guided by great
feelings of love. (To paraphrase a 60s cliché.) But Carey said
the Perl footprint was tiny compared to all you need to run Java, and
that the Perl modules were easier to structure so as to modularize the
code.

LinuxWorld Expo wrapup

LinuxWorld continues, even as it directs more effort toward
enterprises, to be a significant event in the computing calendar. I
attended many serious and well-prepared talks, and met key people in
the industry ranging from hackers in the .Org Pavilion to vice
presidents.

The move to Moscone Center West was beneficial. Everything was easier
to find and more logically located. It was nice to have the .Org
Pavilion and other small booths outside of the main hall, away from
the carnival atmosphere, so that one could actually conduct
conversations.

The control exerted by management was pretty intrusive. Next year,
they can save money and simplify things by RFIDing all of us. Guards
at every door could be replaced by sensors that go off if a warm mass
passes through without a valid RFID. Cameras could monitor our facial
expressions and report our opinions of the speaker more accurately
than evaluation forms. Perhaps while Richard Stallman is circulating
his license for review, IDG can circulate the source code for the
Linuxworld Expo RFID Monitoring system.

Why stay current with blogging, when you can let the stuff pile up for days and post it all at once!

• 
  Had lunch with my editor, Chuck Toporek, to celebrate finishing
  Mac OS X Tiger In a Nutshell.
  I finished the last chapter the day before I left for Portland, and not a moment too soon.

• 
  It was pretty cool seeing my name on
  Pro Perl Debugging
  in the Apress booth. I’ve never had a co-author credit before. Apress
  asked me to be in the book signing group, but I wouldn’t feel right
  signing PPD as if it was mine. I added to it, and helped Richard write
  it, but it’s 95% his words. I guess I’ll have to wait until next year to
  be in the signing line for the Tiger Nutshell (if not something else…)
  

  
  

• 
  I met lots of behind-the-scenes O’Reilly folks like Tony Stubblebine,
  and Justin who was kind enough to update the headshot on
  my blog. Now I look
  slightly less goofy.
  

  
  

• 
  Talked to Denise Kalos, O’Reilly’s VP of HR, and Joanne Van Kampen, her
  head talent finder, about the problems of finding qualified technical
  people. I’ve had
  a programmer position open for a while, but my problems hiring are nothing compared to those of a company
  like O’Reilly. I have a new favorite interview question: “What are you
  an expert in?”
  

  
  

• 
  While I was out in the hallway talking to Denise and Joanne, I was wanting
  to make my way back into the hall because I knew the yearly awarding of
  the White Camels
  was coming up. I heard “Stas Bekman”, and I figured it was time to
  get back in there. As I walked in, I heard “There he is!”, and David
  Adler in a tie, hand out to shake mine. I had one of the three awards!
  I was so honored. I only wish I’d been able to be there for it!
  

  
  

• 
  You know the party is a good one when someone comes around asking “do
  you have any EMT skills?”
  

  
  

• 
  Kathy Sierra’s tutorial “Creating Passionate Users” was great, when
  I could hear it. It just didn’t work well in the room she was in.
  With such an interactive session, it was very frustrating to have
  unamplified audience members.
  

  
  

• 
  One of the booths was Gibson, the guitar manufacturer. A half dozen
  guitars hung from their kiosk, plugged into mini-amps and headphones. They
  were cool to have there, but basses were severely underrepresented.
  Some of us can barely handle four strings, much less six.
  Maybe next year, O’Reilly can get Ernie Ball, the guitar and string
  manufacturer who have
  
  dropped all Microsoft products in favor of open
  source products response to a raid by the BSA.
  

  
  

• 
  Schwern, Ian Langworth, Robert Spier and I talked to
  SpikeSource about doing testing
  on Perl and CPAN. It’d sure be great to have a company to help us with
  testing on many platforms and configurations.
  

  
  

• 
  phroggy so to clarify, we’re at the bottom of the stairs in front of Starbucks.
  
  waltman there’s a starbucks INSIDE the convention center?
  
  roger69 yes there is
  
  roger69 this is the pacific northwest, after all
  
  

  
  

• The Ruby on Rails guy used a great term that I love, derisively referring to XML in config files as having to do XML Situps:
  

  
  <situps>
  
  <situp position=”up” />
  
  <situp position=”down” />
  
  </situps>
  
  

• 
  Before my talk on project estimation and tracking, I asked Damian Conway
  to introduce me, per his suggestion in “Presentation Aikido.” He came
  up with about a minute’s worth of the nicest things I’ve ever heard
  anyone say. I was so honored. Key point: “Andy is the Anti-Damian.”
  

  
  

• 
  Next year, I need to develop a clone before I go, so I can go to sessions,
  but still have my clone go to the hallway track and keep up there as well.

What do you have to say about OSCON that hasn’t already been seen in reply to blogs that were actually current at the time, unlike my days-late wrap?

I had two fascinating interviews today about two initiatives, one in
the People’s Republic of China and one in the United States, for
promoting public-sector computer applications–and thereby the use of
open-source software.

Beijing’s rural, low-income, and educational initiative

The Beijing municipal government is getting ready to launch version
1.0 of a system aimed at bringing applications to rural areas. They
plan to use the same model for low- and middle-income residents of
cities, small and medium enterprises, and educational institutions. Linux (through the well-known

Red Flag
distribution) is a key part of the rural system. What’s interesting for me
is the role of government.

Years ago, the Chinese government tried to promote computer
applications by simply funding ISVs to create applications. As
Americans might have guessed, these failed to produce significant
results and turned out to be wasteful. The government realized they
needed a different path.

The current approach is: “It is better to discover a market for ISVs
than just give funding support to ISVs,” as summarized for me by
Liming Li of Intel China Ltd. Some details of this intriguing
philosophy came through in a hastily arranged interview Mr. Li set up
between me and Hu Qing Hua, director of an organization called the
Beijing Software Industry Productivity Center.

Mr. Li and I met at an O’Reilly party last night (he is a contributor
to an upcoming book, Open Sources 2.0) and realized after a brief talk
that China’s new effort to promote rural and low-income development
would make a great subject for an article. He tried to set up a
meeting that evening with Mr. Hu, but could not. So I rushed out of
another meeting at 9 AM today and sped down to South San Francisco
with Mr. Li, guided by the car’s navigational system, to hold the
interview in a spare hour before Mr. Hu had to take a plane.

Mr. Hu’s English was fairly good, but we relied on Mr. Li for
translation. I still may have misinterpreted some statements, so I can
only report my best understanding of the topics.

The slogan for Mr.Hu’s center is, “If you want to do software business
in China, go to Beijing through the Beijing Software Industry
Productivity Center.” In the complex business/government environment
in China, the Center manages to hook up multinationals with local
companies–but now they’re taking a significant next step and
representing the needs of ordinary Chinese end-users as well. The
project is named Strong Wind (Chang Feng).

In this new model, the Center provides funding and organization for
technical experts to meet with users and define detailed requirements
for some application to support farming, education, etc. These are
turned into applications based on commodity hardware from companies
such as Intel and local PC OEMs, running Red Flag Linux. Each project
has a coordinating committee that involves the various actors–local
and international, hardware and software–responsible for putting
together the solution.

The Center aims, therefore, not to request particular pieces of a
solution, but to put together have a comprehensive business model
covering the whole environment and to keep this in front of the
vendors. To save costs, Red Flag Linux is used and applications are
doled out to local ISVs. Large vendors such as Intel do integration
and support.

The Center keeps s firm hand on the project too, through an
organization of 80 people dedicated to testing and validation.
Considering that 20 packages may make up a single software solution,
this testing is very important. And after two months of testing in the
Center, some 500 units are sent to the end-users for field
testing. The vendors can then refine the system before its ultimate
deployment. Strong Wind is nearly ready for a 1.0 release, and they
plan to have 10,000 units deployed among users by the end of this
year.

Further funding can be provided for particular applications that the
Center recognizes a need for, based on the requirements gathering
mentioned earlier.

So far as they know, there is no one inside or outside China using
this particular model, but a lot of interest is being shown by
governments in South-East Asia, notably Malaysia, Thailand, and Viet
Nam.

The government is also working with the telephone company to bring
fairly higher bandwidth (such as ADSL) to as many areas as possible.

Because IT and Internet access are often associated with grass-roots
communications, I tried to formulate a polite question about whether
the system under development could contribute to rural and low-income
people communicating with each other. But the concept was too
abstract, so I couldn’t get a conversation going around it. The system
is apparently designed around delivering content.

U.S. government representatives meet at LinuxWorld

The other news I got today was about “Linux in Government Day,” held
at
LinuxWorld in San Francisco
for 50 representatives, aides, municipal employees, and others
interested in bringing free software into government applications. The
event was organized by Leon Shiman, who is an open
source developer, the secretary of
X.org,
and advisor to Massachusetts CIO Peter Quinn on bridging government IT
and community processes.

It was interesting to find out that government agencies develop
support for their software among their own staff, even if the software
is commercially provided and comes with support contracts (which often
go unused). This makes it much more feasible, in terms of staff
readiness, for them to adopt open source.

Government attendees were concerned with spreading code development
across government boundaries. They know they need to create and
sustain cooperation among agencies and sectors.

Leon’s key point is that cost is not the main factor to consider in
choosing open source; rather; the license is fundamental. Free
software flourishes because it draws out the creativity of many
diverse contributors, but he says “You can’t develop together if you
can’t see into the software.” He pointed to the license for the X
Window System as the most open (a subject of eternal debate, of
course), because it allows proprietary usage while protecting the
openness of code base. X is also uniquely “non-sectarian” regarding
operating systems and platforms–it has been ported just about
everywhere.

Highlights reported to me by Leon included:

• An address by law professor and free software supporter Eben Moglen on
  the legal issues involved in governments using open source.

• Reports by the CIOs of California and Massachusetts on state policies.
  Quinn closed the session with a detailed summary of the implementation
  issues that arose in the state when implementing open source policy:
  staff education, technical transition, etc.

• A talk by an expert on Latin American government usage of open source,
  explaining its “dramatic adoption” and the roles played by in
  communities and government.

• A talk by the CEO of
  Trolltech,
  about how that company fashioned a bridge between open-source and
  proprietary development, so their Qt libraries can be the basis for
  the highly popular KDE while still offering commercial licenses. This
  is important because governments, like many private institutions,
  worry about whether they can keep part of their code closed while
  making use of free software.

• Two talks that demonstrated open source’s strength through
  flexibility: one on Firefox functionality (central to government
  concerns because so many applications are delivered through the
  browser) and Linux Terminal Server Project.

In both China and the U.S., systems are bending in response to
changing environments. The supporters of open source and open
standards take widely varying steps to deal with the current state of
each system. I think everyone will be feeling their ways forward as
systems and users evolve.

For me the highlight of this year’s OSCON was hearing Kathy Sierra’s tutorial, “Creating Passionate Users”. The work Kathy and her partner Bert Bates have done in researching this topic is very important and useful. It should come as no surprise, that they are working on a new book for O’Reilly with a tentative publish date of January 2006. From what I have seen in her presentation and what I have seen in their Creating Passionate Users blog, this will be another popular book for them. The more I read and think about this, the more I want to read about it. This makes perfect sense, because it is exactly what I learned in the tutorial.

To give a sense of this great material, I’d like to share some of the highlights from the tutorial. My hope is that you’ll also see the value of this material and make an effort to incorporate some of the concepts into your products, communities, and relationships.

I think it is appropriate to start with a quick mention that some of the concepts from the tutorial come straight from the introduction to the bestselling book, Head First Java by Kathy and Bert. Quoting from that introduction: “Your brain craves novelty. It’s always searching, scanning, waiting for something unusual. It was built that way, and it helps you stay alive.” Most of the time your brain is discarding everything that gets thrown at it so that it can be ready to process something that really matters, like a tiger jumping at you. So as Kathy so eloquently states, one of the biggest challenges in engaging, inspiring, and emoting our customers is to get past the brain’s crap filter. And came through loud and clear from the tutorial, there are some real subtle, yet powerful ways to accomplish this task.

One of the first things covered in the tutorial set the stage for the remainder of the tutorial. To understand how to create passionate users, one first has to identify the characteristics of passionate users. Start first by thinking about yourself. What products, ideas, concepts, tools, or people are you passionate about? If you think about the kinds of things you do when you are passionate about something you might come up the following. When you are passionate about something you: talk about it, defend it, teach others about it, are irrational, evangelize it, provide feedback, and even try to improve it.

A second useful tip to begin thinking about passionate users is to list the characteristics of the things people are passionate about. Common characteristics might include things that: inspire us, are fun, are rewarding, are easy to start (or learn), have their own language, and that have a clear benefit. Clearly there are numerous characteristics listed, and even more possible, but one of the most important characteristics is the ability to continue grow and expand, especially if that gives people the ability to become an expert. An important follow-on question is: “How do you give people the perception that becoming an export is something attainable and desirable?” An example company discussed was Apple. For Kathy, Apple is like a crack dealer. They have mastered the ability to provide the first tier for free. But they also provide room to grow into more powerful features that lead to higher tiers. Of course, as you step up to a new tier, in order to gain the additional power or capabilities, customers become willing to pay for these new tiers.

An interesting trend that Kathy discussed was that sometimes the growth path for a product, may not even have anything to do with the product or service itself. So sometimes, customers will become passionate about your company or product by simply teaching people how to become an expert. Two examples given were: RedBull hosting a DJ Academy and Coldplay’s lead singer Chris Martin pushing Fair Trade. The key to this concept is that there needs to be a continuous growth path (a continously progressing goal), a never-ending path to learn about a topic that people could be passionate about.

I mentioned the fundamental theory of the brains crap filter earlier, but Kathy provided a couple of tips that might be useful in many different situations. First, any change in light will immediately grab our attention. Do you remember why? Yes, oh my, it’s a tiger! Watch out! So be careful using this technique, it could actually back-fire. I’m sure many web interface designers already know this. It certainly explains a lot of the obnoxious page designs we see on the Internet. Second, another trick is that anytime we use faces, our brains will respond positively. Research indicates that our brains have special abilities to pay attention to faces. So anytime your documents or web pages include faces with strong reactions, readers will be able remember the content more.

An industry that Kathy believes that we can learn from is the gaming industry. Game designers use two key techniques to keep players engaged in a game. First they produce a state of flow. From WikiPedia, flow is defined as the feeling of complete and energized focus in an activity, with a high level of enjoyment and fulfillment. One situation that demonstrates flow is when we become so engrossed with something we are doing, we loose track of time. Secondly, game designers provide an experience spiral. Basically this involves using a compelling benefit to motivate players to complete the loop (the level). Players will complete some activity until the loop is complete. When the player completes the level, they will receive a payoff. Usually this payoff provides new capabilities which establishes the next level and begins the loop again.

Ever wonder how to get readers of our documents, such as requirements, test plans, technical documentation, or blog entries, to recall and retain information? Kathy provided two tips to improve recall and retention. Both of these tips are well illustrated in the Head First books. First, use visuals and text to attain an estimated 89% improvement. Second, to obtain a 40% increase, use first person conversational language. You know what I mean? The interesting point Kathy made is that the brain thinks that it has to pay attention, so that it can hold up its end of the conversation, even when the conversation is with a book or a web page. Another helpful tip Kathy shared was using the power of story. This is especially important in creating communities. providing new community members with a back story and the history behind the community helps create myths, lore, and the illusion of insider knowledge.

In the end, I would be remiss if I didn’t share “the secret” promised from the tutorial. As any good speaker should do, Kathy pointed out that the one thing to take away from her tutorial. If we want someone to remember things we publish or what we are trying to communicate, make the other party feel something. According to Kathy, “The brain remembers that which it feels.” Once again, this is another key concept from the introduction of the Head First books. Creating emotions in people causes an increase in brain activity. Brain activity is fundamentally a chemical change in the brain. These chemical changes clue the brain into paying attention, thus bypassing the crap filter.

Now that I have heard this material first hand, thought about other things like my OSCON presentation and the flight home, and finally returned to study this material, one thing is clear. The concepts presented in Kathy’s tutorial make sense and feel right to me. However, as I have written this review and considered implementing some of the concepts I learned, it is going to take some time to get used to thinking differently. In addition, implementing them will take more time. The process of locating and gathering images and faces, annotating the images, creating the stories, and constructing experience spirals will take a lot more time than we have taken in the past to communicate. Keep this in mind, the purpose of speaking, writing, or teaching is effective communication. This only happens when the recipient of that communication understands what you are trying to communicate. If we know they are filtering almost everything as crap, then we need to work hard. If we don’t, then why even try?

Did you just see a tiger?

When did “stacks” become the new buzzword? The big vendors at OSCON all seemed to have “stacks” that did things, even though they looked suspicously like “frameworks” (and probably just like whatever buzzword proceeded “frameworks”).

Related link: http://linuxworldexpo.com/live/12/events/12SFO05A

LinuxWorld, for me, has belied attempts by many people to dismiss it
as a corporate marketing promo. The .Org Pavilion (where non-profits
at the center of the open source movement gather) was flourishing as
always today. A forum on Linux in government was held, though I won’t
hear the results till tomorrow morning. And I’m meeting tomorrow with
somebody from the Chinese government to talk about their use of Linux
to promote rural development.

I heard Seth Hallem, CEO of
Coverity,
discuss the results of running their automated bug-checking tool on
successive versions of Linux. The most interesting finding was that
Linux experiences dramatic leaps in numbers of bugs between minor
versions (2.2 to 2.4, and 2.4 to 2.6) but that the number of bugs
steady declines as new releases come out within each minor version. In
other words, introducing major new functionality comes with new bugs,
as one would expect, and giving the developers time to shake out the
software pays off through reduced bugs.

Hallem also pointed out that the sheer increase in the number of
kernel maintainers has led to more bugs, because they take time to
learn where the danger points are. However, half of the bugs Coverity
found were in drivers, and each system has only a tiny percentage of
these.

I heard some leaders of the Free Standards Group promote LSB. Like
other standards groups, FSG is better at finding new horizons than
establishing boundaries. Basically, no matter how much you
standardize, there’s another untamed frontier of things you’re tempted
to make work together. And so, on top of current work in packaging and
installation, shared libraries, configuration files, and file
placement (FHS), FSG is launching initiatives in system commands, ABIs
for system interfaces, the desktop, and system management.
Particularly interesting for me (although I don’t see the field being
popular or ready for standards for many years to come) is identity,
which I covered a week ago in a

blog from the O’Reilly Open Source Convention.

The opening keynote by Charles Phillips, president of Oracle
Corporation, sounded a note that’s all too typical for that company,
when he indicated they have 81 percent of the commercial database
market on Linux, and then tossed in the comment, “That’s not enough;
somebody else has the other 19 percent.” (That somebody else is mostly
IBM, presumably with DB2.)

But Phillips’s talk was significant because he made it clear that
Oracle–like other major computer vendors (IBM a couple years ago, and
Novell over the past year)–is walking the walk. Like them, he made a
prominent announcement that Oracle is putting Linux to major use
inside its own offices. And the reasons are similar: to find out what
the experience is like so they can help their customers make the
move. And for Linux’s intrinsic virtues, more of which I’m sure to
discover as the conference proceeds.

Now tonight I’ve got to pack away all the corporate marketing
promo swag.

Related link: http://www.computerworld.com/developmenttopics/development/story/0,10801,103777,…

It’s great to see coverage of women in computing, but some fact-checking would have been in order.

The latest Computerworld CareerMail included a link to a story about the “Women In Open Source” panel at OSCON. Unfortunately, there was a little goof in identifying O’Reilly Media:

Allison Randal, president of the Perl Foundation and an editor for Hillsboro, Ore.-based IT book publishers O’Reilly Media Inc., said she has been able to get involved in open-source projects by being assertive and just working hard.

I’m guessing that author Todd Weiss assumed that the Hillsboro , OR” on Allison’s nametag referred to O’Reilly, not Allison herself.
It’s baffling to think that Weiss seems to know nothing about O’Reilly, who were sponsoring the conference.

Last Friday after Miguel de Icaza’s closing keynote, Joseph Hill, who co-presented “Cross Platform Deployment With Mono” with me, met an independent .NET consultant who had a great suggestion: “Why not add a .NET track to the O’Reilly Open Source Convention?”

For Joseph and me, the timing of this question couldn’t have been better. Throughout the convention we had been commenting that there wasn’t much representation from the Mono community. Besides our session and the BOF that we sponsored, the only other Mono-related activity was the “Mono Bootcamp” by the authors of “Mono: A Developer’s Notebook”, Edd Dumbill and Niel Bornstein.

This has been stuck in my mind, since leaving Portland, and today curiousity got the better of me. So, to better understand the use of .NET and C# in the open source community, I did some research. During our collaboration, Joseph had been telling me about a few of the more well known projects like NHibernate, NUnit, Blogx, mojoportal, Muine, Blam, F-Spot, nGallery, and ASP.NET Forums - so I knew that there had to be more out there. Although, Miguel talked about a few of the key Mono-based open source projects during his keynote, specifically Beagle and iFolder he didn’t even scratch the surface.

What I found after searching SourceForge and NovellForge is a thriving and growing community of .NET related open source projects. From just the Trove Foundry in SourceForge, there are over 2,600 projects for C# and about another 100 for ASP.NET. To browse those listings refer to:

• Sourceforge Trove Foundry C#
• Sourceforge Trove Foundry ASP.NET

Additional projects can be found by simply searching SourceForge for .NET. Besides SourceForge, there are also more projects projects based specifically on Mono at the Novell Forge Mono Community
.

One obvious observation is that some of the more popular Java open source projects are now appearing for the .NET community. Examples of these include:

• NAnt
• NDoc
• SVG#
• iTextSharp
• NFop
• NLucene
• NAuthorize
• NSpring

Besides these projects I found a wide variety of projects covering many popular types of software. In fact, I think that the .NET open source community is significantly more mature and active then I ever imagined. So, I’m not sure what the process involves, but I’d definitely like to suggest to the O’Reilly Open Source Convention organizers that it might be time to move .NET open source out of the emerging topics into its own track.

Would you be interested in presenting or seeing more sessions on .NET open source at OSCON?

Related link: http://www.rubyist.net/~matz/slides/oscon2005/

In the words of Danny O’Brien, Ruby moved from being ignored to winning in about three weeks. How did it do that? In part, Matz made a powerful concept ubiquitous and simple. Plenty of languages have useful higher-order functions, but few allow you to pass them to just about every built-in function or method. By shaving off the rough corners of syntax and relentlessly pushing for consistency, Ruby makes blocks so easy to use you don’t have to think of closures, coroutines, and other complicated things. They’re just idioms. Matz’ OSCON 2005 presentation explains.

What will be the good next example of generalization and synthesis in a programming language?

Related link: http://www.oreillynet.com/pub/au/2333

I’ve spent the last week blogging OSCON 2005. This is the first time I’ve tried to blog a conference, and the experience was an eye-opener. There are a few things I got right, and a few others that went horribly wrong:

Bring your own laptop.

I borrowed someone else’s, and that turned out to be a problem on two fronts. First, it crashed several times, once when I had been neglecting frequent saves. Very frustrating. Second, I couldn’t safely bring it to the convention floor, which lead to other problems (see below).

Make sure you have a backup plan in case your main computer goes on the fritz.

I learned this one by getting bitten for not doing it. Thankfully, I was able to repair the laptop I had borrowed, but I lost a lot of time and (worse yet) flow.

Just because you’re cranky, don’t take it out on your subject in the blog.

I was rightly (mildly) chastised for somewhat stinging commentary in one of my entries. I believe it’s OK to rant (a little) about what’s making you cranky — blogs are after all supposed to be informal — but the person behind the main subject of the article doesn’t deserve snippiness directed at them.

Make sure the laptop you bring onsite contains no personal or financial data.

A conference full of geeks with over 2000 attendees is bound to have at least one bad apple. I was pleasantly surprised by the wide array of kind, helpful souls everywhere I turned, but there’s surely at least one cracker in the crowd. The odds against a cracker-free geek conference are just too long. Make sure the resulting unpleasantness doesn’t include someone getting to your financial data, stealing your credit identity, and bleeding you dry. Firewalls and locked down configurations are nice, but nothing beats just not having sensitive data out in the open in the first place. Thankfully, this is one of the things I had right at the outset.

If you’re stuck taking notes on dead trees, make sure you have enough pens.

I ran two pens dry and was well into a third by the end of the conference. I also just about filled a notebook that was empty when I started. I had no idea that I’d have so many notes to take!

Blogging sessions takes a lot longer than you think.

Relatively often, speakers have had to cut down their talks to fit within the time allotted. That means they will be very information-dense. If you want to do a decent job reporting on them, expect each entry to take quite a while to write. If you don’t know how long it will likely take, time yourself writing an entry on something else, such as a political speech or lecture at the local college.

Geek conferences imply sleep deprivation. Accept that, and do what you can to make it easier.

I went into the conference short on sleep, and then proceeded to get about half my normal sleep every night for a week. By Thursday I was cranky, making mistakes, nodding off during sessions, and otherwise much the worse for wear. In retrospect, I should have been much more careful to get caught up on sleep before the conference started. If I do this next year, I’ll also change my blogging schedule to get at least a couple more hours sleep each night (see below).

Blog during the day; don’t save it all for when you get back to your room.

Conferences are exhausting. If you just take notes during the day, and save all of the blogging work until you get back to your room, you will seriously short yourself on sleep. You’ll also risk getting behind (as I did), and that will only get worse as the conference accelerates through the week. Also, your writing will suffer from your exhausted mental state, and may eventually reach pure stream of consciousness whether or not that was your intended effect. There’s a lot of down time; use some of it to make your evenings easier by getting a chunk of the work done early.

That said, don’t let blogging take over your time at the conference.

Meeting people in person and joining ad hoc groups clustered in the hall are a big part of the conference experience. Don’t miss out on that by spending every break session glued to the keyboard. It’s not an easy balance to make, but I’m definitely glad I chatted at lunch or in the hall a few times this year.

Murphy was a realist.

I won’t claim that he was an optimist, as the old joke goes, but unhappy things do happen. For example, during the last week I’ve had several computer crashes, one lost article, two dead pens, a missed flight, a problem with new airport security procedures that nearly resulted in a second missed flight, a daughter with a nasty stomach infection, area-wide loss of DSL routing for almost 20 hours, and more general nuttiness than I ever expected. I’m not sure I could have prepared for all of these, but it’s worth having a few contingency plans, and a realistic schedule that assumes things will go wrong, rather than always perfectly.

Here ends the last of my OSCON 2005 blogs. It’s been quite a ride, and a lot crazier than I expected, but fun nevertheless. The book writing session recommended keeping one’s writing speed up by writing every day. I doubt I can manage that, but I’ll try to blog here from time to time to keep in shape. So long, folks!

“There’s no more. That’s the end of magic . . . .”

What suggestions do you have for future conference bloggers?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7178

Miguel’s talk was definitely a good choice for the closing timeslot. He talked about numerous happenings at Novell, from their efforts to move all employees off Windows / MS Office onto Linux / OpenOffice.org, to several new X technologies that impressed the heck out of me.

Novell currently has some 5500 employees, and has already managed to move all of them over to OO.o. They are also 50% finished with moving every PC to single-boot linux, and expect to reach 80% by November. In the mean time, it sounds like many employees still multi-boot.

Eating their own dog food is one thing, but Novell still needs to work on the Linux desktop to increase its mass appeal. To this end, they have considerably hardened the Mono VM (a clone of Microsoft’s .NET VM), to the point that it can now withstand weeks of continuous heavy load without erroring or crashing. From this point, they are working in two directions: making all user hardware Just Work, and implementing all missing desktop applications in Mono languages, such as C# and Python (via IronPython). It sounded like they are also putting some effort into convincing Windows ISVs to migrate to Mono.

They’ve been working on useability issues in Gnome as well; Novell likes to video users with three simultaneous camera angles (face, hands, and monitor), show the videos to the developers, and watch their mental models get massively readjusted to match reality (well, more closely, at least).

Miguel also showed off some individual technologies, like two-way iPod sync, multi-hop directory sync, and so on; some of these were actually ports of old-world Novell technologies to pure Mono code. He bragged that with Beagle they managed to be the first out the door with desktop search, though only by a mere 6 hours.

Nearing the end of his talk, Miguel showed off some of the amazing changes happening to X these days. The Cairo compositing / rendering model (similar to the PDF rendering model) is now available for X, and can be accelerated in OpenGL using Glitz and XRENDER. Ugly hacks for window transparency and other such effects are no more.

X itself can run on top of OpenGL using the Xgl server, which is nearly complete. Xgl comes with a Composition Manager, which allows all sorts of composeable eye candy and special effects; Miguel showed off some silly-but-cool wobbling window effects based on this.

Finally, since Xgl has the full power of OpenGL behind it, all of the virtual desktops are wrapped around a cube that is just dragged to show the desired desktop. It’s even possible to start a movie playing, drag the window halfway across the boundary between desktops, rotate the desktop cube through 45 degrees, and then watch the movie, now split across the border and in correct perspective across each face — without any visible rendering issues.

All in all, a very cool talk, and a great way to close out this year’s OSCON.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6878

MJD started off the session by asking people what they wanted to start with, since he had more slides than he could get through in 45 minutes. Everyone wanted to learn how to flip a coin over the telephone, so he proceeded to start in the middle, head for the end, and then jump back to the beginning again. That turned out not to be a bad thing, as he managed to fill in all the gaps pretty well.

He spent some time talking about remote coin flipping, and subsequently remote proof of knowledge without revealing anything about said knowledge. I’m afraid I’d mangle the details if I tried to relate them, but the protocols are interesting and actually fairly easy to understand. Someone in the audience even suggested an improvement for MJD’s remote coin-flip protocol.

MJD then spent a nice chunk of time on various hard problems, discussing the concepts of NP, NP-complete, undecideable, and so on. For examples he used Hamiltonian Cycles, the Knapsack Problem, and the Halting Problem. He gave a very interesting specific case that shows just how subtle the Halting Problem can be: Start with a number N; if it is even, divide it in half; if it is odd, multiply by 3 and add 1; repeat until N is 1. Noone knows if this code is guaranteed to halt for all natural numbers.

There’s no way I could do this talk (or topic) justice here, so let me simply say that it’s a fun way to spend an hour if you get a chance at a future conference.

What’s your favorite hard problem?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6806

Ask described four methods for incorporation of extensions: mixins, multiple inheritance, delegation, and hooks. Mixins, he claimed, are the best option for frameworks, allowing new methods to be added directly to the base classes that programs will be using. See Class::DBI::Plugin for an example of this technique.

Multiple inheritance has problems with encapsulation and generally makes it easy to shoot yourself in the foot (which I’ve seen brought up several times in the context of method resolution order on #perl6, though they seemed to have settled on a pretty good compromise for Perl 6’s default MRO). Of course, as Ask points out, Catalyst uses MI, and the Catalyst team seems happy with it.

Delegation, in which the main program assigns certain tasks to other objects (or even subprocesses), tends to be a sledgehammer — it’s very hard to make small tweaks without writing a lot of code.

That leaves hooks, in which plugins register with the main program to handle various predefined tasks. While there can be an initial pain to add hooks all over the place, Ask claims that this will pay off in finer control and a clear API for extensions.

Most of the rest of the talk followed qpsmtpd through several iterations of its hook-handling model. Each iteration did more work implicitely, making plugins successively easier to write. qpsmtpd eats its own dog food by trying to do as much as possible using plugins right out of the box. At this point it seems to merely have a small chunk of core functionality and a pile of hook management magic to make plugin code simpler.

qpsmtpd’s hook model was originally based on Apache / mod_perl hooks and return codes. For every hook, all registered handlers are called in sequence until one of them returns a code indicating that it has completely handled the hook. With recursion properly handled by the core hook handling code, even logging plugins that trigger normal plugins that call logging plugins Just Work. At this point, qpsmtpd even allows the result of a plugin to be caught and mangled by another hook, and primitive plugin inheritance is even possible.

Key in all of this is “Make it fun.” A large dose of sugar, a clean design, and a semantic model with great depth for hacking, are all intended to make plugin writing fun, so as to attract more hackers and build a community around the project. On that note, Ask pointed us to smtpd.develooper.com, svn.perl.org/qpsmtpd/, and develooper.com/talks/ for more information.

What’s your favorite extensibility technique? Why?

Related link: http://conferences.oreillynet.com/os2005/grid/

Asa Dotzler of Mozilla started off the Friday morning festivities with his views on ways the Linux desktop needs to improve to appeal to a wider audience:

Ease of Migration

Install alongside Windows and bring over every document and config setting possible.

Stability

Guarantee API and OS-level standards stability, to ease the burden on application vendors.

Simplicity

Optimize for the core 90% as Firefox does, rather than the rare, squirrelly, option-profuse edge cases.

Comfort

Make Windows users comfortable — he even suggested swapping OK and Cancel in dialogs to match the Windows placement, rather than the Mac placement.

Drew Endy of MIT explained how the technology for DNA synthesis has gotten so easily available that you can now enter base pairs into a web form, and get the appropriate DNA shipped back to you; you can mess with a creature’s DNA, create new organisms, and see what happens (fluorescent mice, anyone?); you can even do exotic things like make bacterial colonies that act as photographic film.

Unfortunately, the vast potential in this field is being stunted by a painful intellectual property environment — it is now considerably harder to license the rights to various biological processes than to recreate them in the lab. It would be nice if any DNA or biological process found in the wild, or whose information content (base pair sequences, for example) were found on the web, were freely available to all researchers. Sadly, this is far from the case, and the licensing hell is greatly holding back basic research.

Thankfully, academia is starting to fight back. MIT is now hosting a Registry of Standard Biological Parts, and the BioBricks Foundation is getting underway to fight this battle on multiple fronts.

On a more corporate note, Tony Gaughan of Computer Associates discussed the licensing mistakes that CA made when open sourcing Ingres (and no, he assured us, Ingres was not an end-of-life IP dump — the Ingres team is still growing, and many CA apps are built on Ingres). CA created a new license, but now believe that was the wrong choice. Tony argued that in fact a major license consolidation (or at least removing most of them from the lists of standard open source license options) would be a major help for the corporate world.

Danny O’Brien gave a very funny talk on evil in the software world, and what he has done about it. Perhaps his most important point was this: It’s worth fighting the bad guys (even much bigger, nastier ones) because even being seen as a David fighting Goliath can have an effect on the war, no matter what the individual battle’s outcome.

Saul Griffith of Squid Labs then showed off his company’s main product, Howtoons. They are comic strips that teach kids how to do hacks of all sorts, from air cannons to soda bottle rockets. They also offer parties where kids can make all sorts of toys. Very cool stuff; I can’t wait until my children are old enough to start playing with these. I particularly liked Saul’s comparison of toy hacking to programming: building techniques are subroutines, code for hacking the universe.

It was quite an interesting morning, though it either had less punch than the day before, or the crowd was exhausted. I’m leaning towards the latter. Still, the Howtoons and open-source biology talks were worth the time by themselves.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6411

I’m not entirely sure what to say about this talk. I was a bit disappointed because the talk was considerably more high-level than I had hoped it would be. To be fair, the main title indicates a high-level view, but I was seduced by the rather detailed-sounding subtitle.

Certainly, he did list a number of useful tools for network security monitoring:

• Snort (of course) for signature-based IDS
• Snort perfmonitor for overall network health charts (and with perfmon-graph, RRD graphs)
• BASE or ACID for Snort data display
• Bro for application logging
• ELOG for team coordination
• WebSVN for RSS feeds of configuration changes for change management
• tcpdstats for statistical analysis of tcpdump captures
• dshield.org, AlertCon, and ITR (Internet Traffic Report) to get an overall view of Internet attack patterns

He also had a couple general tips:

• Use a central log aggregator/analyzer (known as an Enterprise Security Manager, or ESM). This will allow cross-tool analysis of patterns, and give the security team just one UI to monitor.
• Watch how hard the central log DB is being hammered. Sometimes a sudden spike in the volume of log entries is your first clue that something is very wrong in your network.
• Poison your own DNS for fun and profit. Make malware callbacks, popup ad generators, sites that employees are not allowed to access, and so forth, hit an internal website instead of their usual destination. That site should send back appropriate messages to the user and advise them to contact the network security team if they feel they received the message in error, or think their system may be infected.
• Don’t just rely on signature-based IDS; use a behavioral (network flow-based) IDS tool as well.

Generally good advice. As I said, my biggest complaint was that I wanted more detail of this sort: “You have to cross check this log against this other log, because either one can be individually spoofed but it’s hard to spoof both simultaneously.” That’s probably a lot more detail than could fit in 45 minutes, I suppose.

What network security logging, analysis, and reporting tools do you use?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6533

A Peter points out: TIMTOWTDI, but some of those ways stink. Peter quickly ran through a series of steps to do first when you are forced to take over code that uses some of those putrid ways.

First off, if the whitespace and formatting suck, hit it with perltidy. If the code is deeply cluttered with useless comments and docs, kill the crap, keep anything that answers “why?” and perhaps subroutine calling conventions, and move the rest out to another file as pure POD. (It seems to me it might be reasonable to just move the POD to the end of the file, but that seems to be a matter of taste.)

With this easy stuff out of the way, it’s time to take a good look at the code itself. First, check if the code uses hashes and regular expressions at all; it’s surprisingly common to find code written by people who simply didn’t know about one or both of these. Failing to find either one should be a tip that the code was written by a very inexperienced Perl coder, and you are likely going to get big wins through refactoring brute force code into native Perlisms.

Some of the refactorings will be easy — converting long blocks of prints into a single heredoc, for example. Sometimes it’s worth it to go the extra mile and switch to using a templating toolkit (an already existing one, please) if the code would end up as littered with heredocs as it currently is with print calls.

Many instances of awful code were written with total ignorance of scoping. Even if the variables are declared at all, they may be declared in a huge block at the top of the file, with thousands of lines below that with nary a declaration. You’ll need to start narrowing variable scope religiously, moving declarations as late as possible. The only exceptions are config variables, which should be kept at top of file, or preferably moved out to a separate file entirely. Variable rescoping can be an arduous process, but it’s well worth it, especially as it is a prerequisite to the next step.

Much putrid code either contains very long blocks of code, or is even completely monolithic, with no subroutines at all. This will require heavy refactoring, but doing so will likely vastly increase the clarity of the code. It also gives you a good excuse to add to the test library (or create one if it’s missing). Peter recommended the use of Devel::Refactor’s extract_subroutine, which works particularly well with the EPIC plugin for the Eclipse IDE.

The next thing to tackle is suspicious code, much of which can be caught with strict and warnings. Peter pointed out (as had several other speakers) that these are not silver bullets; they merely catch a certain percentage of easily-identified errors. You still need to do some heavy reviews after the code is strict and warnings clean. Peter recommended starting the process by putting use strict; no strict; at the top of the code, and slowly moving the no strict; down, fixing bugs and assorted nastiness as you go. Then lather, rinse, repeat with warnings.

Finally, look for calls to external programs. In Peter’s experience, this is almost always an indication that there is a CPAN module that should be used instead.

By the time you’ve gone through all these steps, the code may not be as beautiful as it would have been if rewritten from scratch. However, it will certainly be much more maintainable than it was, you will hopefully have amassed a sizeable test suite while doing all that refactoring, and if you did a good job, all of the special knowledge burned into the code from years of use will not be lost as it would be in a fresh rewrite.

Go forth and conquer!

What was the worst mess you ever had to maintain?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6811

I suspect I’m not alone in watching the slow march of Ponie milestones and wondering “What the heck is going on under there?” There’s clearly a pile of work getting done, but I had no idea why it was such a hard problem (other than a vague feeling that perl5’s internals were rather . . . hairy, shall we say). Nicholas made the problems all too frighteningly clear.

The transition from Perl 5 to Perl 6 is fundamentally different from previous transitions:

• The gap between major releases is at least three times as long as ever before.
• The existing Perl 6 compilers are all clean reimplementations, instead of iterative improvements on a previous working version.
• Unlike previous upgrades, Perl 6 is not automatically backward compatible with Perl 5; some kind of wedge is needed.
• CPAN exists, and is one of the primary contributors to Perl 5’s success; we cannot break it.
• XS code exists, both in CPAN and in the wild; we really don’t want to break that either.

These are big issues, and there are others (the desire to have less hackish Unicode and threads support, for example). To get us to the final happy place, several parallel projects exist (apologies if I have any of the below slightly wrong):

• Pugs, a “top-down” Perl 6 compiler in Haskell, allowing us to experiment with and refine the Perl 6 language specification
• Parrot, the new virtual machine in which the official Perl 6 compiler (and hopefully, compilers for many other languages) will live
• Perl 6-on-Parrot, a “bottom-up” Perl 6 compiler written directly to the Parrot VM
• P5toP6, Larry’s automated translation tool, which reaches deep into perl5 to automatically convert Perl 5 source to equivalent Perl 6 (yes, with comments and formatting retained as much as possible!)
• Ponie, a port of the Perl 5 VM to run on top of the Parrot VM, allowing both pure Perl 5 and Perl 5 + XS code to run natively on Parrot

Each of these projects addresses a different part of the problem, and thankfully they seem to work well together, often pushing each other’s feature set.

The focus of this talk was Ponie, which as you can see above is the only project of the bunch working on XS compatibility. XS compatibility is a big issue. Not only is there a ton of XS code on CPAN that we want to be able to use automagically, but companies worldwide have tons of XS code floating around that we don’t want to break, even as we convince them to migrate to Parrot and Perl 6.

It turns out that XS code uses the very same APIs that the Perl 5 VM uses internally. That’s a problem, because those APIs have many issues:

• The Perl 5 VM is full of bugs and quirks, and has no formal specification; in effect, the “right” (read: relied on by existing XS code) Perl 5 behavior is defined as whatever perl5 does.
• Many “polymorphism” cases in Perl 5 are actually handled by pages of hardcoded if trees and other checks, spread throughout the perl5 source tree.
• Tieing and overloading don’t mix properly in Perl 5, because perl5 implements tieing as magic, which is limited and inconsistently checked for in numerous places.
• The API is very thin, including lvalue macros that expand to direct pointer manipulations.
• Unicode and threads were hacked on to the API instead of being fundamental parts of the design.
• Numerous bugs exist in the perl5 core code that have never been understood before, but which get in the way of larger fixes and refactoring.
• And the list goes on . . . .

Just to add a little more trouble to the pile, the Parrot VM made many fundamental design decisions exactly opposite of the way the Perl 5 VM works. For the most part, this is a good thing, as Parrot was built with the lessons of perl5 in mind; many bad design decisions have been fixed, but that doesn’t make Nicholas’s job any easier.

The obvious question is, aren’t Inline.pm or the Pugs <-> Perl 5 symmetric embedding good enough? Do we really need to go through all this trouble? Sadly, the answer is that both of these partial solutions are fundamentally broken. The Pugs <-> Perl 5 linkage leaks memory like a sieve, and there’s not much anyone can do about it, without doing most of the Ponie work anyway. Even an Inline.pm port can’t do much better (even were it to magically fix the memory management issues inherent in cross-VM object handling), because exports (especially non-trivial import magic) are really hard to get right — tied symbol tables are yet another thing that just doesn’t work right in perl5. And in both cases, there’s a lot of wrapping going on — performance would be less than stellar.

That leaves the Ponie project, and Nicholas with a lot of work to do. I wish him luck!

How much XS code have you written that remains outside of CPAN today?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6385

(I’m beginning to think someone out there doesn’t want me to finish posting entries for the conference sessions — this time SBC lost DSL routing for half the San Francisco Bay area for nearly 20 hours. Sheesh.)

This talk was packed to the gills, and really should have been in a larger room. I wonder if this was true for the rest of the security track, which seemed to be consigned to the smaller E meeting rooms. (Hello, O’Reilly? The SANS schedule alone ought to convince you that it’s probably a topic of interest to more people than can fit in E144!)

Nitesh is a security audit manager at E&Y, and made the decision to prohibit his team from using any closed source attack and penetration tools. [EDIT: Originally listed some more detail here, but that information actually was a from a different talk that I had accidently merged in my notes.]

He started by listing the standard attack and penetration phases: Discovery, Scanning, Enumeration, Vulnerability Exploit, and Rootkit/Log Cleaner Install.

For the Discovery and Scanning phases, he described several ways to use Google to find enormous amounts of information. Using Google for this has several advantages: searches are much faster than direct scanning attacks, searches don’t tip off the target by making a connection to them, and Google’s cache can provide detailed information about the target that would not be available from stealth scanning.

As sample Google searches, he suggested intitle:”Index of” admin and “VNC Desktop” inurl:5800. Webcam index pages and error messages also have distinctive, easy to search for signatures; one of his oreillynet.com articles has more detail on this technique.

For direct scanning of the target, Nitesh suggests Nessus, which he points out is far more than just a standard vulnerability scanning tool, but rather a framework for writing scans, with a scripting language of its own (NASL).

Nitesh recommended not using off-the-shelf exploit scripts, as often as not they are really just disguised malware that attacks the script user, installing a back door for the “exploit” writer’s enjoyment. (I’ve never downloaded an exploit myself, for precisely that reason. Mmmm, paranoia . . . .)

He took some time to impress on everyone the seriousness of cross-site scripting attacks (which prey on failure to encode HTML sent to the browser properly), by exploiting a badly-written shopping-cart app to give himself any price he wanted for any item. As he pointed out, the cart app in question has been known vulnerable for years; it’s just criminal that the responsible company hasn’t done anything about it. To give an idea of how often these types of bugs are exploited in the real world (usually for phishing attacks), he pointed out that at least one large bank sees 10-15 distinct new phishing attacks targeted at their customers every day.

He then went through a number of other useful tools to keep in your toolkit:

Absinthe

Blind SQL injection

BURP Proxy

Acts as a special proxy that allows you to manually edit HTTP requests and responses on the fly

Metasploit

Point, click, root

ettercap

Network Man-In-The-Middle attacks

Wikto

Web server testing, including automated Google scans

WHAX

Live Linux distro with all the A&P tools you want

He also made a pitch for his O’Reilly book, Network Security Tools. [EDIT: There was originally a block in here about reporting UIs, but this was actually part of a later talk that I had accidently merged in my notes.]

All of that was good information, but I expected the session to be details on how the sneakiest new attacks actually worked, rather than basic info about current toolkits for vulnerability assessment, peppered with occasional bits of wisdom won on the battlefield. Perhaps next year.

What’s your favorite network security tool or toolkit?

Related link: http://petdance.com/perl/crisis.pdf

Project management, in Andy’s system, is the process of answering the
question “Will we make it?” Obviously one goal is to make the customer happy
– but not at the expense of an accurate schedule. If your project is off
track and you won’t be able to deliver what you want to deliver when you want
to deliver it, why not find out as early as possible and figure out how and
what to change to make it work?

That’s the real trick… but if you’re smart and well-disciplined, you can
minimize disruption.

If there’s a secret (and this is what the agile development community has
been saying for a while — neither Andy nor I make a secret of that) it’s that
you have to be relentlessly honest about what you can and cannot handle. You
don’t have to have perfect knowledge, but you have to stop deluding yourself
and your customer that changes are free, that you’ve made more progress than
you have, and that your initial estimates and guesses are completely right and
will never change.

If you keep your tasks small and make good estimates and keep track of your
tasks and revise your estimates and review your schedule based on what you
know, you’ll always know where you are and how far you have to go.

If you let the unknown take over your schedule, if you let unfinished work
slip over and rework creep in, and if you can’t point to a single simple
project board somewhere and say exactly how much you have finished and how
much left to do, you can’t answer the fundamental question of project
management.

Maybe you’ll succeed anyway… but if you have that honesty and confidence,
both Andy and I believe that your project stands a far greater chance of
success.

What’s your secret to a successful project?

One of the few sessions I was able to attend at OSCON was “What Developers Want” by Murugan Pal, Founder & CTO of SpikeSource. Coming at the end of a long first day, I thought it was very interesting. He began by sharing that the most commonly accepted opinion about what developers want is only good documentation, a highly capable editor, and a good debugger. I don’t think many developers would argue with that, but Murugan went on to opine that these are just the basics and don’t really explain what are the things that developers really want.

So after covering this initial premise, Murugan shared a list of wants that he believes more closely represents what developers want. Much of this is also available, in detail, in an article on the O’Reilly OnLamp.com web site.

• Minimal Input Maximum Output - Very simply what developers want is to invest as little as possible in order to obtain significant advances in our productivity.
• Shortest Learning Curves - Again, very similar to point one, developers want to be able to ramp up very quickly on a new language, technology or tool.
• Challenges & Hacks - A subtle point, but none the less true, is that developers want a language to stay challenging. If something becomes boring or too simple, then a developer will become bored and lose interest. Akin to this concept, if a technology is complex, deep, and challenging, then the community will begin to create hacks that leverage the power and ability of the technology.
• Tips, Pitfalls & Best Practices - Also, independent of the language, technology, or tool, developers want a community that surrounds and supports itself by creating, supplying, and sharing tips, pitfalls, and best practices.
• Instant Gratification - Finally, developers want to receive instant gratification. These days who doesn’t? But specifically, developers want to receive an instant payoff. They want to still have the language, technology, or tool to provide instant gratifications moments when things fall into place and they receive a jackpot, i.e., when all of sudden things work and they become instantly gratified.

Murugan also provided a new way to look at the evolution of software development. By looking through the history of software development, we can identify major trends that serve as an interesting model as we attempt to anticipate the future.

• 80’s - Vendor Driven
• 90’s - Standards Driven
• 00’s - Market Driven
• 10’s - Collaboration Driven

Finally, Murugan shared a list of some of the web sites that he believes represents this new “collaboration driven” software development. In addition, these sites also typify and support the five wants for software developers.

• koders.com - Koders.com is the leading search engine for open source code. Our source code optimized search engine provides developers with an easy-to-use interface to search for source code examples and discover new open source projects which can be leveraged in their applications.
• stumbleupon.com - StumbleUpon helps you surf the web. It is a free tool which lets you browse, review and share great webpages while meeting new people.
• welie.com - web design patterns - Welcome on my small web site. As you can see, it is mainly about Design Patterns. With these patterns I try to capture every bit of good design that I encounter. I will update the site regularily and add new patterns.
• wikihow.com - wikiHow is a collaborative writing project to build the world’s largest how-to manual. With contributions, we can create a free resource that helps people by offering clear, concise solutions to the problems of everyday life.

Do you agree?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6842

This year’s MJD Lightning Talks were wide ranging, and for the most part a blast to watch. My biggest complaint was that there weren’t enough of them — I’d happily spend an entire afternoon (four 45-minute session periods) doing a huge set of lightning talks. I note that the Python folks are currently having to do an LT session of their own at the moment, and MJD’s are already two periods long, which indicates to me that there are easily enough topics to fill an afternoon if speakers from all tracks felt more welcome.

In fact, I’d say that inviting all tracks to a common block of LTs would probably make a wonderful event. MJD’s LT session is listed in the Perl track, but I believe he usually has a good mix of Perl and non-Perl topics; this year we ended up with about the same number of Perl and non-Perl talks. I enjoyed the non-Perl talks very much, and I’d love to be able to hear LTs from all of the other languages as well as the Linux and security tracks, for example. Somehow it needs to be made more clear next year that everyone is invited. Perhaps the session could be put in a “General” track, or titled “All-Tracks Lightning Talks” or somesuch (but hopefully better worded).

Here are the talks given this year, broken down into Perl and non-Perl (I’ve used the names listed in MJD’s talk list when my notes were found lacking; apologies if any of these are not correct):

Perl Talks

• Darren Duncan talked about Rosetta, his database abstraction layer. Rosetta apparently allows queries to be specified as ASTs, which will be automatically converted to any (supported) data provider / query language backend as needed. Sounds good, but I’ll reserve judgement — DB abstraction is usually a harder problem than it first appears, especially if efficiency is important.
• Eric Wilhelm explained how the detailed manufacturing of the metal cladding for San Francisco’s new de Young Museum was driven by Perl scripts processing images drawn by artists and architects. Pretty cool (and just plain pretty) stuff.
• Tim Maher explained what he meant by Minimal Perl, the title of his upcoming book. Tim recommends teaching and working within a subset of Perl, crafted to work to the strengths of new users and avoiding esoteric complexities. When his book arrives, it will be interesting to compare the language features he chose with those used in the fourth edition of Learning Perl.
• Marvin Humphrey pitched Kino Search, an indexer/searcher toolset written in Perl. He explained that Kino Search is not a spider, and not a web app; it’s more of an RDBMS for search indexes, providing much of the backend magic and some clever algorithms to give more relevant search results. Kino Search users can then put arbitrary front ends and data spidering tools in front of these libraries to fit their needs. This the another of the “Looked good, but 5 minutes is just not enough to show how much of the complex problem space this addresses” talks.
• Will Whittaker followed up last year’s talk on using mod_perl to make Apache speak SMTP (using Apache::SMTP) with a talk on how to use Apache as a traffic shaper to give suspected spammers brutally slow net connections (using his Apache2::Protocol, Apache2::Protocol::SMTP, and Apache2::Throttle modules). Clever and wonderfully evil.
• Grant McLean persuasively argued (as did several other speakers at the conference; it seemed to be a major meme) why Perl 5 programmers should just say no to prototypes. I only had a brief stint years ago thinking they were cool, but now I’m in agreement with everyone else — Perl 5 prototypes shouldn’t be used except in very special circumstances. They are very anti-DWIM and fundamentally broken in Perl 5 for lots of important cases.
• Ask Bjørn Hansen talked about perl.org’s new single sign-on system, Bitcard. Bitcard is open and free, both for users and websites, and supports APIs used by other languages and platforms. The goal is to never have to write another website user authorization system. Identity systems seem to be another major conference meme this year; there was even a keynote about it.
• Earl Cahill rounded out the Perl talks with a pitch for CGI::Path and CGI::Ex::Validate, which his team used to produce data-driven multi-page website flows, such as user signup. Validation code is automatically generated both in client-side JavaScript and server-side Perl for security and user friendliness. It wasn’t clear from the talk whether the modules were useful in a mod_perl environment, or only a true external-program CGI environment, or both.

Non-Perl Talks

• Andy Lester gave two great lightning talks. The first was his Seven Dirty Words of Programming. The words (and phrases) were:
  1. “Can’t you just . . . ?” (implies the listener is stupid; Andy suggests “Why doesn’t foo work?”)
  2. “My code/your code” (if you own your code and won’t let anyone touch it, you will learn that “can’t be replaced” and “can’t be promoted” go hand in hand)
  3. “Efficient” (overbroad and way overused; be specific!)
  4. “Intuitive” (just another way of saying “it makes sense to me“)
  5. “Fault” (who cares, and at a minimum, blame code, not people)
  6. “Too hard” (missed the detail on this one)
  7. “Later” (either put a real date on the calendar, or treat this as “never” and be rid of it)
• Andy’s other talk was “How to Not Be a Nobody”, in which he pointed out that open source is inherently social, and what you could do to be a part of this society. Here are some of his suggestions (with Perl-specific annotations):
  • At a conference, eat lunch with people you don’t know
  • Write a module for your favorite language or environment
  • Fix bugs, especially those in official bug tracking systems (rt.cpan.org and rt.perl.org)
  • Add a feature to an existing module
  • Improve some docs (annocpan.org)
  • Improve some tests (qa.perl.org)
  • Answer posted questions (lists.cpan.org, perlmonks.org, comp.lang.perl.misc)
  • Go to a local user group (pm.org)
  • Give a talk (OSCON, YAPC, Perl Mongers, etc.)
  • Write an article (perl.com, theperlreview.com, etc.)
• MJD himself did a talk (presented silently) on programming by deleting from a full file, rather than adding to an empty one. A highlight: the special slide used when he was heckled. It already (correctly) had “Shut Up, Uri!” in blinking text. I wondered if it was a setup, but then I noticed the frequency of Uri heckles in other sessions . . . .
• Casey West pitched JSAN, the JavaScript Archive Network. It’s great to see some Perl fu spreading out to the other languages, and it sounds like it’s being received well, too.
• Ben Tilly spoke of tactical and strategic thinking, and pointed out that getting plastered the night before giving a talk is definitely not strategic thinking. I believe he knew whereof he spoke.
• Elizabeth Cortell had the coolest talk, explaining how to make a lunchbox out of one of the thicker O’Reilly books. “Bet you don’t have one of these bad boys around the office!”
• Rich Bowen explained why he hates Apache, which was pretty much an extended (and hilarious) rant on stupidities in Apache configuration. (He also explained that he wrote some of the docs for it, so knows first hand how borken it can be.)
• Of course, there was the (in)famous Juggling Multiple Languages talk — pretty well done, though having heard the spoilers, I didn’t get the full effect.
• Tim Chambers finished off the LTs with his Ergonomics in Emacs talk, which seemed to consist of massive rebinding of the Emacs key tables (and a partial conversion to VI-style modal input?)

Phew, quite a collection! Of course, I already said I want more next year. (”More, MORE, I’m still not satisfied!”)

What do you think about having a full afternoon of lightning talks next year, with all tracks explicitely invited?

Related link: http://shiflett.org/archive/132

I had a blast at OSCON this past week. It tends to be my favorite conference each year for a number of reasons - the presence of so many smart, friendly people from the various open source disciplines being one.

I gave a tutorial on PHP security. You can find the slides here:

• http://brainbulb.com/talks/php-security.pdf (PDF)
• http://brainbulb.com/talks/php-security.swf (Flash)

I got a lot of positive feedback, although most people seemed more interested in the faux cover I created for my upcoming book than the talk itself. C’est la vie. For those who keep asking for it, here you go.

I also gave another PHP Security Briefing, so those slides have been slightly updated:

• http://brainbulb.com/talks/php-security-briefing.pdf (PDF)
• http://brainbulb.com/talks/php-security-briefing.swf (Flash)

Don renewed my interest in smugmug, and I created an OSCON 2005 gallery there with my pictures. I took a picture of a spider in the Japanese garden that almost turned out. I couldn’t tell until I looked at it on my computer, but the web behind the spider is in focus, and I wanted the spider to be in focus. Oh well. I took a picture of a rose in the rose garden that turned out a bit better.

As always, I spent most of the week forgetting to take pictures, but luckily there were always people with cameras around. One of the funniest moments was when Marcus, frustrated by his camera refusing to take a picture, asked it in desperation, “Do you take pictures?” It was as close to angry as I’ve seen Marcus, which is what made it so funny.

Geoff told Don about an idea we’ve had for a while about RSS feeds and related links. It would be nice to see something like this catch on, since all we can do now is scrape pages, which is unreliable to say the least. This open sharing of ideas reminded me a lot of Foo Camp.

Tim unveiled O’Reilly Connection, a new social networking site. Although it’s a hassle to go through the add a friend process yet again, at least this one has a FOAF feed, so your data is somewhat free (data libre?). Perhaps they’ll add a FOAF import tool soon. Like many of these sites, it’s written in PHP. Here’s my page. I wasn’t as far from sober as my photo makes me appear.

Now I’m left in nostalgia, listening to Hide and Seek by Imogen Heap, a song Tatiana (my O’Reilly editor and friend) played for me while Marcus and I were staring at this.

More conference coverage is available on Planet OSCON.

I just returned from USENIX Security ‘05 in Baltimore. I stayed in Washington D.C., and it is not close to Baltimore as many people think! I attended a tutorial on Tuesday, and the Wednesday technical sessions.

Butler Lampson from MIT and Microsoft gave his keynote address on real-world security. He stated that real-world security is feasible if it costs less in inconvenience, simple enough for users, and simple enough for vendors. He listed several reasons why we do not have real-world security: people don’t buy it ($$$$), and systems are so complicated now that they have so many bugs. He gave high-level reviews of locks, deterrence, and the Access Control Model, and presented a trust / “speaks for” relational model of security.

Professor Ben Schneiderman, Professor of Computer Science at the University of Maryland, discussed Human-Computer Interaction (HCI) opportunities for improving security and privacy. He reviewed the usability design goals, and discussed the importance of usability in controlling security and privacy, as put forth by the CRA and 2005 PITAC Report. Professor Schneiderman offered strategies to improve security usability: using multilayer interfaces, showing consequences of decision, and using information visualization to link relationships and to understand hostile events.

After lunch, Douglas Maughan, program manager at the Department of Homeland Security Science and Technology Directorate discussed some of the issues and tools the department is currently working on. Mr. Maughan discussed the research and development priorities at the DHS, the importance of cybersecurity, and its scope. He engaged the audience to discuss several important issues, such as the United States’ control of DNS. Mr. Maughan presented two DHS projects, DETER and PREDICT. DETER is a shared testbed infrastructure for medium-scale security research including repeatable experiments, and especially for experiments that may involve “risky” code. The Protected Repository for Defense of Infrastructure Against Cyber Threats (PREDICT) is a repository of defense infrastructure data, where the aim is to have private corporations donate real incident data for security researchers and academia to use.

Professor Avi Rubin at The Johns Hopkins University gave an update of his experiences on electronic voting in the United States. He spoke of his recent experiences at an annual conference of all state Chief Justices. Then, Professor Rubin discussed how many people in this country still do not believe the problems that exist in electronic voting in this country.

Related link: http://www.oreillynet.com/pub/wlg/7479

Jacinta Richardson recently told me that “I personally would love to hear a talk about documenting existing projects or about rewriting documentation to aid the non-technical user (or both!).” So I think I’ll blog about it, to get my thoughts into order.

The first topic, of course, is ‘why?’ Why go to the effort? Why degeekify documentation at all?

There are perfectly good (and perfectly bad) documents out there on almost every technical topic, and we’ve all heard people assert that the programs themselves are documentation! The information is out there, why reorganise it?

Well, the short answer is to go see Damian Conway’s ‘Presentation Aikido’ speech.

Okay, it’s not very short. A shorter answer is to see this weblog OSCON 1.1: Presentation Aikido, particularly the line, “As Damian hammers home throughout the tutorial, your audience is giving you their most precious resource — their time — and that is a privilege that you should honor.”

Yes, those of your users who understand programming and know the language you’re programming in, could read the program to learn how to use it. Eventually.
The rest could do so after a week or two’s study of the language, and possibly a college degree in programming.

Or they could read the programmer’s documentation, or the program specifications. (And some user documentation reads very like a program specification.) This would take less time.

But wouldn’t it be easier, and more fun, for them to read something like this?
How to Change a Tyre

David Eddings had one of his characters say “All the books in the world won’t help you if they’re just piled up in a heap.”

I’d like to expand on that, and assert that “All the information in the world won’t help you if you can’t make sense of it.”

More later….

Do you have any documentation you read for pleasure?

Wondering where ODBC, ADO.NET Futures, XQuery and XQJ Futures and of course JDBC Futures are headed? DataDirect’s Design Previews may just be the thing for you…

Every year, my company DataDirect Technologies does a six city Design Preview tour where we have a chance to talk about the direction of our numerous product lines, get your feedback and give you an opportunity to meet the top people behind our products.

This year is no different and through out September we are bringing our road show to the following cities

• Durham, NC 14th September, The Solution Center
• Washington DC 20th September, Tyson’s Corner Marriot
• New York, NY 21st September, Marriot Marquis (Time Square)
• Boston, MA 22nd September, Sheraton, Boston Hotel
• Santa Clara, CA 28th September, Westin Santa Clara
• Irvine (Costa Mesa), CA 29th September, South Coast Plaza Westin

We will be covering our full range products (ODBC, ADO.NET, XQuery and JDBC) and this year, introducing a new topic: Open Source Databases and giving you an insight as to their readiness for your most critical applications.

Want more details? There is a full agenda posted.

Want to attend? We have a registration page posted below that you’ll need to fill out. We’ll send you additional details, directions to the hotel once you’ve successfully registered.

http://www.datadirect.com/designpreviews/index.ssp

Related link: http://conferences.oreillynet.com/os2005/

Nat Torkington and his team did a superb job choosing topics for the
O’Reilly Open Source Convention. Just take this as given: if something
important is happening in the open source space, it’s at this
conference. There may be only one or two talks on a cutting-edge
topic, when the public hasn’t caught on to it in a big way yet, but if
you’re observant you’ll find the topic here, and key people in its
space available to talk to. I’ll show one example on the subject of
identity.

Identity: whose is it?

Substantial attention was given at OSCon to identity, a problem the
computer field has to solve if we are to make the most of what
individuals have to offer online. Dick Hardt of
sxip
gave a lightning keynote, Dave Smith spoke about
Passel,
and
Johannes Ernst presented
LID.

Identity, in this context, means offering verifiable and
persistent information about yourself to people online so
they can make judgments ranging from the most basic (”Should I read
an email from this person?”) to the most significant (”Is this person
offering advice really a lawyer?”).

Clearly, in an online world where people increasingly generate their
own content and form communities relative unmediated by servers,
identity will become central, along with reputation and other aspects
of trust.

In his keynote, Dick Hardt relied on the trendy “2.0″ meme, reserving
“Identity 1.0″ for the traditional provision of online identity, based
on a single all-powerful authority such as a CA or a credit-card
company, and heralding “Identity 2.0″ for forms of identity that decouple the
user from the authority.

I dealt extensively on this topic back in April 2004 in my two-part
article

From P2P to Web Services: Addressing and Coordination
and

From P2P to Web Services: Trust.
I think the downbeat assessment of the field I presented in that
article is still accurate. None of the very eager and idealistic
players in this space have solved the real-world difficulties of
maintaining, verifying, and presenting information about people and
institutions.

On the one hand are heavy-weight standards that take years to develop
and are dominated by huge companies: Liberty Alliance, WS-Security,
SAML, and so on. The size and complexity of these standards lay down a
stiff ante to any organization that wants to get involved in identity
services. Yet they barely scratch the surface of what is needed. This
is due to the perennial scourge of standards: they strive to be broad
frameworks, so they leave the details to other committees to
standardize later. And the problem with later is that it’s always
later.

The talks on identity that I heard at OSCon had similar
limitations–although I’ll show later how the proponents handle my
objection.

It’s fine to say, as Dick Hardt or Dave Smith do, that a person
ought to be able to obtain a valid ID from a trusted third
party and then present it as one presents a driver’s license in order
to buy alcohol. (Both relied on this familiar analogy from everyday
life. Me, I get my alcohol from the spigots that flow freely at
conferences such as OSCon.) The question is: who will grant that valid
ID? It’s a social question, not a technical one.

That’s why, in the title of this section, I asked whose identity it
is. The designers at this conference have a different concern: they
want to make sure your identity is just yours, because you control how
much of it to reveal.

What we all want to avoid is the existing situation with SSL
authentication in browsers. Only a limited set of certificate
authorities, such as Thawte and VeriSign, have a chance of being
recognized. (I checked a version of Firefox and found 21 authorities
in its Certificates list.) A user can install another certificate
authority, but who ever does?

Worse still, how many users reject a certificate from a web site when
it’s not secure–that is, who backs away when a dialog pops up saying
the site failed to be validated by the certificate authority? Most of
us forge right ahead.

So we all want a more inclusive system of third-party verification
with lower barriers to entry. For instance, Hardt possesses a driver’s
license from British Columbia, which is useful even though he has no
relationship with British Columbia other than having proved his
ability to drive there. We can all have multiple forms of such
flexible identity. (He said that federation, as in the Liberty
Alliance, could be considered “Identity 1.5.”)

The SAML-style frameworks try their best to set up robust frameworks
so organizations can exchange information about the characteristics
needed to trust people (for instance, who gets to look at financial
records) and whether individuals possess those characteristics.
But first the organizations have to agree on the characteristics
and their possible values. This is where the real effort lies;
not in exchanging the information once they agree on it. I cover this
problem in the previously mentioned articles.

Another problem, that of privacy, has to be faced. Most people
understand the commercial benefits of identity management, especially
online wallets and single sign-on. But this is the unfriendly side of
online identity, the side that makes people afraid we’ll go through
life with all kinds of sensitive facts tattooed to our online
selves. What Hardt and the others at this conference promote is a
friendly side of online identity, where we provide tidbits when
they’re useful to us and build communities and new services on
identity.

For instance, you could share your college degree, authenticated by
your college, to a forum where you’re trying to throw your knowledge
around. Similarly, you could decide whether to reply to someone’s
question based on his rating as a helpful and competent member of the
forum. Many sites contain valuable rating information, such as eBay;
perhaps they could allow users to share it outside their boundaries.

The goals inevitably involve trade-offs, because while it’s a big
advance to give a person a choice as to whether to share information,
sites can also require that information as a condition for logging
in. This conflict will never go away, although provision for
pseudonyms can soften the choice in some situations.

The underlying technologies are familiar and well tested
(cryptography, certificate authorities, asymmetric keys). CACert
(mentioned in an

earlier blog of mine),
is becoming a no-cost, low-barrier certificate authority for the
masses.

The designers of the heavy-weight standards such as Liberty Alliance
as the WS specs are designing from the top down. They have daunting
business needs and are trying to put together a system that will
ensure these needs are met in an air-tight manner. By shoving off the
immense social changes that would be required for people to bring out
digital identities, they risk that the systems will never be used even
if they can be built.

When you build a specification you hope to be used, you can’t get away
with saying, “Somebody else has to solve the social requirements,”
because it’s up to you to design your system to so those requirements
can be met.

The designers of the low-bandwidth, easy-entry systems such as sxip,
Passel, and LID are proceeding, instead, bottom-up. As with the
historic development of the Internet and the Web (or cute little tools
such as
GreaseMonkey,
described next), they want to provide identity capabilities and just
see where these go. Still, they won’t succeed unless some of those
same social requirements–giving everybody a certificate, deciding on
what parameters are important and how to represent those parameters
digitally–have to be solved.

Some unusual aspects of GreaseMonkey

Aaron Boodman described his GreaseMonkey Firefox extension to a fairly
sizeable crowd. Along with a few impressive demos and a quick lesson
in GreaseMonkey scripting, he laid out some lesser-known aspects of
this new and suddenly popular system. (It is being generalized and
ported to other browsers besides Firefox.)

First, he pointed out that GreaseMonkey scripts are ephemeral hacks by
nature and not likely to ever becoming something more. This is because
they are based on characteristics of downloaded pages that are not
under the control of the script, and therefore are fragile. (He didn’t
use the term screen scraping, but that’s essentially what GreaseMonkey
scripts do.)

For the same reason, Boodman has decided to put his library of scripts
into Wiki form, to encourage users to update and improve the scripts
without the formality of source control or project teams.

Other features under development include:

• User notifications for new scripts. These don’t involve users asking
  the script site for a script that applies to each page they download;
  that might be a privacy risk. Instead, news of new scripts will
  distributed regularly, perhaps once a week.

• A centralized management area for scripts called Greaseproxy.

• A possible rewrite in C++, mostly for security.

On the security front, Boodman admitted that GreaseMonkey had
originally (”like Windows”) been written with no concern for security.
He promised that all this had changed in the latest version, numbered
0.5. He now recognizes that a Web site’s content can abuse scripts,
notably by substituting functions written by the malicious Web site
developer for functions of the same names defined in the GreaseMonkey
script. To stop this, he’s instituted the use of a JavaScript feature
called XPCNativeWrapper, which ensures that the function being run is
the native one rather than one defined on the page, and a new
structure placing GreaseMonkey above the document root, so that the
document cannot override the functions in the GreaseMonkey script.

Boodman pointed out that GreaseMonkey can be useful for prototyping.
It’s much easier to try out a feature in JavaScript than to rewrite
one’s Web pages, although ultimately the feature should be implemented
in a more traditional and robust manner. He also claimed that Web
sites benefit from being hacked up by users through GreaseMonkey: the
sites get new features for free and end up with happier users.

Miguel de Icaza on desktop advances

The OSCon organizers saved the ever-popular Mono designer Miguel for
the closing keynote, and I wager it helped to keep people around at
the conference. He demonstrated the somewhat hallucinagenic 3D
effects Novell has achieved in the X Window System by integrating the
Cairo vector graphics library and Xgl, an X server based on OpenGL so
it can use hardware. Watching Miguels’ windows wriggle, fly toward us
and back, and wrap around, I was reminded of a demo
of Sun’s
Looking Glass
I saw a year ago. I found Looking Glass even more pleasing
aesthetically, but I haven’t heard anything about it recently.

A Java developer told me at the conference that one of the biggest
weapons Novell can wield in trying to persuade customers to migrate
from Windows to Linux is to tell them Mono will let them keep all
their ASP.NET scripts working. And that was one of Miguel’s themes at
the keynote.

He also said (with his characteristic flippant bluntness) that Novell
is finding out “what is wrong with the Linux desktop” by gradually
migrating its entire staff to it (they are 50% done). They perform
extensive user testing, with three cameras on each user to capture
their behavior.

Key areas where they’re working now include making hardware work, and
implementing missing applications (although I’ve seen other
implementations of some apps he showed). Some of the new applications
include iFolder, which keeps different systems in sync, a media player
with ipod synchronization and CD burning, and the Beagle search tool
(which I covered in an

earlier blog.

Short takes

href="http://conferences.oreillynet.com/cs/os2005/view/e_sess/6399">
Wez Furlong introduced PHP Data Objects (PDO). The goal of PDO is
pretty much the same as its Perl counterpart, Perl DBI, of Java’s
JDBC: to provide a portable interface that can let people use a
database application with a different database engine without having
to rewrite the library calls. (The SQL, in all these cases, may still
have to be rewritten because no two database engines have quite the
same SQL interfaces.) Furlong did not demonstrate any other strong
advantages to using PDO instead of PEAR DB or even a database-specific
library. PDO implemented various features such as error reporting and
streaming results (forward-only cursors) in its own way, but didn’t
break new ground that I could see.

Randy Ray
had an unmatched opportunity for forty-five minutes to tell a
completely full room why they should use SOAP for some projects
instead of relying on REST. He did not succeed. Because he didn’t get
through all his slides, I can’t tell whether he tucked some knock-out
argument in the last ones. But I would expect a talk like that to
explain why standard object-oriented virtues–such as strong type
checking and routine error reporting–were sometimes useful in Web
Services. I didn’t detect any such argument. Instead, he focused on
things I find irrelevant, such as the freedom with which REST
applications can define protocols. He did generously remind the
audience of XML-RPC and suggested it as a half-way measure between the
loose, light REST approach and the heavy-weight SOAP approach.

As you can tell, while I stated at the beginning of this blog that the
choices of topics at OSCon were outstanding, some of the presentations
did not live up to their potential. Usually I was glad I went and
learned something–there is no doubt that the presenters knew and
cared about the material–but I sometimes felt either that the
presenters focused on aspects of the subject that weren’t the most
interesting to me personally, or that they didn’t go into enough
depth.

On the other hand, an unexpectedly delightful

keynote
on how to adopt open source in business was delivered by Kartik
Subbarao of Hewlett Packard. I won’t try to reproduce his perky
analogies, in which poor open source utilization comes out as a swamp
and effective open source utilization comes out as Venice. I’ll just
mention that his insights align well with those in our recently
released book,

Open Source for the Enterprise.
Subbarao ended with an invitation to work on the
HP Linux Common Operating Environment (LinuxCOE).

David Heinemeier Hansson offered another interesting

keynote
about the philosophy that made Ruby on Rails a success.
He concentrated on three reasons:

• It places convention over configuration, meaning
  that the designers guessed what people want and try to provide the
  right defaults so there’s minimal configuration. (People can override
  the defaults if they want.)

• Change is instant: you don’t have to recompile or copy something to a
  server; just refresh. (The language characteristics of Ruby make this
  possible: introspection, open classes, and code execution within class
  definitions.)

• The system is complete and integrated, from JavaScript on back to the
  database engine. Nothing is left for the user to do in some other
  component of the system.

In short, “constraints are liberating.”

Asa Dotzler, Community Coordinator of Mozilla Foundation, delivered a

keynote on a few basic things
the Linux community can do to bring desktop use to the masses sooner,
such as slowing down library changes to ease application development,
and cutting down on the slew of configuration options in applications.
(Editorial complaint: nobody should talk about the confusion of
configuration options in open source applications until they’re tried
to do something basic such as turning off color printing on Windows.)
He pointed out that Linux enthusiasts like to concentrate on porting
more and more hardware, but that a lot could be accomplished apart from
hardware through “low-hanging” fruit such as the ones he described. An
audience member mentioned that this philosophy of simplification lies
behind the popular Ubuntu distribution.

Biologist Drew Endy discussed

Open Source Biology
in his keynote. On eBay you can buy equipment that let you change an
organism’s genome. There are many exciting (and perhaps scary)
applications of this, but due to the imprudent legalization of
patenting genes, many useful biological functions cannot be
manipulated without permission from some discoverer.

Endy also warned about the quality of the programmed organism (this is
the scary part), and risks of other intellectual property claims. In
several notorious cases, GE crops have turned up where they shouldn’t,
because nature doesn’t recognize property boundaries or license
agreements. But Endy also asked whether reverse engineering would be
possible or legal so that users could take control over their
crops. He finished by saying that the public must be brought into
these discussions, as with open source software, and announced the
founding of an organization with this goal.

This is still nearly the only conference where I expect the keynotes
to be interesting–and where they routinely exceed expectations. I
expect OSCon to continue to increase in size, and for the field of
open source to continue providing good fodder for it well into the
future.

Earlier blog on this conference:

OSCon: Developers and testers as heroes

Related link: http://conferences.oreillynet.com/os2005/grid/

There were several times during the Thursday morning keynotes that I found myself thinking “Wow. Just, WOW.”

Dick Hardt, now founder and CEO of Sxip, gave a rapid fire, absolutely hilarious speech on Identity 2.0 (distributed, user-empowered, secure identity management). It’s probably the first presentation on identity management I’ve ever seen that was actually good — sadly, most of them are rather strongly soporific, and it’s an important issue. It’s pretty difficult to describe his speech, but let me simply say that if you have a chance to see it elsewhere, do so.

Robert Lang showed some of the really stunning advances in origami that have occured since computational origami was invented, in which the desired end shape is described to a computer, and it computes the correct folds and folding order.

This resulted in amazing art that can take an entire day to fold by an expert. (Fish with individual scales? Fully realistic insects? Wow.) With the computer in charge, they have even been able to use a laser to score and stress the paper so that a light tap will cause it to fold itself.

The “real world” applications are equally impressive — a space telescope that has an external lens 100 meters across, which must be origami folded into a small cylinder to fit into a lifting rocket, and a tiny folded blood vessel stint that can be inserted easily and then expanded to force the vessel open.

Nick Gall at Gartner talked about the commonalities between computer networking and containerized shipping, and the lessons that can be learned from both to maintain the practical freedom to change. He talked about spanning layers within protocol stacks that turn stacks into hourglasses, generic enough to have a wide variety of users and apps at the top, and federated enough to have a wide variety of implementations at the bottom.

He claims that any spanning layer only needs to define three standards: an identifier, a format, and a protocol. Examples were IP address/datagram/IP, email address/RFC 2822/SMTP, and so on. With these standards forming a spanning layer, implementers are free to change anything above or below that layer, without anyone being the wiser. This allows freedom to change to be practically true, rather than a dream that dies with a half-life of about 5 years as less open systems become a complex, fragile mess.

David Heinemeier Hansson explained why Ruby on Rails is so popular: essentially they trade away configurability to serve the common 90% better through powerful design conventions. As David says, “Flexibility is overrated. Constraints are liberating.” The constraints that Ruby on Rails places on the developer allow the Rails stack to provide a whole lot of implicit magic and sugar, which lets coders concentrate on their own code, not infrastructure pain.

Furthermore, he believes Ruby on Rails should provide all of the needed infrastructure from AJAX code in the browser to database abstraction on the server, with one language used to write all of the tiers. As he said, “The buck stops here.”

A very energetic HP representative (Kartik Subbarao) used an extended water:earth :: open:closed metaphor to explain that companies that use open source but don’t act like good community citizens are really just making life hard on themselves.

Finally, Nat interviewed Mitchell Baker of Mozilla about the creation of the Mozilla Corporation as a wholly owned subsidiary of the Mozilla Foundation. Despite trying several times, Nat couldn’t tease a scandal out of this; it just appears to be a restructuring needed to allow Mozilla to do the many jobs it needs to do.

Who wanted to see some of those talks again, just because they were so cool?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7002

(Auggghhh! The original version of this entry was lost due to system crash. I’m recreating it here, but it may have lost something. Or it may not have had it to start out with, we’ll never know.)

Practical Perl testing was an odd session, with four instructors taking two session timeslots to deal somewhat randomly with various topics relating to Perl testing. Meanwhile MJD (in barefoot mode) lay on the floor chatting with Schwern and interrupted at random.

During the course of the rambling talk, they recommended several helpful modules:

Test::Differences

Use eq_or_diff instead of is_deeply to get better display of data structure differences

Test::LongString

Show shorter and hopefully more helpful failure messages when very long strings don’t compare equal; several people pointed out that this module fell short of what it should have done

Test::Deep

Very flexible deep comparison of data structures

Test::Harness::Straps

Allow you to create your own testing harness; heavily work in progress right now

Test::Exception and Test::Warnings

Test for thrown exceptions and warnings (or lack thereof)

WWW::Mechanize

Great gobs of sugary user agent goodness; greatly aids in testing and automating web site interactions

HTML::Lint

Check for HTML errors in strings or files; also includes Test::More style wrapper and weblint CLI tool

Test::HTML::Content

Flexibly compare HTML source against desired structure

Devel::Cover

Make sure tests are actually exercising all existing code paths

Test::Lectrotest

Specification-based automated boundary condition tests

Along the way, they recommended a couple good books: Sean M. Burke’s Perl & LWP and a Java testing book whose title I have clearly written down incorrectly, as searching for it is bringing up no obvious matches. Sigh.

The Java book recommended a testing mnemonic called “Right BICEP”, which roughly stands for:

Right: Correct output with normal inputs
B: Boundary conditions, random parameters, etc.
I: Inverse relationship (didn’t quite catch the details on this one)
C: Cross-check using a different algorithm
E: Error handling tests
P: Performance within reasonable bounds, and not suddenly worse after a change

Finally, MJD asked about doing test-first development when you’re unclear on the form that your app will eventually take. The most common suggestion was to write an exploratory mockup, write a full set of tests for it, comment out the entire mockup, and start uncommenting and fixing the mockup until the tests are happy again.

(Now that I have written this entry twice and fought with a broken machine in between, I’m literally falling asleep at the keyboard during page refreshes. I’ll save today’s sessions for tomorrow — sorry, folks.)

What favorite Perl module or book would you recommend for test writers?

Congratulations to Jonathan Robie who has I think deservedly, has been regonized for his dogged determination to see through an epic specification effort at the W3C.

Here’s a pic below…

As a friend and a colleague of Jonathan, perhaps I am biased but I recognize that his efforts in getting XQuery this far where nothing short of herculean. He is one of the key people responsible for for spawning what are now significant activties in organizations such as my own, DataDirect Technologies but also the gorilla types including Oracle, IBM and Microsoft. No word on what Google is up to yet, but I have to imagine they are closely watching a standard that gives new meaning to querying data.

So let me congratulate Jonathan on his Infoworld Innvotator Award. But I’m not sure the industry you helped create is going to let you sleep much in the next year or so. With the burgeoning XQuery update specification, I suspect we will all be keeping you very busy!

Related link: http://pugscode.org/talks/yapc/index.html

If you’re even vaguely interested in the fate of Perl 6, you MUST read the slides from Autrijus Tang’s amazing Pugs talk at YAPC. Short version: Perl 6 is already being implemented (no, really!) on a Haskell-based compiler, and it’s shaping up to be everything originally promised. This Parrot’s definitely not dead after all; it was probably just pining for the fjords…

Offline, shit. I’m still only offline. Every time I think I’m going to
log onto the network. When I was offline over the weekend,
it was worse. I’d wake up and there’d be nothing…
I hardly said a word to my co-workers until I said yes to my cell phone.
When I was here I wanted to be there. When I was there, all I
could think of was getting back online.
I’ve been here a week now. Waiting for a module, getting
softer. Every minute I stay in offline I get weaker. And every minute
Java squats in the bush he gets stronger. Each time I look around the walls move in a little tighter.

This F2F stuff is strange mojo. You talk to people by looking at them and making sounds. You share their music by hearing the bass from their headphones. You exchange notes by writing on business cards. You shake hands. You don’t use Google Maps for directions to the Intel booth giving away the thumb drives. You point. With your finger.
You do this all day.

It’s an odd week. It’s the Reality Distortion, and it seems to show up earlier every year. It’s creeping into Wednesday now. I wished Perl Conference 1 went on for another week. Now I talk to so many people each day that it seems like every day is a week. Randal and I were talking about something, and I was convinced that we had had some shared experience. I could picture it in my head. He was sitting on my right and we were working on this thing at the same time. I could see my Firefox window and his Safari window. He could see it too. But where was it? We knew that we’d hung out recently, but where? New York, Boston, San Jose? We each went back through our mental calendars calling off cities we’d been in recently. “No, haven’t been in Chicago this year”. “Wiat, I was in Boston when you were in New York”. We both exhausted. Ten minutes later, unable to drop the déja vu, we figured out it was San Francisco for WWDC; an alternate universe connected to this one only through weblogs. We send each other messages from the other side of the space-time rift through RSS feeds.

I was hardly online yesterday. I didn’t even get to post my news from the second day. I was too busy talking to people, pushing The Perl Review on people, taking business cards, and thinking “Just keep your eyes open”. Plenty of people want to talk to me about doing business with Stonehenge, and that’s a good thing. The problem is that I’m not only the guy to talk too, but the guy that handles a lot of the other things to keep our business running. I’m exhausted before I get here and that gets exponentially worse. Tomorrow I won’t even remember my name. At 34 years old, I’m starting to feel like I’m not keeping up with the open source pace. How is everyone else writing all these MVC frameworks and updating their modules up-to-date and going to this conference? Obviously I’m just a slacker. And damn it, parts of Intermediate Perl are due next week. Allison is breathing down my neck. I tell people I blame it all on the Tour de France and OLN. Three hours of coverage everyday is more TV than I usually ever see. Darn that Livestrong! I want a bracelet for conference survivors.

Despite the fact that Jim Brandt and I told people in “Conferences for Beginners” that this isn’t the business card sort of conference, I’m handing out and asking a lot of cards, I wonder briefly about RFID tags on the conference badges and some sort of device to collect the input (maybe a cell phone). I space out a lot. Potential business contacts most think I’m weird.

I run into Jordan Hubbard (Apple Computer) in the exhibit hall. He’s on his way to a huge Apple multi-player game fest on Apple hardware and giant displays. They had the same thing at WWDC and I though it was very cool, even though I’m not much of a gamer. I imagine a world where the person to drop Jordan’s name the most gets a free lifetime subscription to the Powerbook of the Month Club. I’m way behind the curve so I don’t think I’ll win.

Andrew Sullivan from Afilias Canada was at the Postgres booth. I love Postgres. I use MySQL because my ISP has it installed, but on my own machines I use Postgres, unless I don’t. Sometimes SQLite is enough. Someone has a Far Side style cartoon showing an elephant roasting a dolphin on a spit.

The Google booth has breth mints. The conference bag has breath mints from Yahoo! (I think). Strange. Well, maybe not. This is a geek conference. Some posits an increasing trend. Next year a sponsor might be Right Guard, or maybe Tide.

Gibson, the makers of guitars and banjos and the the like, is a major sponsor this year. They’re giving away two guitars. I fill out the little form and drop it in the fish bowl wondering how much it might sell for on eBay. That’s probably not right. I’ll have to use it for a Christmas present if I win. I don’t think there is any danger of that. For the rest of the day I refer to “the gibson girl“. People ask me if I mean Debbie Gibson (who is now Deborah Gibson, serious artiste). I keep dropping the reference anyway.

Sarah Burcham (St. Louis Perl Mongers) in back in town. I hadn’t seen her in years. She’s old school Perl Conference from back in the day—you know, the people that still say “O’Reilly & Associates” and “Netscape” and still call this shindig “The Perl Conference”. Cool deal. She purposedly rubs her feet on the exhibit floor carpet then comes up to shock me. Everyone seems to have seen her wiki except me, and she tells me it’s still in beta and its not public.

There is a caricature artist in the booth around the corner from the Stonehenge booth. I’m all over that. I’m a really bad reporter though: I can’t remember the name of the company, but they have an Eclipse plug-in to do flow-chart programming. Forget about code as words. Drag and drop the date processors into place and get on with like. It looks a lot like Anthracite from Metafy, and I already know that is cool.

I finally get to meet Julie Miller from Apress. They have a very nice booth in attractive black with yellow highlights. Geeks like black. Apress is getting into the Perl space, and I expect some good things from them. They publish one of the most wise Perl books out there: Writing Perl Modules for CPAN by Sam Tregar. Don’t let the publication date fool you: it’s still really good information, and it’s very useful even if you don’t want to put your stuff on CPAN. Apress is looking to fill out it’s Perl catalog, so send book proposals. Failing that, stop by the booth to get a nifty limited edition t-shirt.

I also ran into Scott Mathews of Geekstuff.com. He used to sell the Perl Mongers merchandise before he decided having boxes and boxes of shirts in his house was more than he wanted to deal with. It’s amazing that the shirts we sold five and six years ago are still showing up at the conference. I see a lot of JAPH and Perl Mongers shirts that I designed. That’s old school too. I’m thinking that the next issue of The Perl Review should have a collage of Perl t-shirts. I know Leon Brocard has a one of a kind orange Perl Mongers shirt I made just for him, so I should get a pic of it.

Before dinner I got to be part of my first O’Reilly author event. I got the O’Reilly author hat that initiates me into the club. I got to sign “Authentic author signature” on O’Reilly book plates. Damian conway started signing Perl t-shirts, which led to a horde of people collecting the whole set of signatures. I’ll have to watch eBay next week. Phillip Torrone and a Make Magazine crew was figuring out the maximum distance where a laser pointer could pop a balloon. I think they figured it was about ten feet. They were using a green laser (maybe 1500 mA, from the looks of it), so those puny lasers from the Dollar Store might have shorter distances.

I finally got to meet Greg Corrin and Kyle Hart from O’Reilly. Since I’m promoting the Llama book, they are going to be my new best friends.

Have you seen Mighty Mouse? Kevin Lim from the University of Buffalo has a video review.

The night ended (and the next morning began) with the Stonehenge party at Ground Kontrol with free beer, free pizza from the best pizza in Portland, Nona Amelia’s. Randal got his picture in News4Neighbors. I played a lot of Tron and Ms. Pac Man. The place was packed all night, and some people had to wait in line to get into the sauna bar, but I think everyone had a good time. We got home very late.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7164

Nat interviewed Sun CEO Jonathan Schwartz briefly at OSCON. When Nat asked about patents and the CDDL, Schwartz mentioned Sun’s promise to allow people who’ve agreed to the CDDL to use their patented ideas. If I had had the chance to ask Schwartz a question, I would have said:

I’m a free software developer and I’ve never agreed to the CDDL. Will Sun offer a formal promise not to instigate patent claims against me if I distribute software under another OSI-compatible license?

Choice is nice and choice is good, but if you reserve the right to punish people who don’t make the choice you like, do you really respect choice and diversity — or are you creating your own walled garden, claiming that it’s open?

What would you have asked?

Pre-Correction: a lot of people noted to me in person yesterday that I sound really tired in my OSCON posts. That’s probably true because it’s true. As I said in the “Conferences for Beginners” BOF, a lot of people come here after cramming in two weeks of work the week before, then get up early and stay up late. I’ve decided to make any corrections by striking the original text so you can still see it, and then correct text. For OSCON attendees only, tonight I’ll treat anyone providing a correction to beer and pizza at Ground Kontrol.

For Tuesday morning I moved into the speaker room to set up camp. Monday was all about the tables outside Starbucks, but I got to the convention center at the wrong time and the tables were taken up by a lot of the facility staff. It must have been their break time. I actually had to get some work done so I wanted to hide a bit too. Conveniently, there is a Kinko’s copy shop in the facility, so once I finished working on the Stonehenge flyer I just had to walk over to them.

The Kinko’s shop was a bit of a Monty Python sketch. I had put my file online (I keep losing my jump drives), so I asked the clerk what I needed to do to give it to her. She asked me if I needed to copy it to something. Repeat that three times. I wanted to know how to give it to her, and she needed it on physical media. Oddly, once I gave it to her, she simply digitally shipped it to the big Kinko’s shop who would run the job and deliver it the next morning. That I had this small step where it had to leave the wire (or wireless) to use sneakernet was a bit amusing.

After that I decided I should get some more work done. Someone (check the perl5porters archives) had gone through the Perl documentation with a Pod spellchecker. I integrated those changes to the perlfaq and also fixed up a lot of bug reports that have been waiting in the queue.

Lunch came around, and various people starting arranging locations and whatnot through the IRC channel and wiki. Those are amazingly useful for dispersed conference attendees. You don’t have to fall into the group you’re physically next too! Of course, things like Thai and Sushi made it onto the short list, leading the group seated at the speaker’s table to break into the usual “Thai again?” discussion. Any time any of us go somewhere, the local group wants to treat us to a special dinner, which to them means Thai food. It happens almost everywhere, so we end up eating a lot of Thai food. It’s not quite as special for us, then. Indeed, one group asked me if the restaurant was the best Thai food I’d ever had, and although I tried to beg off the question, I finally had to answer “Well, the best Thai food I had was in Bangkok”. To the people who are forced into Thai food everywhere they go, this is uproariously funny. Remember: if you want to impress the guest of honor, ask him what sort of food he wants. ;)

I ended up at Jax Bar because it has free WiFi, which we ended up using a lot less than you might think. Against my better judgement I got the Cuban Pulled Pork sandwich, which is the same as a bagel outside New York or a cheesesteak outside of Philly. It was decent pulled pork for something above the Mason-Dixon line. I could say “It’s nothing to blog about”, but it’s too late for that.

Our lunch crowd was Randal and I, Jim Brandt and Jeff Till (both from Univeristy of Buffalo), Bill Odom (TPF), Curtis “Ovid” Poe (Portland Perl celebrity and Kineticode guru), and Dan Brian. (I really should provide links to all these, but that means I have to find them (Google is my bookmark manager). Maybe I’ll update it later—I just want to finish this and I’m only up to lunchtime!) Most of the conversation was dishing on Perl book publishing. Most of us knew things that aren’t public (”Who really wrote that book?”, “Why does that editor hate Perl?”), and most of us have had book proposals die horrible, flaming deaths. Book publishing is an odd industry, and we’re mostly lucky to have a publisher like O’Reilly (and Apress more and more) who are willing to take a chance on topics that might not sell. Now that I’m an official O’Reilly book author, I get to puff out my chest and wear the O’Reilly author hat.

Afternoon is a blur. I think I did work in the speaker’s lounge, but I can’t remember who I talked to or what I did. The CVS commit logs aren’t helping either. Maybe I didn’t do any work. Maybe I just stood motionless in the middle of the hall staring into space as crowds flowed around me. Maybe I’m too old for OSCON.

While most people took off for dinner, the Stonehenge staff got together for a lightning meeting, which we try to do twice a year. I hate meetings that go over 45 minutes, so we got everything done in 30 minutes. We had to get everything done in 30 minutes so we could make it to the O’Reilly User Group reception to hang out with all the cool user group leaders, eat free food, and drink free drinks. Tom Phoenix (Stonehenge Consulting), Ricardo Signes, and Ronald Kimball (Boston.pm) played Zendo, a game of inductive logic in which the Master demonstrates a rule and the players have guess the rule by demonstrating it. Okay, it’s really fun even though my description sucks.

Our game was interrupted by a real fire and a staff member telling us to leave everything behind and go to the elevators. I told everyone to take their stuff with them and use the stairs. We continued the game of Zendo outside.

Then it was time for the Tuesday evening festivities. David Adler (New York Perl Mongers) gave out the White Camel Awards to Stas Bekman, Eric Cholet, and Andy Lester. Larry Wall talked about the personalities in spy movies. He says Perl was almost called “Spy”. SPECTRE is not a good spy name, but CHAOS is. Everyone fits into the spy ecosystem somehow.

Paul Graham talked about how business try to control time and space. You have to be in this cubicle at this time to this time. Additionally, here are some meeting places and times to break up your day into unmanageable little bits. He claims that whole concept is set up not to get work done but to control employees. He also says MBAs aren’t the right people for new business. You need to “create business”, not administer it. Worrying about the color of the office walls or the type of chairs to get aren’t the right thing if you’re just starting out. This sounded a lot like Joel Spolsky’s “Hitting the High Notes” (which has an opera example involving the high F in Queen of the night from The Magic Flute, which my wife can actually sing (with ease ;) ). Maybe Paul will post the complete talk online (and indeed he has) so you can read it yourself and not judge it by my mostly incoherent babbling.

All of that was just a lead up to Damian Conway’s “Dead Langauges” talk, which was about half in Latin. Seriously. He demonstrated his evolution through Lisp, Postscript, C++ (and his rewrite, SPECS). He ends up with his program that translates Latin into Perl programs. Damian doesn’t much like the syntaxes languages offer so he keeps creating his own to fit over them. When Iu get a chance I’ll translate one of his C++ trick to Perl.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7457

Writing of any kind is hard work. Writing a book is extremely hard work, and may be grueling pain all day long for 2 months straight, or a few hours every day for a year and a half. Your significant other and/or family will forget your name, your fingertips will bleed, and once the book is out you have to pray for good sales (or a nice job offer) to make it all worthwile.

And it doesn’t end there — you’ll have to keep track of errata to feed to your publisher in a hurry when they give you a couple days notice before a reprint cycle (if they’re a good enough publisher to correct errata between print runs at all). And then you’ll have to get started on the next edition, and start all over with the pain and alienated family — and at this point, possibly alienated employer.

Still interested in writing a book? Great! O’Reilly wants you. To help you along the way, Mike offered several suggestions:

• Know your material. If you don’t know it cold, you won’t write a good book about it, for many of the same reasons Damian tells potential presenters to only speak about subjects they know well.
• Be passionate about the material. Nothing is worse than a dry book, and if you get known as a boring writer, you may find it harder to sell another book down the line.
• Write about something new. Either have a very unique presentation, or be the first with a good book in a growing field. O’Reilly doesn’t like to glean a few sales from a saturated market, and you’ll kick yourself if months of hard work led to meager sales because you picked a subject that’s been beaten to death.
• Be realistic about your writing skill and the schedule you will need to keep. Write a chapter or two to gauge the time and effort involved, extrapolate that out to the full chapter count, and then assume you may need 50% over that.
• Write all the time (preferrably every day) to improve or at least maintain your writing flow and speed. If you let days or weeks pass between writing, you will fall back to writing slowly again, and that will kill your schedule.
• Be aware of the strongly seasonal nature of book sales. Ask your editor when are the best times to release books in your field, and hit that mark. Releasing a relevant book just before a conference can be a huge boost to sales.
• Total sales are all about momentum. Get buzz going before the book is finished, blog about it continuously, release it at a good time, convince people to buy the book from Amazon to move it into their bestseller lists, whatever it takes. Once you’re a bestseller, you tend to stay there, but it’s hard to get there in the first place.
• O’Reilly seems to have decently author-friendly policies, but how much you earn depends on you. Make a book that is excellent, price it appropriately (Mike mentioned that O’Reilly has recently been realizing they produce much better books than other publishers selling for the same or more; this will change), get momentum and keep it, and you will get 10% of O’Reilly’s wholesale.
• O’Reilly’s wholesale discount is usually 50% of list price, but it can be significantly less than that, all the way down to 25%. Book stores generally want a larger discount on a more expensive book (because it gives them greater profit per book), but that can actually be a net win for the author as well.
• If you want to shrink the discount, and therefore get a bigger cut of the retail price, you have to write a book of very high quality and enduring value. Of course, those are the same things that lead to greater unit sales anyway. Take advantage of the chance to improve your take twice over by doing an excellent job up front.

Above all these individual recommendations there is one overarching one: Talk to your editor. They have a lot of experience, and are there to help you create a bestselling book. Don’t have an editor? Introduce yourself to one. Several O’Reilly editors are at the conference, no less.

To all of you willing to try, good luck!

What other recommendations do you have for aspiring authors?

Related link: http://www.ukuug.org/events/opentech2005/

Last weekend saw a “very British” conference - the href="http://www.ukuug.org/events/opentech2005/">backstage.bbc.co.uk Open Tech. Aimed at
people who enjoy spending their Saturday travelling to West London and listening to people talk
with passion about what they’re working on.

Over the course of the day, a wide href="http://www.ukuug.org/events/opentech2005/schedule">variety of sessions were
presented. From
Danny O’Brien’s updated and entertaining Living Life in Public talk (following on from Life
Hacks), to presentations on software and culture, and exploding thigns from James
Larsson.

The headline event was the official launch of the href="http://backstage.bbc.co.uk/">backstage.bbc.co.uk
developer network; giving anyone the chance to remix BBC content in a free
and open fashion. Followed up by Jeremy Zawodny
talking on “why the future is open, or should be”.

Also in the day was the announcement of “Free Culture UK” - the
evangelist??? companion to Creative Commons UK. The building of systems and
structures around the CC movement in the UK is gaining pace; especially as
the BBC Creative Archive moves forwards to illustrate the potential.The
“ href="http://www.ewanspence.com/blog/2005/07/24/the-apple-ipod-shuffle-shuffle-from-opentech-20
05/">ipod shuffle shuffle” (take 4 ipod shuffles+owners and give a random owner
a random shuffle) was the talk of the event for a couple of hours, with
audience reaction being a mixture of shock and amusement ( href="http://www.chrisgreen.co.uk/43">mostly).

One eagerly awaited (and packed) session was a panel discussion entitled
“where’s the British
EFF?”. Covering what groups currently exist, what they
do and what past groups did, and should there be a British EFF? With the
weight of support from the room, shortly after that session ended, a signup
board to let you add your name to donate a fiver a
month as supporting
membership appeared on pledgebank (The site where
you can say “I’ll do X, but only if Y people will too”, from those nice people at href="http://www.mysociety.org/">MySociety). href="http://chocnvodka.blogware.com/blog/_archives/2005/7/28/1084210.html">More href="http://news.bbc.co.uk/1/hi/technology/4724089.stm">details.
Less than a week after the idea was announced, 500 people have pledged their support and money.
You can too…

BBC Research & Development were showing off href="http://dirac.sourceforge.net/">Dirac, their “free as in
totally” video codec, Kamaelia their simple,
concurrent, networking infrastructure; and other groups with innovations such as a href="http://promise.tv/">PVR capable of
recording all digital TV stations for a week. With an enlightening talk,
running demos and the prescence of a number of engineers to discuss their
work, this was one of the more dynamic, and popular, technical stands. Of
particular interest was a printed document ( href="http://kamaelia.sourceforge.net/GrandChallenges_final.pdf">PDF) being given out by
BBC R&D
entitled “Grand challenges in Online
Research“. Taking one challenge per
page, it provides an inspiring overview of an issue, why it matters, and
what BBC R&D is doing to meet that challenge. It’s very significant reading
and provides some indication of what issues may need solving for large
scale online distribution, and all the tangentially related problems from
heavy infrastructure to software on the desk. It’s href="http://kamaelia.sourceforge.net/GrandChallenges_final.pdf">worth reading.

All in all, a very successful day, with two href="http://backstage.bbc.co.uk">launches, one
potential big project and href="http://www.boingboing.net/2005/07/25/promise_tv_pvr_recor.html">several href="http://www.cubicgarden.com/blojsom/blog/cubicgarden/socialsoftware/offline/?permalink=ope
ntech2005.html">revelations, and a number of smaller ones coming out of it. href="http://www.ukuug.org/events/opentech2005/recording/">Recordings of most sessions are
now online if you want to hear what happened.

Other writeups from href="http://www.ewanspence.com/blog/2005/07/27/opentech-2005-thoughts-after-the-event/">Ewan
Spence, Dave Cross, and the full
list on technorati and photos on href="http://www.flickr.com/photos/tags/opentech">flickr.

Dave Green and Etienne
Pollard deserve a special mention for the vast
amount of work that they did on the event; and thanks to everyone who
helped in any way, large or small, with the event.

For my regular reader, “This Month (or two) in BSD” will return in
September after a summer break - July and August
will be covered together.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6440

This year, Damian had over a dozen new or greatly improved CPAN modules to discuss, so only had time to touch on them lightly. The following is a quick list, for those that want to investigate further:

IO::Prompt

Improved in several ways, including a new menuing feature

IO::Interactive

Handle all of the special edge case testing to determine if the user is interactive, or piping information in or out; also, provide a special filehandle that will either print or blackhole, depending on the interactive state

IO::Busy

Deal with users who try to type when they shouldn’t during long-running tasks

Log::StdLog

Just do simple logging, by providing a STDLOG filehandle that just does the right thing

Debug::Phases

Indicate beginning of Perl’s compile and run phases, and report time spent compiling

Sub::Installer

Handle symbol table and strict/warn manipulation for you when you want to just do your own exporting

RegExp::MatchContext

$PREMATCH, $MATCH, and $POSTMATCH, except faster and better

Contextual::Return

Declarative wantarray

Perl6::Export::Attrs

Perl 6-like export syntax for Perl 5

Config::Std

A somewhat improved .INI format, and the ability to preserve comments and ordering when updating a config file

GetOpt::Euclid

Generate command line options parsing from the documentation

Toolkit

Have all the modules and imported functions you want, all the time, with just a single use

Which new Damian module is your favorite?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6378

Greg managed to fit a lot about historical issues surrounding release engineering and source control for Linux in his 45 minutes, and still had time to explain the various solutions the kernel team has tried. I’m going to just give the highlights of the issues before relating the current ways that the kernel team is trying to fix these issues.

Up until 2.5.3, Linus was accepting all changes to the Linux kernel only as emailed patch files, and eventually the poor scaling of this came to a head. At that point, he switched to using BitKeeper, a proprietary distributed revision control system, and while the license made people crazy, processes improved significantly, making kernel life generally better for a while. But the license issue would come back to haunt them when BitKeeper changed the license in a way that the kernel team could no longer work with.

At the same time, release engineering was pretty poor, and as the time between successive releases began to get longer and longer in the 2.6 timeframe, end users and distributions started taking exception to the wait (especially for security fixes).

Over the past few months, the kernel team has implemented several new tools and processes to deal with the outstanding problems.

First, a bug fix patch-only tree was created that had very strict requirements for accepted patches. This was used to produce 2.6.x.y patch releases to fix critical issues only.
Each .y patch series is dropped and recopied from the mainline with each .x release. Each .y series usually contains around three orders of magnitude fewer changes than are in a .x release, making people feel very safe in following .y patch releases.

To deal with the problem that .x “release candidates” were nothing of the sort, a new policy was created recently that after each .x release, a one-week window is opened for changes (the “patch flood”). After that week, rc1 is released, and really is a release candidate. After that point, all accepted patches must be bug fixes only, with no new or changed functionality. Once bug fixes die down, a new .x release is done, and the cycle repeats.

Finally, when the BitKeeper license changed, Linus and crew were left holding the bag, so they investigated the available options and found them all lacking. There was only one thing to do — they created their own. More than one, actually. The main one right now is git, but some subsystems are using Mercurial instead. Greg mentioned others, but I have forgotten them.

These new SCMs share two major attributes — they are lightning fast at importing patches, and they are distributed by design. They are also all very young, and Linus has said that in another three months he will see how far each project has gotten, and choose a new one for himself (and presumeably his core team).

Switching to a whiz-bang mode, Greg talked about a number of cool things recently merged or soon to be merged into the mainline kernel, including the Xen virtualization technology, lots of new file systems, improved internal APIs, and so on. He also proudly announced that Linux now supports more devices on more platforms than any other operating system ever (Linux passed NetBSD last year, an impressive achievement). In fact, there are now a number of operating systems that directly use Linux drivers so that they won’t have to recreate the whole driver corpus.

Finally, he talked about stability of APIs. Internal kernel APIs are never going to be stable, but external APIs should remain so — though he admitted that this only applied to syscalls, not to sysfs and procfs; stability of the latter is a subject of discussion these days. He pleaded with vendors to get out-of-tree drivers into the mainline so that they can be magically fixed every time internal APIs change, and pointed people to the stable_api_nonsense.txt file in the tree for more details.

I asked if these changes meant that a 2.8 series may never come, and he said that the new processes were forcing developers to do a much better job, no longer ripping out and replacing humongous chunks of code, but rather incrementally improving things until each major change was completed. They are discovering that they may not ever really need a new “pure development” kernel series, just more happy 2.6.x releases for years to come.

As an end user, that’s just fine with me.

What do you think of these process and tool changes?

Related link: http://conferences.oreillynet.com/os2005/

The software developer is the hero of our day. This is not only
because of what developers have accomplished, but because of their
integrity and transparency. They can’t fake what they do.

A developer can’t make his code perform better by ingesting dangerous
medicines. He can’t improve benchmarks by carrying out shady
accounting deals. He doesn’t get anywhere by wearing glitzy clothes
and appearing on TV shows after two-hour make-up sessions. He’s not
likely to make his project progress by splitting and polarizing a
population with appeals to deep-seated fears. We’re disgusted by all
the other public figures society has thrown at us; only software
developers are pure.

The developer as hero was celebrated several times over in the morning
keynotes at href="http://conferences.oreillynet.com/os2005/">OSCon. Tim
O’Reilly and conference coordinator Nat Torkington shared a
presentation celebrating the progress of programmable Web
services, popularly called Web 2.0. Tim pointed out that major Web
sites such as Apache, Google, and eBay have provided programming
interfaces (Web Services) for a while, but that now we’re seeing “the
first true Web 2.0 applications”–meaning applications that create
something new rather than just enhancements to the underlying web
site. He illustrated the trend with a hack that combined craigslist
with Google maps to show real estate offerings for given
neighborhoods.

But another set of heroes emerged in the next keynote, given by Kim
Polese of
SpikeSource.

SpikeSource and the ascent of testing

Polese started with some crackling one-liners illustrating the trend
she’s seen in the evolution of open source: From DIY to DIT (do it
together). From Egosystem to Ecosystem. From an architecture of
participation to building of participation.

Basically, Polese said businesses have come to accept open source and
are performing their own integration in order to get it to work for
them. She recognized and articulated a concern I’ve had for some time,
and which by chance I tried to turn into a blog last week but never
quite pulled together. So I’m going to plunk down that
not-yet-published blog right here and then return to her view of the
situation.

The lament of the Linux knowledge worker

I’ve been living with and struggling with a mental disconnect in the
free software arena for several years.

On mailing lists, news sites, and trade journals I read glowing
reviews (or anticipations) of wonderful new packages and support for
the most advanced computing uses. Then I return to my little desktop
systems, where my wireless card and power management still don’t work,
my GNOME desktop often thinks my left-most mouse button is depressed
when it isn’t (leading to wild errors), and remote file transfers have
been crippled by an apparent bug in the latest distribution. (I’m
always trying new distributions in the hope of finding a happy moment
where some version has all the critical functions I need. The closest
I came was a year ago when I paid $99 for the first version of Red Hat
Professional Workstation.)

These are just the problems that hold my productivity back every day;
there are others I haven’t bothered to list.

I’m not surprised at the configuration problems on my system, and I
don’t blame anybody. It’s the fallout from open source development.
I’m getting the kernel from one project, the desktop from another
(both KDE and GNOME–I’m desktop fluid), drivers from still other
projects, middleware such as the ESD sound daemon from their projects,
and of course, applications from their various producers. No matter
how stable and carefully developed all this software becomes, getting
it to work together is hard.

GNU/Linux is the most volatile of the major operating systems. That’s
partly because it’s the youngest (certainly compared to Unix and BSD,
and even compared to NT and its derivatives). But it also attracts a
user community that’s always pushing it ahead; this is what brought it
to the astonishing success it has today.

Most major distributions have come to face the tension that rapid
development produces between the goal of widespread adoption and the
goal of continuous improvement. Debian split into a stable and
unstable distribution, Red Hat split into Red Hat Enterprise Linux and
the more community-focused Fedora project–even Gentoo is considering
an Enterprise version.

Maybe I should just go ahead and pay $99 a year for a commercial
distribution, but I think there would always be something out of sync.
You can never rest with the system you’ve installed: before the CDs
were even shrink-wrapped, some of their contents had become
out-of-date, and the user must routinely update a system for the sake
of security as well as functionality.

In his keynote today, Andrew Morton (former Linux kernel maintainer) pointed out that the readiness
of Linux for the desktop is routinely underestimated because the
people who evaluate and report on it are the power users or knowledge
workers. Naturally, they have the most varied needs and are the
hardest to please. He repeated the generally accepted view (see, for
instance, my

report from the Desktop Linux Conference)
that desktop Linux will start with kiosk application, move up to people
using one or two major applications, and gradually continue up the
knowledge chain as it appears in more and more places and companies
have reasons to enhance it.

So I wonder whether the computer industry will pass through a period
during which people see it as prestigious to have a PC running
Windows. (”I see you’re stuck with Linux, Jones, but after all, you
don’t have documents to post through SharePoint.”)

More on Morton’s talk
later in this blog.

The SpikeSource approach to free software inconsistency

Large organizations like open source, but they find that it takes a
lot more internal support, especially for precisely the problems I
encountered on my desktop: the integration of components that weren’t
previously tested together. According to Polese, companies are
internally creating “formalized processes” such as review boards,
support desks, and incubation centers. But it’s time-consuming and
takes away attention from whatever their core business is.

SpikeSource aims to improve the situation by recruiting volunteers to
do this drudgery, and providing better tools to help them do it. To
see whether they were really following through on this tall promise, I
visited their demonstration center, which was quite fancy (it
shouldn’t be a surprise, given that they’re partnering with O’Reilly
on
CodeZoo).

Their work did look pretty spiffy. Their download and configuration
utilities for selected stacks of products, such as the servers and
language tools in a LAMP system, were very easy to understand and well
integrated. Their automatic test generation utility for Java code did
a very full job (producing comprehensive code coverage in a set of XML
files) but it seemed to me it could serve only as a first step for
programmers who customized the XML to run the tests they really need.

More keynotes

The morning was rounded out by several other presentations, each
interesting for very different reasons.

Andrew Morton

Morton gave a short speech in a manner so low-key and unprepossessing
that one could miss his irony and incisiveness unless one listened
carefully. He struck a balance between praising the contributions of
the large companies participating in OSDL (the company that pays his salary, as they pay Linus Torvalds), and acknowledging the
importance of the wider free software community. The latter folks are
often suspicious of OSDL, rarely for any well articulated reason.
Morton labeled the OSDL member companies “blundering elephants” but
insisted they were dealing sensitively with the Linux community.

The same winning cynicism came out when he explained why Linux was
spreading everywhere. He was almost dismissive is saying the operating
systems and other infrastructure had become “commoditized,” that their
implementation was “common knowledge,” and that they “have no
intellectual property value left in them.” (So why was there so much
argument about implementing features at the
Ottawa Linux Symposium
I
reported on
two weeks ago?)

He also claimed that free software developers are “significantly
better programmers on the whole” than their equivalents in proprietary
software companies. But he pointed out that OSDL member companies
contributed very important developers to Linux.

He made three major comments on the development process. (He wrote me email to help clarify his statements.)

• First, any major kernel developer practically has
  veto power over any decision, but they manage to move ahead
  nevertheless.

• Second, he validated the complaints of some people in the wider
  community that Linux was being loaded down with enterprise features.

• He said that the free software community was not good at collecting
  input from end-users (by which I assume he meant unsophisticated
  end-users among the general public), and that companies had processes
  in place for doing that.

Jeremy Zawodny, Yahoo!

Jeremy gave a brief talk on two themes: how Yahoo! makes use of free
software, and what it’s doing these days.

The role of free software at Yahoo! is pretty much the same as in
other organizations. They have lots of servers, and lots of desktop
users running the usual office suites and other things. The servers
mostly run FreeBSD, but they use Linux to run commercial software
packages because these tend to be ported to Linux rather than FreeBSD.
Jeremy reeled out the usual list of benefits of using free software:
flexibility (it’s easy to change), quality (it’s reliable), good
documentation (including often the source code), availability, and
support on community forums.

He said not to worry whether free software is scalable. Yahoo! is handling billions of page views per day, and a hot item can get a million views in the hour after it first goes up.

He also promoted Yahoo! for opening up its APIs with the goal of
becoming a full-fledged participant in Web 2.0. The Yahoo! Developer
Network and Yahoo! Widgets (originally Konfabulator) are the main
exhibits.

Jonathan Schwartz, Sun

I was nervous about attending Schwartz’s presentation, partly because
I’ve been annoyed by a lot of his published statements (as I pointed
out in
one blog
which happened to be written when I was feeling relatively kindly
toward Sun) and partly because I was afraid of what the audience would
do to him.

Everything turned out fine: Schwartz was personable and
natural-looking, and Nat Torkington carried out an informative
interview. Nat gave Schwartz plenty of room to make his points, while
lassoing him on his occasional slipperiness, such as sliding from
discussing “free as in speech” software to “free as in beer”
software. Another example was an exchange about software patents,
where Schwartz played to the audience’s abhorrence of the patents
(saying they’re “largely inapplicable”) while allowing that patents
could be used both to promote innovation and to squelch it.

I would have liked an exploration of Sun’s relationship to the
OpenOffice.org project and their controversial decision to include
some Java code in it.

I was impressed with Schwartz’s defense of Solaris. He said Solaris
should not be compared to Linux (a software project) but rather to
particular distributions of Linux being offered in the marketplace. I
can acknowledge his proud claim that OpenSolaris could provide
competition that spurs other free operating systems to improve. He
also vigorously defended OpenSolaris’s CDDL license, saying their
license is open to linking with other software and that it’s the GPL
that it is prohibiting collaboration. And he claimed that a large
community of contributors were flocking to OpenSolaris.

He said that “just because the software or service is free doesn’t
mean it doesn’t have value,” and pointed out that it can provide value
by becoming a substrate for support, update networks, and ads. He also
boldly declared that “ultimately, every single product at Sun will be
free and open”–but of course, his definition includes the current
state of Java.

Notes on other sessions

I got around to at least parts of a number of sessions today.

Theo Schlossnagle on database replication

During the part of the talk I attended, Schlossnagle made the
interesting point that database administrators really want to
replicate only the results of an update, not the update itself. He
used, as an example, a complex query that involves a huge search
bringing in multiple databases, potentially requiring hours–but
ending up with a change to only five records. I didn’t stay to find
out whether he had a way to propagate the five changes rather than to
rerun the update.

Ted Leung on Chandler

I sat with about 18 attendees for the beginning of Leung’s
demonstration of how to build a new component (called a “parcel,” just
to be different, I suppose) for the emerging Chandler office suite
from Mitch Kapor’s
Open Source Applications Foundation.
I didn’t find out the answer to the main question I’ve had all along
about Chandler–why? Why start a whole new project from
scratch to produce yet another office suite?

I can guess the answer, though, from the little that I saw. I imagine
that anyone who learns Python could extend Chandler (the point of the
talk) and therefore provide new features to the community fast. The
parcels are supposed to allow easy integration with Web Services and
other web sites. Chandler’s visual components remind me of XUL.

Some general notes

Various O’Reilly staff and Safari staff held well-attended sessions
today. I was pleasantly surprised by the interest attendees had in our
work and thinking. But I realized it made sense, given the strong
interest in search engines nowadays. The very same needs–that is, the
aspiration toward reliable sources of knowledge in this chaotic
information world–also leads people to look for high-quality
publishers.

But there are others making advances in the publishing industry. One
up-and-coming organization is
Osoft,
makers of ThoutReader, whose extraordinarily clever designer Gary Varnell

posed with me in a photo
today.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6441

Perl 6 has been in planning and design for 5 years now. During that time the design has been almost constantly improving. Finally the Cabal is seeing a light at the end of the tunnel, and portions of Perl 6 have even been implemented (for details on that, see pugscode.org).

Still, the last year has seen many refinements and refactorings in the design, and a few new features. Amongst them are:

• Massively DWIMier and more powerful string quoting and interpolation
• Generalization of several useful operators, with for example the trans(literation) string method handling multiple before and after pairs at once, and allowing single characters to transliterate to longer strings
• The fish operator, =<>, which iterates over the command line arguments in the same way that Perl 5’s diamond does while still maintaining the wonderful laziness of Perl 6 filehandles-as-generators
• The reduce meta-operator, which allows such snazzy code as $sum = [+] @nums and $first_defined = [//] @vals
• (edit: I originally had an item from Damian’s Perl 5 modules here)Powerful add-ons to return with but
• . . . and several others that I missed as I took notes and Damian rushed to fit things in

Progress of the various Perl 6 implementations was covered in another talk, which I skipped in favor of an update on the Linux kernel (see my next blog entry). I already work on the Pugs implementation, so already know that it’s going quite well indeed. New people are always welcome, as Autrijus has managed to prove that anarchy can produce great software — just ask on irc.freenode.net #perl6 for a committer bit, and join in any way you like.

Why aren’t you already contributing to Pugs?

Related link: http://conferences.oreillynet.com/os2005/

I waffled a bit when deciding whether to write about the evening “extravaganza” last night, and the morning keynotes today. Eventually I decided that while I didn’t have much to say about any individual talk, there were some common threads worth mentioning. So here I sit, writing for you instead of playing Robotron at Randal’s annual party. Feel my sacrifice!

First, a lot of awards were given to various deserving people, but there’s something just plain odd about listening to several completely separate awards ceremonies, most without much rehearsal, recipients who actually were there, or in one case anything to give out (because the awards got shipped to the wrong place). It would be nice if they had planned things a bit better and actually did all the awards at once. Perhaps even (perish the thought) told recipients that they had been nominated and might win, so that they would bother to show up.

Hmmm, perhaps that’s something to put on the conference feedback form so I can try to win one of the Gibson guitars or bits of computer equipment being given at random to people who submit their feedback. Quite the incentive, that. I wonder if we can convince a car company to be a major sponsor next year . . . .

Second, many of the speakers spoke of coming to a new level of network effects in the Open Source world. Depending on the personal biases of the speaker, each described this differently, but fundamentally it seemed they all agreed on a few things. Basic infrastructure is there, they all said, and now all the cool people are doing stuff on top of it. No longer do people have to worry about how to simply get computers to speak to each other. Energy that’s no longer spent trying to fill in the basic foundation for modern Internet living is now used to create user-centered web technologies (del.icio.us, for example), services companies (SpikeSource), enterprise software (SugarCRM), and so on.

The key point is that several of these new creations absolutely depend on social and network effects. The value in del.icio.us comes directly from its users; SpikeSource would be untenable if they couldn’t get help designing tests and fixing bugs from hackers on all the projects they work with. New web sites and web tools are popping up right and left that extend these network effects beyond just one closed world controlled by a single company (eBay, for example), whether the corporations involved like it or not. The Greasemonkey Firefox extension allows people to create tools that force companies to network — the Google Maps / craigslist user script that makes every street address bring up nearby housing offers is just one example of this.

Tim O’Reilly, both in his keynote and in the fireside press chat, seemed to be spending a considerable percentage of his neurons trying to puzzle out where this is going, and O’Reilly has created several websites recently playing around with various concepts surrounding network effects, information markets, consensual filtering, and so on.

From the outside at least, it seems that Tim is doing what Paul Graham spent some time discussing in his speech — trying to get in on the next big thing not by assigning employees to work on various old boring projects, but letting people try the entrepeneur thing by launching a completely new wacky concept to see where it goes. I can only hope Tim will let the lucky people who come up with the next big thing share in the wealth.

Finally, a few old traditions came through again. Larry twisted another metaphor to the breaking point (and might I say that his kids have a great start on the spy card game they are creating). Damian twisted minds and code in 5 dead languages (Lisp, PostScript, C++, SPECS, and Latin), which somehow involved dozens of scary pictures of Russian Lara Croft imitators. Jonathan Schwartz frustrated another interviewer (Nat) by refusing to be led into battle on several topics, and even said some good things amongst the usual corporate officer evasions. (I think corporate officer evasiveness may actually be an SEC regulation.)

All in all, a fun bunch of talks.

It’s day three here at OSCON. The new venue this year (Oregon Convention Center) is beautiful and spacious, but I do miss a certain vibe from the previous two OSCONs at the Marriott in downtown Portland where everyone attended and stayed to congregate in the same places. The sessions were very crowded last year though, so the venue upgrade was inevitable.

Hot topics and observations so far:

• Perl is dead, long live Ruby (on Rails), PHP, and Ajax. Personally, I’m not buying this theory just yet.
• Lots of buzz about new O’Reilly sites: O’Reilly Connection, CodeZoo, and Make.
• It appears there are a lot more females in attendance this year, which is very encouraging.

My lovable boss, Tony Stubblebine, has a nice write-up about the response to the O’Reilly Connection preview on his blog.

CodeZoo has announced new components for Python and Ruby.

Related link: http://use.perl.org/~Limbic%20Region/journal/26062

Highly productive hacker Autrijus Tang started the Pugs project in February and, with a little help from his team of lambdacamels, has implemented a huge portion of Perl 6 in six months. #perl6 participant Limbic~Region has just posted an interview with the fearless Autrijus, now that his self-imposed six month sabbatical has ended. Don’t fear; Pugs and Perl 6 will continue… perhaps with your help.

Have you used Pugs yet?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6989

Imagine having a burglar alarm that could detect motion in your living room, then immediately call you, a nearby family member, or trusted family friends who could investigate — in that order, the first person who might answer the phone.

Maybe that doesn’t sound very impressive, but consider that you can build this system for the price of a webcam, a telephony card, and a little bit of work setting up and configuring Asterisk. It’s even more impressive if you can
plug a phone into your laptop in a hotel conference room, flip on a light in your living room remotely with X10, and receive the phone call even before the web browser updates the view from the web cam.

That’s what Brian Capouch did in his Tuesday afternoon OSCON 2005 tutorial on understanding Asterisk. This impressively large and featureful piece of software is the brains of a telephone system. There are a lot of terms and
ideas to learn, but if you have some experience with telephony (or are willing to learn), the software is stable and sensible.

Though there are a lot of boring applications (if replacing hundreds of thousands of dollars of proprietary PBX devices and software with a commodity PC and open source software is boring), there are a lot of cool ideas
too — building a custom voice mail system, having nearly unlimited extensions, having phones that don’t care where you are in the world, but always ring if you have an IP connection. My brother wants to put a soft phone in his son’s room and another in our parents’ house, so that Jacob has a direct line to Grammy and Grampy.

If you’ve done telephony before, you know that the fairly useless telephone companies have predatory pricing. You do have to spend a bit of money for hardware (unless you use software phones and pay someone else for access to
the standard boring telephone system), but Capouch revealed that the newer Linksys WR54GS wireless devices (and especially the NetGear versions) run Asterisk nicely. Adding a small USB device and upgrading the antenna or the
wireless card can give you up to ten miles of wireless-enabled calling — perhaps for $250.

Installation and configuration are still more involved than the average phone user might want, but complication has a way of vanishing as new contributors and businesses start to expand their audiences. It’s hard to imagine how the
current system will last, in the face of convenience, cost, and customizability. VoIP is coming.

Oh yeah, Brian Aker has a whitelist of people who can call him between 11 pm and 7 am — with Asterisk. What would you do?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7077

Kathy Sierra’s Creating Passionate Users is the only tutorial I took that really could be classified as training, rather than a lecture or Q&A. In a flashback to days spent in corporate training classes, the students were broken up into teams and asked to work on exercises several times during the 3 hour session.

It’s difficult to figure out exactly what I liked about the class — perhaps it was that Kathy’s obvious enthusiasm for applied cognitive science was infectious. Perhaps it was the very interesting opportunity to view geek addiction from the other side — trying to make it happen in your user community.

Right there is one of the keys — people love to form communities. Don’t just let them, help them. This doesn’t mean some lame forum section of your website (although certainly that can be one method among many). This means forming the same sort of tight-knit communities that surround products as varied as role playing games and horse training.

To form a rich, vibrant community, the product has to have a long term richness and depth, so that a progression forms from newbie to master. As in various programming communities, a rich ecosystem of knowledge will form around this progression. Some users will spend all of their energy climbing the ladder of mastery; others will act as mentors, or evangelists, or trivia collectors, or what have you. The more depth a subject shows, the richer the community can be.

The ladder of skill is itself important; as game designers know, many users just want to be able to reach the next level to be able to say that they have. “I’ve got an 80th level ranger, and I’ve played through the entire Fubari world!” Other users want to have some new skill or power when they climb the ladder. In games this may be a new weapon or special ability for one’s character. In other fields, the user herself is gaining the ability: Kathy mentioned being an amateur photographer and being proud to learn the technique that makes waterfall photographs look blurry and surreal.

Some companies choose to form a community with skills very closely related to their product, such as Nikon teaching photography techniques — some of which can’t be done without buying one of their more expensive cameras. Others choose to form a community around something only peripherally related to their product. Red Bull knows that DJs and dancers probably use their product to stay up for those “up until they close” dancing nights, so they decided to create what is apparently a pretty good DJ university.

Even though apparently unrelated, this can still benefit your product — people tend to associate emotions between unrelated things they think about at the same time. The fun and excitement of learning to be a great DJ will be associated with Red Bull; the dancers on the floor will eventually notice that the DJ perpetually has cans of Red Bull next to the turntables, and so on.

Kathy’s key point is that “The brain pays attention to that which it feels.” Without emotion, the brain’s chemical crap filters pretty much wipe all memory of something happening (without the mechanisms that cause this, research indicates that we would all annoyingly never be able to forget anything). And if nothing else, rich user communities are full of emotion.

The key then is to make your users into the kind of people who love being in the community so much they will happily post thousands of pictures of themselves holding your product (an iPod, say) in front of every landmark on the planet. Or trade every last bit of trivia about your product (I’ve known Deadheads that amassed room-filling libraries of bootleg recordings, trying to have one or more copies of every concert the Dead ever performed). Or spend countless hours mentoring everyone they possibly can to bring them deeper into the community (PerlMonks is a good example of this).

It’s this kind of irrational, unbridled passion for something that makes sure it cannot fail (barring mind-bending corporate incompetence).

And after all that, I’ve only described one of the threads in Kathy’s talk. I’ve suddenly got an urge to spend mounds of cash on cognitive science books so that I can climb the ladder in the passion-creating community . . . .

What’s your passion?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6436

Damian Conway’s Perl Best Practices tutorial is actually a full day class (and could clearly have been considerably longer than that). I took only the morning session because there were too many good tutorials Tuesday afternoon, and frankly if I liked the morning session, I could always buy the book.

As it happened, I picked up a copy of the book after signing up for OSCON tutorials, but before I flew to Portland. This meant that I had already had a chance to read through some of the material before coming to class, and mull over some of Damian’s recommendations in greater depth than possible during the presentation itself.

I’m somewhat split on Damian’s recommendations so far (the ones found in the morning session and the first 7 chapters of the book). First off, I’m pretty sure that someone who follows the entire recommendation set (all 256 of them) is considerably less likely to write horribly unmaintainable write-only code. As a first approximation to improve the lives of those on your team, just taking the entire set of recommendations at face value is not a bad start.

But that’s only a first approximation. Many of the recommendations are excellent, some are so-so, and a few seem completely wrong to me. As Damian pounds into everyone’s head, every team should take the time to really think about each recommendation and decide if it is right for them, if they could at least live with it for the sake of consistency, or if it needs to be changed to fit the environment. If I was making a team coding style recommendation today, there are few items I would drop or change, with perhaps a few added.

For example, Damian would rather programmers never use unless and until — and heaven help you if you want to use either one as a postfix statement modifier. Sure, I agree about until (I find it’s confusing unless the boolean test is extremely simple), but statement modifiers are precisely the case when I love to use unless.

next PROSPECT unless is_interesting($first_date) just makes sense to me, and ugly replacement constructs like if !is_interesting bug me. Damian countered that the real answer was to write a wrapper, sub not_interesting { !is_interesting(@_) }. I object to this on the grounds that the latter is inefficient (an extra subroutine call with no actual work done) and represents a negated boolean subroutine name, which I find at least as onerous as unless ever could be. He would probably point out at this point that the subroutine name only appears once in a given statement, but a boolean expression tends to get longer and more complex over time, and programmers are lousy at using De Morgan’s laws correctly in a negated context — so our mutual objections aren’t really parallel.

I guess we’ll have to agree to disagree. For what it’s worth, Damian notes that his anti-unless stance is one of the most controversial in the book, and a number of expert Perl programmers disagree with him. It’s nice to be in good company.

If you think this deeply about every recommendation, even if you finally decide you don’t agree, Damian has set out to accomplish his primary goal anyway. The most important part of best practices (in Perl or elsewhere) is to take the time to really think about how and what you do from day to day, decide which things work and which things don’t, and apply evolutionary pressure to the habit pool.

On the whole though, I suspect with some soul searching you’ll find that most of the recommendations are pretty sound — I for example plan to change my line-wrapping style the next time I touch any code. Pick up the book (it’s got a lot more detail than the tutorial), think about the recommendations, and as Damian puts it “Write a love letter to your future self.”

Which recommendations do you disagree with enough to ignore, and which ones convinced you to change your current practices?

I’ll talk about these later, but here’s the breaking news.

Stas Bekman - for mod_perl goodness

Eric Cholet - spam filtering for perl.org

Andy Lester - Perl testing and the Phalanx Project

Note: As I correct mistakes in typing, diction, noun choice, I’ll simply strike out what was there. You can use this to gauge how exhausted I am and how that changes throughout the conference.

Monday morning Randal and I made it down to the Oregon Convention Center around 10:30. Everyone was in their tutorial sessions so the place had the feel of a slepy airport. It’s spacious, mostly on one level, and there are several indoor shops, including Starbucks, Kinko’s, and a gift shop. There are some, but not a lot, inflatable seats and sofas along some of the walls, and a few small tables outside the Starbucks. It’s too bad that there aren’t more tables—the conference support for the “hallway sessions” isn’t there, or at least not yet. Some bigger tables, especially with power pulled to them), would sure help people sit next to each other.

I wander around a bit, find my press pass, get a speaker pass somehow, and also end up with a decent conference backuppack. Usually us low-life press and speaker types don’t get the bag or the cool stuff that comes with it, but I’m not complaining. Well, I could complain a little. The backup is tapered toward the top so It doesn’t fit my Powerbook. Oh well.

As I wander around I notice a lot of nice Dell monitors. That’s right: Dell, not Apple. Apple decided in one of their fits of cultish behavior not to sponsor things that aren’t them. There are no shiny Apple displays, the registration desks are missing the cool Apple displays, and there is no Apple terminal room. Actually, there might not even be a terminal room. I haven’t found it yet, but I haven’t looked that hard either.

A large group led by Schwern heads out for lunch. I guess the conference isn’t providing food this year (and I realize that I don’t see any snacks in the afternoon either). There are plenty of places close by, including which serves all-fresh-never-frozen food. Schwern’s group ends up there, but the line is already long so a group splinters off to go to Red Robin>, a burger chain only a short block away. The splinter group ends comprising Randal, I, Adam Kennedy (Perl Parsing, JavaScript Archive Network (JSAN), Perl modules list), Bill Odom (The Perl Foundation), Ricardo Signes (Rubric), Kevin Altis (Pythoncard.org), and one more Python guy I’m forgetting.

The conversation at lunch is good: it’s not the fistfight that people expect between proponents of different languages. The toughest part of the whole affair is the waitress who wants to ask too many questions, and pauses a bit too long between them. Just when we think she’s finished bothering us and we start talking again, she asks another question. It was so effectively distracting that I started to smell intent. No matter: once we got our lunch and she left us alone, we spent a lot of time talking about CPAN and JSAN, the new JavaScript Archive Network, and how all of that might apply to Python. The short answer is that these things work because there is no barrier to entry (i.e. no editorial control) and that the base layer is just a storage mechanism. Everything else is a separate project the build on top of (or to the side of) everything else. That way, you don’t have to have all the answers to start building the archive. At first it’s just the storage, and you have to be patient enough for eveything else to show up incrementally. All the CPAN add-ons weren’t built in a day, or even by the same people.

After lunch I set up camp outside the Starbucks in the convention center. It’s at the nexus of two major halls, so just about everybody has to walk by the table. Chip Salzenberg (Geeksunite and now Cloudmark spam ninjas) shows up and joins the camp, followed by Dave Adler (New York Perl Mongers), Ricardo, Jan Dubois (ActiveState), Gisle Aas (creator of libwww-perl), Josh McAdams (PerlCast, which is podcasting from OSCON), and many other people who join and part throughout the afternoon. It’s definitely the cool table (there are only six to choose from, so the odds where already good and we had already stolen the chairs from the surrounding tables, lessening their chances). That was the afternoon hallway session and many people wandered by with a longing look in their eye not realizing they could just pull up a chair and we wouldn’t mind (and I’d talk about that later in the “Conference for Beginner’s” BOF later that night.

The best thing about conferences are the things that don’t happen in the talks, known as the “hallway sessions”. A conference is all about getting people together, so it’s kinda silly to program their whole day to make them sit in a dark room and not say anything. Get out and met people! Seee the faces behind the screen names. Autrijus Tang and his Pugs crew turned conferences and workshops into multi-day hack fests. They discovered that although online collaboration is good, in person is even better. Unfortunately, the United States government is so paranoid about foreigners and air travel that Autrijus has given up being mistreated by our government and won’t come back to the US until we fix it (and that’s not looking good).

I skipped dinner, still digesting my lunch, because the Perl Trainer’s BOF started at 6. I had just enough time to take a backstage tour of the convention hall with Randal and Jim Brandt led by Ted, the guy in charge of the electrons for the facility. We got to see the network closet which collects allthe internal traffic and sends it up a radio antenna “pointed somewhere”. We saw where they store hundreds of chairs, tables, whatevers, and got to ride in the staff golf cart. It looks big in the user space, but it’s even bigger behind those locked doors. Ted was able to explain to me why my laptop wasn’t charging from the wall outlet in the conference hall: the janitorial staff run high powered vacuum cleaners from those wall outlets and don’t bother to tell anyone when they short them out.

Peter Scott’s annual Perl Trainer’s BOF (Birds Of a Feather) had 15 or so people, but this year a lot of the people where teaching in academic environments. Although we all introduced ourselves at the beginning, some people didn’t recognize Randal’s or my name, so when one guy started talking rather unfairly about “Learning Perl” (he hadn’t seen the latest or the previous edition, so was operating on very, very dated information, but maybe he didn’t see the press release that turned up on my RSS reader while he was talking—now that’s internet time for you), he not only didn’t realize we were the authors, but also thought we were O’Reilly editors and wanted to pitch a book. No biggee, stuff happens and I’ve done that sort of thing myself.

Following that, Jim Brandt led the “Conferences for Beginners” BOF. At YAPC::NA in Toronto he had noticed that about half the people had never been to a conference before. They didn’t quite understand the culture (Why is everyone looking at their laptops during talks? Why are all the laptops Powerbooks?) and that they were reticent to join in conversations or groups since they didn’t know anyone. This is our first time putting together a talk like this and we hope to do it at other conferences as a sort of training camp or kindergarten so that people can get the most out of their conference dollars.

That was about all I could handle for the night, and although it was late (8-ish?), the sun was still up. My body was still on East Coast time though. Despite that, I needed some marionberries. It’s a local Oregon food: a cross between the tart raspberry and the sweet blackberry made in Marion County, Oregon. You end up with a killer super-sized berry that ends up in cobblers, pies, jams, and milkshakes at Burgerville. I’d already been to several restaurants in the area but hadn’t found the marionberry yet. Randal, Jim Brandt, and I ended up at McMenamins Pub, an Oregon chain. In fact, we ended up in the same pub where Randal had his first beer, the “Ruby”, a raspberry ale. I felt like a part of history, and definitely had a new trivial item since his first wasn’t “Pearl”. I didn’t see a “Python” ale on the menu.

Glory be, McMenamins had Marionberry Cobbler, although the server apologized that they were out of whipped cream. Damn the whipped cream, full marionberries ahead! Marionberry Cobbler all around my good man!

As promised, the cobbler was good, but Randal also noted it was probably made from frozen berries so I shouldn’t think that I’ve had the true marionberry experience just yet. I should wait for a marionberry milkshake at burgerville, although I’m unlikely to get one. I can also stop by The Oregon Store in the airport to by some preserves on my way out.

So ended day one, exhausted and sated. To me, it already felt like wednesday.

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6913

Lawrence Rosen’s Law for Geeks is more an extended Q&A than a tutorial, but it’s entertaining and informative nonetheless.

Rosen has had considerable experience advising individuals, companies, non-profit groups, etc. on the law as it applies to Open Source. It was impressive how often someone would ask about an unusual hypothetical issue, only to have Rosen respond with a completely obtuse but related real world case. For example, Rosen once had a client who had worked years ago for a company that developed a snazzy piece of software, then decided to shelve it indefinitely and later went bankrupt. The client had kept a backup he’d made when the project (and company) were still alive. Can the client now release the code from the ancient backup? (Answer: the law is clear that the answer is no, but the reality is somewhat different.)

Drifting from question to question, Rosen took the time to debunk misconceptions (for example, “don’t ask, don’t tell” patent avoidance doesn’t protect you as much as you think), explain various nuances between related legal concepts, and serve quite a few reality checks along with the theory and code of law.

There were, however, a few key threads throughout:

• The devil is in the details, and the practice of law is usually much greyer than the letter of law. Many decisions around intellectual property issues are more about risk analysis than right and wrong.
• Get a lawyer’s advice (did you expect him to say otherwise?). Recent case law makes this even more attractive, as all you’ve got to lose is the fee. Still, you need to filter his advice through your brain.
• Industry standards should be free from patent restrictions (or be freely licensed in Open Source-favorable terms). Though obvious to most of us, this is a current battle that he and others are waging within various standards bodies.
• Current Open Source licenses could use a lot of improvement, and current law is at best an imperfect match for the software industry. Thankfully, there are growing ranks of OSS-aware lawyers who are working to improve this situation.

Strangely, the talk was valuable less on a direct knowledge-transfer level (though there was certainly a lot of that), but more on a meta-level, convincing me that these issues are difficult and important and pat answers are not enough. I made sure to pick up Rosen’s business card . . . .

Have you gotten legal advice on your open source project or presence? Was it worth the time and money you invested?

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/6439

Derek J. Balling earlier today recommended O’Reilly require future OSCON speakers to attend Damian Conway’s Presentation Aikido. I’d tend to agree.

I’d heard in the past that this particular tutorial was an absolute must if I ever made it to OSCON again, and I’ve definitely enjoyed Damian’s talks before, so this one was a no-brainer for me to take — especially since some year soon I’d like to give a talk at OSCON myself. I wasn’t disappointed. I enjoyed the class immensely, and learned a good deal about technical speaking. More than that, I was left thinking “Oh man, I’m so glad I didn’t try to give a talk without taking this class first!”

As Damian hammers home throughout the tutorial, your audience is giving you their most precious resource — their time — and that is a privilege that you should honor. The last time I attended OSCON, several years ago, I remember leaving at least two sessions early because the presentation was just plain awful. The speaker was unprepared, the slides were horrid, and energy and enthusiasm were nonexistant. I remember thinking “This person not only wasted the fifteen minutes I stayed, but I am now too late to see more than half of my alternate choice. Bah.”

Don’t do that to your audience. You owe it to the people investing their precious time to repay them several times over. Even if “it’s just a team brown bag”, take the time to learn to present better. This class is an excellent way to jump start that goal.

If you missed it this time, and you can’t make it to whatever conference he’s presenting it at next, I recommend extreme measures. Convince someone at your company to throw large wads of cash at Damian until he agrees to give the talk to your team. If anyone balks at the bottom line on the PO, title your request “Building Competitive Advantage”. At least then you’ve built a certain air of mystery around it, while at the same time being completely honest. (Attend the talk for more on the air of mystery.)

Oh, and I am not That Guy in the Second Row. Honest!

Should potential conference speakers be forced to attend this tutorial first? Or should you just skip sessions taught by speakers who haven’t done so?

I was sitting in O’Hare, on the floor next to a power outlet, waiting to get on my American plane and Gershwin’s “Rhapsody In Blue” comes on in my iTunes. I laughed out loud.

On the plane, I chatted with the guy next to me. “What are you going out to Portland for?” He gets the look on his face that I immediately recognize: “I don’t want to have to explain it.” “An open source conference” he tells me. Turns out he works at UBS with my friend Eric Sinclair, and already has my testing talk circled on his list of stuff to see.

I’ve also got 50 copies of my job ad printed as flyers. Who knows what I might find?

Update 2005/08/03: On Wednesday morning Tony affirmatively states that there is no legal issue with Class::DBI. He indicated to me in earlier private email that there were other legal issues involved and that he could not say anything about how that related to the community. He has relented on that part of the debate.

OSCON has started off lots of gossip and queries about Class::DBI’s personality problems. The news around the camp fire is that people are going to jump ship to get away from the proble.

In a previous weblog I reported on a row between Tony Bowden and Sebastian Riesel. Tony wrote to me to explain part of his story, but also explained that he has separate legal proceedings that prevent him from explaining his actions. That, along with personal attacks he made towards me and seemingly coy statements about why this such a big deal, worry me.

I’ve seen this sort of thing before while consulting with Stonehenge. People who want to get open source into their business have to convince their higher-ups that it’s a good idea. Some developers and maintainers are so truculent, wacky, eccentric, or whatever that the business people balk at using the otherwise technically excellent software. Tony’s taken it a step further by dangling threats of lwasuits over everything and everyone that comes near him.

It’s easy to see the business case against this situation. What do you do when you need support, or you want to request a feature? Certainly propietary software could have similar outcomes: no patch and no feature. Open source developers, however, will also write you back with personal attacks and behave poorly in public. Now legal action is in the mix.

In Tony’s private mail to me, he continued to insist on legal action along multiple axes, and made some statements that made me think he might now direct some of that at me. He wants to shut down the discussion with the threat of lawyers. What do you do when volunteers and open source developers can’t get along or act like adults?

Based on my recent personal correspondance with Tony as well as my contact with the other principals involved, I think Tony’s now a liability for Class::DBI, and I can’t recommend the module until he breaks his association with it. He’s passed the point of no return. It’s too bad that his otherwise good work is now overshadowed by a personality conflict.