Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7546

Is this your first time at an OSCON? Would you like a bit of help figuriung out what to do and see, how to talk to the Big names, and so on?

Monday night, Jim Brandt and I will discuss all the things about conferences that aren’t in the conference guide. Many frequent conference goers use a host of tools to supplement their conference experience, and we will share these tools with beginners. We’ll talk about things like IRC, kwikis, SubEthaEdit, blogs, Open Source luminaries (talking to the pros), and BOFs.

Related link: http://www.mono-live.org

Recently I shared my vision for the value of Live Linux distributions that minimize the obstacles for learning open source software development. This came as an epiphany for us at the SNAP Development Center after we released SNAPPIX version 0.7. In the beginning, we only wanted to build a showcase for SNAP Platform, but once we had a working copy, the possibilities became instantly obvious. This was also true when I heard about a new Live Linux CD featuring Mono. This article expands on this premise, reviews Mono Live and shares an interview I conducted with the originator of the CD, Joseph Hill. It is my hope that this Live Linux CD plays an important role in the adoption and spread of the Mono project (as I have previously written here).

Introduction

Mono Live was released on May 24, 2005. Although there are several Live Linux CDs with Mono, they are all based on Knoppix and use KDE as the default Linux desktop. This distribution is based on the Ubuntu Live Linux CD. In many ways, the outstanding experience available from the Mono Live CD is made possible from Ubuntu. Since Ubuntu features GNOME as the default desktop, it only follows that due to Mono’s heavy reliance on Gtk+, Mono Live would provide superior functionality. The current version of Mono Live includes Mono, version 1.1.7 and all of the key platform components, including a broad assortment of Mono Gtk-sharp based GUI applications, ASP.NET applications, and the software infrastructure to run the application. In addition to the core mono-based tools MonoDevelop, MonoDoc, and xsp, the Live CD also includes Postgres, pgAdmin III, and Glade.

Mono Live also has a major bonus. When the Mono Live CD is inserted into a MS Windows machine, it automatically provides the ability to install the complete Mono for Windows installation package. Thus, Mono Live serves a dual purpose, first as a complete Live Linux distribution, and second as a complete Mono Windows distribution.

History

According to Joseph Hill, the founder of the Mono Live project, the idea first came from seeing Knoppix about a year and a half ago, and realizing that a live CD might be the perfect vehicle to expose .NET developers to Mono and Linux. I would definitely agree with Joseph. I believe that this is a much needed and important step for the Mono project. Building bridges between the Windows .Net community and the possibilities and choices gained by new deployment alternatives is mandatory for Mono. For although the Windows installation experience is far superior to the Linux installation, the Windows version of Mono still lacks key pieces. Currently MonoDevelop and many of the marque Mono applications are still only available on Linux. I know that the Mono community is working hard to balance these disparities, they are slow in coming. Early this year, there were rumors within the community that MonoDevelop would be available by the O’Reilly Open Source Convention, but unfortunately that has not happened.

Review - Sample Applications Galore

For me, the stability and tight integration and implementation in Mono Live is superb. For one who has struggled with configuring Mono in the past, using the CD was a huge relief. With just a simple boot from CD, I had a rich Mono platform to explore and use. For that reason, I believe that Mono Live accomplishes what Joseph set out to provide. It especially excels in demonstrating the capabilities of the Mono platform. The CD comes with a rich collection of sample applications, both Gtk-Sharp GUI-based and ASP.NET. From the Mono community of applications, the CD includes the following Gtk-Sharp applications:

• Tomboy - Tomboy is a desktop note-taking application for Linux and Unix. Simple and easy to use, but with potential to help you organize the ideas and information you deal with every day.
• Muine - Muine is a music player for GNOME.
• Blam - Blam is a tool that helps you keep track of the growing number of news feeds distributed as RSS. Blam lets you subscribe to any number of feeds and provides an easy to use and clean interface to stay up to date.
• F-Spot - F-Spot is an application designed to provide personal photo management to the GNOME desktop. Features include import, export, printing and advanced sorting of digital images.
• Beagle - Beagle is a search tool that ransacks your personal information space to find whatever you’re looking for. Beagle can search in many different domains.

These applications are currently available in binary form, but a future release of Mono Live will include source code as well. As mentioned, the CD also provides the leading sample ASP.NET applications for Mono. The sample ASP.NET application included are:

• IBuySpy Portal - his project is a port of the original IBuySpy Portal Solution Starter kit from http://www.asp.net. The principal features of this version of the portal are that it adds support for Mono, and uses PostgreSQL for the database.
• BlogX - Mono Blog is a Mono-compatible version of the BlogX weblog engine originally developed by Chris Anderson. Because BlogX is written in C# and uses XML to store content, it is one of the easiest complete, open-source .NET applications to get running on Mono. This project will attempt to make it even more simple by correcting all Mono incompatibilities in the source, and by maintaining all Makefiles needed to build the project.
• nGallery - nGallery is a FREEWARE, OPEN SOURCE implementation of a image gallery written purely in managed .NET and C#. nGallery provides a solution to store and display your image galleries on your own Web site, as well as providing means for customizing and extending nGallery to your own personal likings.
• ASP.NET Forums - The Mono Forums power the new GotMono forums at http://forums.gotmono.net and are a port of the original ASP.NET Forums from http://www.asp.net/Forums/. The principal features of this version of the Forums are that it adds support for Mono, and uses PostgreSQL for the database, so that it can be run entirely on Linux.
• mojoPortal - MojoPortal is an Object Oriented web site framework written in C# that runs under ASP.NET on Windows or under mono on GNU/Linux or Mac OS X.

These sample applications run using the xsp server included with Mono. With desktop and menu links to each, running these sample applications is only a simple click away. In addition, in order to support these sample applications, Mono Live features the Postgres relational database management server. The CD comes pre-installed with all of the configuration necessary to access the pre-created databases. All of the ASP.NET applications include the full source code.

As with all Live Linux distributions, Mono Live features USB Flash compatibility allowing for creating and saving work. As with many other specialized Live Linux distributions, Mono Live inherents a broad level of functionality from the base Linux distribution. The Ubuntu Live CD provides complete network, file system, and device functionality.

Results

Reaction from the Mono and .Net community has been strong. After two months of availability, Mono Live has been downloaded more than 15,000 times. Novell Corporation, the host for the Mono project, has also demonstrated their support of the concept of a Live CD with Mono. During July of this year, they requested Joseph to assist in creating Mono Live DVD, a Live Linux DVD version based on the Novell Linux Desktop. This distribution should be available soon. Finally, interest in Mono Live continues to grow. Various user groups from the Dallas area, where Joseph resides, have invited him for demonstrations of the features of the Live CD and Joseph and i will host a Birds of a Feather session to discuss and demonstrate Mono Live. For those interested, the session will be Thursday August 4, 2005 in room E142. The session will begin at 8:30pm.

Roadmap

Building on the initial success, the Mono Live CD will naturally mature as the core Mono functionality matures. One challenge going forward, Joseph admitted,is resisting the temptation of the community to believe that the Mono Live is a “new” Linux distribution. Instead, Joseph believes that the future developments of the Mono Live CD should focus on providing an inviting experience from learning and using the Mono platform. However, as I have wrote previously I believe that special purpose distributions, like Mono Live, will serve an important role in the future of open source software development. The complexities of installing a stable working environment from an ever changing base of components and packages can be a huge obstacle for neophytes. I believe that having the luxury to install a well integrated and stable Mono platform is critical. For me, the moment I booted the Mono Live CD, I immediately wanted to replicate it onto a machine. I believe that many others will want the exact same thing.

What follows is an interview I did with Joseph Hill. His answers provide a clear picture of the origins, the obstacles over come, and the future of Mono Live. I originally intended on just including several quotes from the interview, but I decided that the entire interview would be of interest to everyone. I want to thank Joseph for his time and passion for bringing this project to fruition.

Interview

Kevin: Where did the idea come from?
Joseph: When I first saw Knoppix about a year and a half ago, the value of the CD as a tool to expose .NET developers to Mono was immediately apparent to me; however, when I researched the idea, I found that there were already a few flavors of Monoppix in the works, so I went to work on other things. I returned to the idea this spring when I was discussing with a friend how none of the existing projects seemed to be headed in the direction I had originally envisioned, and he pointed out to me how easy it was to customize the Ubuntu LiveCD.

Kevin: What were the objectives heading into the project?
Joseph: The primary objective of the project was to show developers what they can do with Mono. That means that not only did we need to make it as easy as possible for developers to get their feet wet with Mono development, but we also wanted to show off the many applications that already run on Mono.

Aside from that, we realized that a lot of our target audience would be coming from Windows, and that their impression of Mono would be closely tied to their impression of the Linux desktop, so we needed to make sure that the overall of experience of the CD from start to finish would leave them feeling good about the whole platform in general. We’ve tried to accomplish that by making the user experience something more like working on somebody’s active Linux installation, where they already have photos in F-Spot, music in Muine, notes in Tomboy, etc., rather than the typical LiveCD experience, which seems to be a little more like starting from scratch on a fresh Linux install.

Kevin: What were some of the biggest challenges?
Joseph: It seems like each major feature on the CD was a big challenge at the time. First, just proving the concept would work at all was daunting task “Can we get Mono on the CD? Can we get PostgreSQL to run right?” Then it was “Does MonoDevelop work? Will it run Beagle? Can all of this possibly work at the same time?”

Luckily, I was able to get a lot of help along the way. I sent Ben Maurer an early build when I thought I was getting close to done with it, and he had a lot of great ideas that really helped complete the experience. He also helped me get Brandon Hale to help us out by contributing custom Ubuntu packages of Mono and all of the applications built just for the CD.

Our most recent challenge has been that we were running out of space, but luckily someone put instructions for taking care of that on the Ubuntu wiki, so now we have plenty of spare room.

Kevin: Are any other application going to be included?
Joseph: Definitely. Now that we have the size issue addressed, we’re really searching for new content. We’re going to need some cool new apps to keep the content fresh, and to justify new releases beyond just getting the latest version of Mono. We already have a few Windows Forms examples picked out for the next release.

What we really need, though, is more documentation. Since we want to make experiencing Mono as easy as possible, and we know that we probably only have the users attention for a short while, it would be really nice to get some tutorials collected on there that will help developers hit the ground running.

Kevin: How has the reception been? Many comments? Like what?
Joseph: The reception has been far better than anything I could have imagined. Mono Live has appeared in a few news blurbs out there, and in the few months we’ve been out, we’ve seen at least 15,000 downloads (which seems phenomenal to me, considering the size of the CD).

Everyone I’ve talked to has been pleased with the CD, but there have been a lot of feature requests, too. That’s great news, though, because it shows that people have really been working with it, and it tells us where we need to go. The most popular requests have been for Samba support, Windows Forms support, and, of course, an installable version of the CD.

Kevin: What can you do with the CD, that would surprise people?
Joseph: I hope there aren’t that many surprises in there, because we really want it to be easy for users to use the CD to its fullest potential.

One thing that hopefully won’t come as a surprise, but seems to be often overlooked: If you put the CD in a machine that is already booted into Windows, you’ll be given the option to install Mono for windows, along with several other handy development tools. In upcoming releases, I want to focus on growing the experience on the Windows side of the CD so that we can really highlight the cross-platform capabilities of Mono.

Kevin: How goes the work to make the CD installable?
Joseph: Honestly, this is really a low priority to me, because I don’t think that’s where our efforts can be best focused. I don’t really want to build a new distribution. I just want to provide users with an inviting experience of the platform.

The real answer to the installation problem is that Mono and (all of the Mono applications) need to be just this easy to get going on any Linux distribution you pick up as they are on the CD. I think Mono is getting there.

Have you tried Mono Live? What do you think?

It’s amazing to work in such a dynamic and rapidly changing industry. It was just last year during OSCON that the debate about whether Sun should release Java was all the rage. However as we head into this year’s convention, that debate has mostly died. It has been replaced with an active open source Java community that is becoming about more vocal and assertive.

To summarize briefly last year’s focus on this debate, first there was a panel session entitled “Open Source Java”. The panel included Tim O’Reilly, Bruno Ferreira de Souza, Simon Phipps, and Eric Raymond. I remember during that session having a distinct sense that nothing significant was accomplished in the session. Since that session I have researched all sides of the debate and lead a project focused on moving the open source Java forward. Now when I review my notes from that session I find that my original sense was accurate, no new ground was covered in the session.

Besides the panel session, Bruno Ferreira de Souza also lead a Birds of Feather session on “Java Livre: the discussion about Java and Open Source continues…”. For me, it was while listening to Bruno that I believe I caught the open source Java bug. From listening to Bruno’s impassioned discussion of the subject I finally understood why open source Java was so important and the potential role it must play in the future of Java. I think my partner at the SNAP Development Center, PJ Cabrera who also attended Bruno’s session, must have caught the bug too because following last year’s OSCON he lead our technical team in the creation of the SNAP Platform and SNAPPIX, both of which include an completely integrated open source implementation of the Java standard.

Now, as we arrive in Portland this weekend, we find, first, OSCON contains a rejuvenated interest in Java, with a significant increase in sessions in the Java track. We also find that the previous year’s debate is completely absent from the schedule. In its place are three sessions specifically about open source Java.

On Wednesday at 11:35am, Geir Magnusson will present “Open Source Java”. As leader of the newly formed Apache Harmony incubating project, Geir will provide an introduction to the Apache Harmony project, provide a status update, and present the near-term roadmap for the project.

On Thursday at 10:45am, Tom Tromey, of Red Hat, Inc. will present “The State of Free JVMs”. Tom’s talk will cover the successes of the recent past, the current state of affairs, and some insight into the developments you can expect in the next year, as well as the rationale for open source JVMs. There will also be cool demos.

Finally on Thursday evening at 7:30pm, the SNAP Development Center Team will present “The Reality of Open Source Java: SNAP Platform and SNAPPIX” Our session will build on these preceding two sessions and examine the myths and the realities of Open Source Java. With a clearer understanding of the realities, our session will establish the grounds for a new on-going discussion about the future of Open Source Java. We will also demonstrate version 0.9 of the SNAPPIX Live Linux CD featuring the SNAP Platform.

With these three sessions, it certainly seems to me that the last year has made quite a difference for open source Java. The debate which has raged on for years is quieting and and we are witnessing a shift in the momentum and visibility of the open source Java community.

What sessions are on your agenda?

A long time ago in a galaxy far, far away, Michael Schwern created a Perl module for objects to represent database queries. You didn’t need SQL or other fancy things. Through various magics, it figured it all out behind-the-scenes and it worked. He called it Class::DBI, and it was good.

Later, Tony Bowden took over the lead maintenance role, set up a mailing list and wiki, and that was good too. One of the hallmarks of a good open source project is that you can pass on the reins to someone else, not only because people want to work on it but because a lot of people want to use it (those always don’t go together ;)

This week, there was a bit of a dust-up on the Class::DBI mailing list. You can still find the archives, but Tony killed the list. (Fear not, there is a new one already, even if it is just temporary).

Sebastian Riedel and Tony apparently had some off-list as well as on-list scuffles. Tony kicked Sebastian off the list and Sebastian came right back onto the list. Rinse, repeat. Finally Tony threatens legal action.

Update: Wednesday morning In a show of his willingness to compromise, Sebastian voluntarily removed his post from use.Perl. I’m optimistic that this can defuse most of the situation.

Sebastian created used an his account on Use.perl (Correction: Sebastian tells me he had the account for a while, but hadn’t used it to post to the journal before) and posted to post his side of the story along with some private email between him and Tony in “What the hell is going on in Tony Bowden’s head?”. However crass that may be, he did it. He also disabled didn’t enable comments so and people couldn’t reply to his post (Correction: Sebastian tells me that this was uninentional. Indeed, whether comments are turned on or off is a preferences setting and not something you have to decide for each entry). From other posts and some private personal email with other people, I’ve gathered there were some disagreements about the feature set that would make it into the next release and how those would work. That seems to be the sort of thing that causes these sorts of problems: two people convinced they are right and unable to work together because of it.

Schwern replied in his own Use.perl journal that Sebastian fudged the story a little bit by linking to a seemingly innocent part of the email exchange. Schwern provided the links that show a lot of inflammatory language from Sebastian.

It gets worse though. Since Sebastian posted to Use.perl and Tony doesn’t like what Sebastian says, Tony threatens Use.perl (a VA project Correction: Chris Nador owns Use.perl, but VA sponsors the site infrastructure) with legal action. Tony, a formerly respected member of the Perl community, now seems to think he has to burn down the village to save it.

No matter who’s right, these things don’t help either side. Indeed, as sigzero points out, this is just more fodder for people who think Perlers are a bunch ill-tempered miscreants. Where a rational person might just take the weekend off and let things blow over (or at least not inflame the situation), both sides seem to want to fight it out. They’ve forgotten that they both want a really cool module to get better, and maybe that means they don’t have to kill each other to make it happen.

Who cares about these petty problems? A lot of people are using Class::DBI and they want to keep using it. It’s not going to disappear. Even if Tony decided to delete every version on CPAN, the BackPAN still has all of the current and previous versions. Someone will just take his place.

The mostly sane workers and users really just want to get on with business, I’ve heard rumblings of a fork in Class::DBI. The only way around this might be to just bypass the personality problem by removing the personalities. I think that would tragic, even if it ends up being unavoidable. Can Class::DBI do that without bifurcating?

Do you think Class::DBI should fork?

Related link: http://www.edri.org/edrigram/number3.15/retention

The idea of collecting and mining enormous (unfathomably enormous)
amounts of data for information about crime goes back a long way, but
recent terror attacks have made it more appealing to many people.

I talked to one privacy rights advocate about the London bombings of
July 7, and asked whether he would re-evaluate the privacy community’s
stance against public surveillance cameras, since these cameras proved
critical in figuring out who did the bombings. He admitted that we’re
likely to see more and more cameras now–at the cost of millions and
millions of dollars that could go toward more effective anti-crime
initiatives.

But he also pointed out, “These cameras might help investigate and
prosecute terror after it happens, but they won’t be effective at
preventing terror, except for the deterrent effect.” (Well, there’s
also the gain of capturing people who helped terror.)

Now the call for the digital equivalent of surveillance cameras, data
retention on networks, is growing. A European Commission directive
wants all network operators to keep traffic information for one
year. This is not Echelon, slurping up everything you say or do. It
means that mobile telephone providers will record all the digits you
press, and that Internet providers will record the addresses you
go. This probably records such “traffic” as the search terms you enter
at web sites.

The objection to these practices has always been that they may be
turned against you. Safeguards such as warrants tend to get weakened
under political pressure; law enforcement either wins examptions from
oversight or just ignores laws about oversight. The data is also a
sitting duck for other malicious snoopers.

The article referenced above gives more information about a petition
against data retention. If you live in the European Union and want to
sign the petition, visit

the site for signing.

I think that if we ever calm down and seriously talk about how to stop
terror, one of the major strategies will involve “human int” (having
people embedded in the organizations that are potentially
dangerous). This, of course, leads to its own human rights issues;
moles can also be used abusively. But it’s a very low-tech solution;
one that involves getting closer to and building trust with the
communities where terror is likely to grow. It requires holding back
from shooting dark-skinned men just because they act suspiciously, and
doing other things to show that society wants all its residents to be
full-fledged members.

Related link: http://www.heron-language.com

I’m a programming language designer … okay, you can stop laughing. If you don’t believe me, you can look at my wallet … it’s empty. I quit my day job two years ago to pursue my programming language project (Heron) full-time. Since then I have been doing a lot of freelance writing about C++, while developing Heron.

Think of Heron as C++ meets Eiffel, they get drunk, take C out back, and shoot it.

I made some small progress on the language, and the last version of Heron wasn’t too bad. I was able to do some cool stuff with it like write an XML parser, but no one really cared. So I decided a few months ago to go back to the drawing board.

I realized that my big problem was that I lacked focus and I was trying to stuff every cool technology I could find into the language. As a result I was doing a mediochre job of everything, rather than a good job at anything. The new version of Heron focuses on its biggest advantages: generic programming and programming with contracts (PwC). Generic programming support allows Heron programs to be very efficient, while the embedded support for PwC makes for faster development and fewer defects.

At this point, I am preparing the first version of the standard library, and I am wondering what I should emphasize. So my question to you is this: what would it take for a programming language to get you excited?

What would you consider to be the viagra of programming language features?

I’ve always been following the latest fads and I see a lot of technologies being used together to create a more interesting web. Take, for instance, RSS. RSS is not a new technology. So why was its global adoption so delayed?

A bit of history

I don’t want to bore you with all the facts so just check the RSS History for the whole thing. Bottom line: we now have different two RSS versions: 1.0 (RDF Site Summary) and 2.0 (Really Simple Syndication). Some websites use the first, some user the last. While it seems that RSS 2.0 is gaining popularity (because it descends directly from the original), RSS 1.0 is still sticking.

My personal experience: in 1999 I co-created a website and since the beginning I always tried to syndicate its content. First we deployed a series of javascript channels (very popular at that time) that allowed any website owner to display auction content. Then we deployed an index.rdf file and started publishing content readable by any feed reader. At that time there were few feed readers though, so it wasn’t very popular.

Syndicating is gaining popularity

Anyway, this website was bought by a large company and one of the first features to let go was, obviously enough, the javascript channels and the index.rdf thing. It didn’t make sense to them to let go content instead of attracting people to their own site. It didn’t make sense at that time, at least, because now it seems that the opposite is again the way to go. Or so we like to imagine because, you know, this is just another fad.

It seems to me that all the enthusiasm we are now witnessing is a product of the power of technology evangelism. People like Dave Winer and Tim O’Reilly are driving every CEO crazy when they talk about monetizing content by syndicating it. Wired and the long tail article also played a major role on this global change. When you see China People’s Daily syndicating their news sections, you can feel that something is moving your way, really fast.

People want content

You can stand still and wait for it to hit you hard, or you can start running now and hope that you don’t fall behind. Because when everybody else has their content syndicated (and maybe monetized, but that’s another story), people will run away from your website (no, really).

People want content and want to consume it at their own pace, in their own terms. They don’t want to view it through that layout you took three months to design and get approved by the board of directors. They don’t want to have to click twice to get to that news piece you just edited last night and took an entire morning to publish. They don’t want to wait for your website to load when there’s a traffic surge.

They just want the content, period. Don’t even bother trying to help them consume it, just make it available.

Why RSS is the way to go

RSS is now the best approach to this paradigm. With feed readers widely available, feed directories and search engines gaining popularity as you read this, rest assured that your content will be consumed.

You just have to let it go. Learning how to abandon your old-fashioned website is now your major concern. In a few months you won’t even look back, or reality (some people call it competition) will step on you.

Are you already syndicating your content? What are you waiting for?

Spontaneous outbursts can reveal much about group behavior.
Advocates of open source showed such a moment on the first day of the
Ottawa Linux Symposium, about which I recently
reported
at length. But the incident I am thinking of was recorded for history
in a

Newsforge article by David “cdlu” Graham:

He [Doug Fisher] wrapped up his presentation to the usual polite
applause and closed his slide show to reveal the message “Windows XP
has locked your desktop,” resulting in the single loudest and most
sustained booing by nearly everyone present I have ever heard,
followed by a member of the audience rushing to the front brandishing
a Linux installation CD to widespread applause.

I was present for this incident, and vouch that the noise and emotion
released was extraordinary. Wow! I felt like I was in the middle of a
scene from some Hollywood epic where the soldiers raise their spears
and roar their defiance to the enemy.

Graham makes a subtle but valuable comment on the incident while
describing the next speaker:

His laptop, in contrast to Fisher’s, ran Linux, but perhaps
demonstrated why Fisher’s didn’t. After a lengthy battle with X to
have an appropriate resolution for the overhead projector, he launched
his presentation entitled “How to talk to business people about the
value of open source,” which sparked the comment of an audience member
sitting near me: “not like this.”

So why was this audience of 800 hard-core Linux developers and
adopters so intolerant of a speaker who uses a Window platform? What
is the deep well from which such emotion comes?

They have legitimate anger, of course, at Microsoft’s illegal
anticompetitive behavior, which has been well established in several
courts, and the company’s continuing hounding of Linux by pressuring
governments, funding SCO, and spreading biased reports. The attendees
also have the same kind of familiar team spirit that breaks out
(sometimes quite violently) when different sports fans come together.

But that does not add up, in my view, to the display of anger I saw
that evening. Another ingredient needs to be added: insecurity.

Linux developers and users are aware that what they put together is
not yet convenient to install or easy to use. They resent Windows not
only because it has a dominating position, but because that position
is maintained by something more organic than the vaguaries of
computing history or corporate behavior. Microsoft (on the desktop, in
particular) awaits an open source challenge, and that challenge is not
yet strong enough.

I have prepared some fairly detailed slide presentations, trying out
Microsoft’s PowerPoint, OpenOffice.org’s Presenter, and KDE’s
KPresenter for that purpose. Presentation suites are either very hard
to develop or are the perennially neglected stepchildren of the office
suite, because I found all three products embarrassingly buggy.
PowerPoint bears the most shame because it’s been around the longest
and Microsoft promotes it as a professional tool worth a lot of money.

Still, if Doug Fisher has spent a career developing presentations with
PowerPoint, why is it a sin for him to use it for one more
presentation? It would certainly have been a smart move to feel out
the mood of his audience beforehand and to take the time to set up
Linux and a free software office suite. But if the Linux and open
source crowd felt more secure in the superiority of their product, his
gaffe would have been greeted with just a few snorts and giggles.

I see too much of this moralistic hypersensitivity among people whose
goals I respect. Environmentalists are telling us to replace our light
bulbs, take public transportation, and recycle everything we can–yes,
all wonderful goals. But some environmentalists also realize that
individual behavior change cannot be achieved on a mass scale through
moralizing; new technologies and institutional strategies must drive
progress.

The same goes for personal life choices, which contribute to the
health care crisis. It would be great for everybody to cut down on
tobacco, trans fats, chemical household products, and epic Hollywood
movies, but lecturing does little to help.

You can apply this principle even to nationalistic and religiously
close-minded political stances. These are tearing the world apart, but
we can’t get anywhere by telling people just to throw away their
commitments to these beliefs and to embrace a neutral, diverse
world. Those attempting change have to realize that people have
reasons for holding on to their stances.

Getting back to open source: the way forward is to build something so
great it compels everyone to use it, and to move institutions
(ironically, the topic of the second presentation that evening at
Ottawa Linux Symposium) to positions where they can make the switch.

The really great aspect of the Ottawa Linux Symposium is that its
attitude overall is in diametric opposition to the defensiveness
displayed toward the Windows-wielding presenter. The Linux developers
and their communities could easily waste time wallowing in excuses
such as “Vendors don’t give us their specs” or “Implementations don’t
conform to standards.” But they don’t do this. They say, “What can we
do to make this work?” And that’s the path to success.

Related link: http://www.theperlreview.com/Interviews/ian-ptdn-20050712.html?ora

I talk to Ian Langworth about his first book, Perl Testing: A Developer’s Notebook and what’s it’s like to be a first time author, what’s new in the testing world, and what he sees coming up.

Perl testing has been a hot topic for a while, and we finally have a book about it. Well, we almost have the book. It’s done, edited, checked, and just waiting to hit the streets. It’s a pretty exciting summer for Perl books.

I’ve been doing more and more work with XML, and my appreciation for that family of technologies is growing by the day. XML and open data standards solved a problem that arose with OpenOffice.org Writer a few weeks ago.

OpenOffice.org is (of course) an open source office suite. I’ve been using a pre-2.0 test version of the software, and it has demonstrated a few instabilities but also has some great new features.

I spent all of Tuesday evening using OOo to draft a detailed outline for a book. The outline contained nearly 200 entries, and while the resulting document was fairly short — only a few pages long — it represented a lot of work.

You can imagine my dismay when the document wouldn’t open the next morning. “Read Error,” the program whined. Something incomprehensible about a format error at (2,2847) in styles.xml.

Not a problem, I thought — I’ll just use the backup that I’d saved. Same error! The previous version of the file - same error!

If I’d written that document in WordPerfect or MS Word, that would have been the end of the story. I’d probably have to rewrite. I know it happens; I’ve been there.

But OOo 2.0 uses a document format that is an OASIS standard, which means that it’s publicly documented XML. Actually, an OOo document is a zip archive containing multiple XML files.

So I unzipped the OOo archive and checked the styles.xml file using xmlwf (checking to see if the XML was ‘well-formed’, which is step one of two on the road to correctness; the second hurdle is validity according to the schema). Sure enough, there was a duplicate element attribute at the line and column indicated in the cryptic OOo error message.

Edit it out, zip it back up, try again, and … same error, different location. But after a couple of iterations the problem was fixed.

Sure, it was a pain, and sure, it should never have happened. But in an imperfect world, I’d much rather have my data in an accessible format that can be manipulated by many different tools than locked up in an undocumented, proprietary format.

Have open data formats saved your day?

My appreciation for Computer Science started very early in high school (circa 1995). My first computer course was an introduction to programming in BASIC. Then I took Advanced Placement Computer Science A and AB, both taught in Pascal. My first exposure to C/C++ was in my Senior year in high school. I did very well in the all courses. I completed programming assignments weeks before they were due. In college, I did very well in courses that had a substantial programming component.

Today, I still enjoy programming a great deal, while concentrating on other fields such as HCI and Security. I credit the many great teachers and professors I studied under over the years. But I feel that there is one other element that drives my interest in computer programming: I always look at it as an art.

When I was a kid, I had thoughts of becoming a scientist one day.

Then I wanted to be an artist.

I feel that I got the best of both worlds. I feel very deeply that computer programming is an art. Why? It is a challenging and creative process. There is always more than one way to solve any given problem. I have seen “Hello, world!” written in various languages, and in various ways –from the obvious to the obfuscated. There are many ways to write a loop. A program that is written in 20 lines of Java, can be be written in 5 lines of Perl. Of course, you can know a plethora of languages, but you have to choose the correct programming language for the job.

My high school teacher told me something that still sticks to me today: a computer program is no different than a novel. Source code should be read like a novel, and great programs should be planned so like any great novel.

Related link: http://www.vim.org/

Doing a new project in Rails, I started using vim when putting it together, then a couple weeks went by, and I never got out of vim!

I used to use GUI-based editors with syntax highlighting, code hints and such - until I found that vim can do all of that! Once I figured out how to make it automatically close all my parens, braces, and brackets, I was sold.

Here’s my favorite .vimrc thing that helped….

Put this in ~.vimrc

:inoremap ( ()<ESC>i
:inoremap ) <c-r>=ClosePair(')')<CR>
:inoremap { {}<ESC>i
:inoremap } <c-r>=ClosePair('}')<CR>
:inoremap [ []<ESC>i
:inoremap ] <c-r>=ClosePair(']')<CR>
function ClosePair(char)
  if getline('.')[col('.') - 1] == a:char
    return "<Right>"
  else
    return a:char
  endif
endf

The other thing that helped a lot was konsole. I like to full-screen all my terminal windows. Never fond of having lots of little boxes floating around. So, when in a Rails dev-mood, I open up a tab at the bottom for EACH of these:

• project
• model
• view
• controller
• lang
• database (pg/mysql console)
• Rails/ActiveRecord console
• misc
• log

I just [SHIFT]-arrow back-n-forth between them, each one taking the full screen, and getting my full attention while working.

Related link: http://www.linuxsymposium.org/2005/

It’s been another Ottawa Linux Symposium, and before it fades into a
daze, let’s see whether I can extract some themes and threads.

The appeal of large systems and Xen

The type of Linux deployment that dominated this symposium was that of
a large, mission-critical system. Xen virtualization in particular was
heavily featured and attracted large numbers of attendees. More than
one attendee joked that Friday was “Xen day.”

Xen lets one piece of hardware run multiple operating systems,
controlling their access to the hardware through a kind of
meta-operating-system called the Hypervisor.

There is nothing new about the idea of virtualization, of course. It
was associated with the IBM 360 further back than most of us can
remember (in fact, IBM executives have told me they think Linux has a
major role to play keeping old 360-series computers going by running
in their virtual machines), and it’s now making good money for
VMWare. (More about them a bit later.)

Like VMWare, the major uses of Xen seem to be server consolidation
(which means running several instances of Linux on one piece of
hardware, a useful deployment because Linux seems to work best running
only one server daemon) and virtual hosting. Speaker Mike D. Day also
showed that Xen could be used to deploy Linux quickly to a large array
of computers.

Ian Pratt gave a comprehensive overview of Xen’s goals and
implementation. He defined Xen’s main achievements as two-fold
(although his talk really focused on the first): isolating different
processes in a secure manner, and controlling resources so different
Quality of Service options could be offered to different processes.

Pratt then laid out some of the ingenuity that makes Xen more
efficient than VMWare or User Mode Linux. For instance, Xen divides
page tables among its guest operating systems and gives each guest
full control over its page tables, so that the hardware doesn’t slow
down under the load of two levels of page management (one by the guest
and one by the Hypervisor).

There is one necessary exception to Xen’s practice of handing full
control over paging to a guest: the guest is not allowed to write the
pages that contain its page tables. If it could do that, it could give
itself access to the other guests’ pages. On the other hand, the guest
must be able somehow to indicate that it needs a new page. So the Xen
team has found some tricks to make it easy for Xen to trap a guest’s
writes to the page tables, make sure they’re legitimate, and let the
guest go ahead with the writes.

Another tour de force Pratt illustrated was how Xen eases
failover. Whether scheduled or in a panic situation, hardware
sometimes has to go down. Xen can make it easier to migrate processes
to new systems with minimal downtime.

This is done by doing a series of pre-copies while the guest system
keeps running and updating its state. The first copy takes a long time
because it starts from scratch, but each subsequent copy has less
state information to transfer and thus takes less and less time. (One
weakness of Xen is that it maintains a lot of state and therefore has
a lot of information to copy.) The amount of CPU time devoted to the
copying can also be titrated to leave plenty of time for the process
to continue handling incoming requests. Pratt actually drew applause
when he showed the CPU utilization of a highly loaded Apache server
during transfer to another node, and added it was down for only 164
milliseconds during the transfer.

As I mentioned, Pratt really concentrated on Xen’s goal of isolating
processes, but the second goal of doling out resources was touched on
by a Rik van Riel in a BOF that evening. He divided the resources
worth tracking into four types: CPU utilization, memory, data I/O, and
network I/O. CPU utilization, he said, was easy to track without
intruding on the guest operating system. So was I/O. Memory, which a
Hypervisor could easily give too little or too much of to a guest
operating system, was a much harder nut to crack. He suggested clever
ways that patterns of waiting for reads, waiting for writes, and the
length of request queues (way-stations for reads and writes) could
tell the Hypervisor whether an operating system was underprovisioned
or overprovisioned with memory. But more experimentation is needed to
see whether these are valid measures.

Xen offers lot of features already on 32-bit x86 hardware, with 64-bit
x86 and AMD coming along too. A number of operating systems can be
guests, including Linux, Solaris, FreeBSD, and OpenBSD. An upcoming
facility called VT-x should allow Xen to run operating systems that
haven’t been instrumented for it–so even Windows will someday show up
on the list.

VMWare is not passively accepting the limits that have long been
assumed on virtualization. They know that if they can break down the
barrier between Hypervisor and guest operating system, and learn just
a bit about what the operating system is doing (taking a spin lock,
for instance, or releasing a disk block), they can achieve fantastic
speed-ups in virtualization. An unannounced speaker from VMWare
presented some of their innovations at van Riel’s BOF, under the name
para-virtualization.

Para-virtualization’s goal is to blur the barrier between operating
system and Hypervisor enough to obtain useful information, while
minimizing engineering costs and the risk of breaking the operating
system. VMWare knows that Linux developers would have little tolerance
for a development process that required them to slow down so VMWare
could keep up, and that Linux distributors would push back if VMWare
slowed down performance or introduced risk.

Their solution is to introduce a new layer (named VMI) that would
cause some 30 to 50 instructions in the operating system to trap into
the Hypervisor instead of executing as normal. This is reminiscent of
the trap instructions introduced by a debugger into a binary, but
would be even less intrusive, requiring no change to the binary of the
kernel.

The solution is unique to each processor being emulated, but could
apply to any operating system compiled for that processor. The speaker
claimed that para-virtualization had been easy to introduce into the
Linux development tree and could be maintained as open source with a
typical open source development and testing process.

A narrow range of other topics

Clustering–which, as speaker Bruce J. Walker pointed out, is the
converse of virtualization because it makes many systems act as
one–also turned up a lot at the symposium. Walker said the topic goes
well with virtualization, because if sites want to use virtualization
as an aid to handling failover, they need to coordinate the computer
nodes between which the operating system crosses. He presented a
proposal at the symposium for adding a pointer to the kernel’s task
structure that (with a very small footprint and no impact on
non-clustered systems) would help clustering systems handle the need
to discover which processes were running on remote systems and to
communicate with them.

The wide range of sessions on kernel changes–including solutions to
improve storage management, such as multipath device access–were part
of evidence that every kernel task (caching, filesystems, etc.) is
being examined under a microscope to determine how it can scale
better, adapt to future evolution, and shave off waste.

Other Linux deployments received less attention at this symposium. I
noticed nothing about interesting but arcane deployments such as
robotics or carrier grade (telephony) applications. Unlike the
symposium I attended four years ago, this one gave just a nod to the
desktop. Instead, the desktop formed the subject of its own two-day
conference preceding this one, as reported in
a recent blog of mine.
A bit more at the symposium was offered on embedded systems. The
developers give a lot of attention to power management, which I
suspect is done for the benefit of embedded developers, but also
benefits desktop users who have laptops.

Concerns about power management have a major impact on support for
hyperthreading, as discussed by Suresh Siddha in his talk on Chip
Multi Processing. The driving factor is that power consumption is the
same on a chip regardless of whether just one thread is active, or
both. If optimal performance is your goal, you want processes
distributed among all processors, even if only one thread is active on
each. But this maximizes power consumption. So if power management is
a concern as well, the algorithm must be quite different, and must try
to fill the threads of each active chip while leaving some chips idle.

I was told that the 2.6 kernel is much larger than the 2.4 version
because developers honored the feature request list of sites running
big iron. The losers in this exchange are embedded developers, many of
whom insist on sticking with 2.4. The 2.6 version’s slow boot-up is
particularly detrimental to adoption for embedded systems.

Several talks offered practical advice on the use of debugging and
instrumentation tools to make developers more effective.

I attended the Fedora and Gentoo BOFs partly to see whether I could
detect any demographic or cultural differences in the attendees, but
they seemed pretty comparable. The Fedora BOF was much larger, of
course. Gentoo has an impressive following, though; it’s BOF was led
by two IBM employees who say it’s gaining adherents among developers
at IBM, and someone pointed out that the Mozilla Project runs its
servers on Gentoo.

To capture some of Red Hat’s and SUSE’s followers, the Gentoo
developers are considering a slower moving, more stable Enterprise
edition, but an Enterprise edition seems like an oxymoron to me for a
distribution that is known as the most adventurous, cutting-edge of
the popular distributions, and for a team that prides itself on
letting each user customize his or her installation.

Summary

Some of the talks at this conference were the most information-rich
I’ve ever attended. When I look over my notes, I am amazed how much
valuable information the speaker conveyed in just one hour.

The detail could sometimes become tiresome, to be sure. I don’t think
an audience was well-served by a description of a feature that goes
field by field or function by function. What made the talks useful
were their summaries of a feature’s requirements, history, alternative
or rejected implementations, and subtle implications of the chosen
implementation.

I sensed less concern at this conference about political trends that
could have an impact on Linux and open source. I just didn’t hear much
talk about them. Perhaps the vote of the European Union against
patents eased the worries of attendees. New respect in the business
community and larger public for open source (note the groundswell of
praise for Firefox) and the gradual receding of the SCO case may also
contribute to this lull in political hyperalertness–although more
threats are likely to arise.

A lot of the folks here are extraordinarily intelligent and capable of
extreme levels of dedicated effort. We’re lucky they’re obsessed with
such things as reverse engineering old video games or getting every
feature of power management to work on Linux. If one of them set his
mind on evil, he could take over the world. (On the other hand, he
couldn’t be as evil as the people who are taking over the
world.)

The deeper theme at this symposium is that open source is constantly
being revitalized by the astonishing energy and intelligence of those
drawn to it for whatever reason. And it’s making inroads in
little-known places. I mentioned earlier the reverse engineering of a
game that runs on Windows: the hacker discovered along the way that
this game uses Ogg Vorbis files for audio and Python scripts to
implement many of its rules. It’s hard to imagine a computer field
without open source–but then, no such field will ever exist.

Earlier blog on this symposium:

Ottawa Linux Symposium, 2005: first day

Related link: http://www.adams1.com/pub/russadam/upccode.html

CD Baby has UPC or EAN barcodes in our database for most albums. Problem is : we let our clients enter their barcode themselves, and we had people entering invalid codes! (1234567890, etc)

Here’s a PL/pgSQL function for PostgreSQL databases that, when called in your table constraints, will not allow invalid barcodes in your database anymore.

CREATE OR REPLACE FUNCTION valid_barcode(barcode text) RETURNS boolean AS $function$
DECLARE
  b text;
  odd int;
  even int;
  s int;
BEGIN
  IF barcode IS NULL THEN
    return NULL;
  END IF;
  IF LENGTH(barcode) < 12 OR LENGTH(barcode) > 13 THEN
    return false;
  END IF;
  -- normalize UPC and EAN to both be 13 digits
  IF LENGTH(barcode) = 12 THEN
    b = '0' || barcode;
  ELSE
    b = barcode;
  END IF;
  -- sum of odd digits times 3, plus sum of even digits
  even = CAST(SUBSTR(b, 1, 1) AS int) + CAST(SUBSTR(b, 3, 1) AS int) + CAST(SUBSTR(b, 5, 1) AS int) + CAST(SUBSTR(b, 7, 1) AS int) + CAST(SUBSTR(b, 9, 1) AS int) + CAST(SUBSTR(b, 11, 1) AS int);
  odd = CAST(SUBSTR(b, 2, 1) AS int) + CAST(SUBSTR(b, 4, 1) AS int) + CAST(SUBSTR(b, 6, 1) AS int) + CAST(SUBSTR(b, 8, 1) AS int) + CAST(SUBSTR(b, 10, 1) AS int) + CAST(SUBSTR(b, 12, 1) AS int);
  s = (3 * odd) + even;
  -- remainder to nearest 10 should be same as last check digit
  IF (CAST((CEIL(CAST(s AS float8) / 10) * 10) AS int) % s) = CAST(SUBSTR(b, 13, 1) AS int) THEN
    return true;
  ELSE
    return false;
  END IF;
END;
$function$ LANGUAGE plpgsql;

In your database table, then, you just need two things:
barcode char(13) (of course)
… and at the end of the table definition …
CONSTRAINT bad_barcode CHECK (valid_barcode(barcode))

That will not allow any invalid UPC/EAN barcodes in your database.

I’m a PL/pgSQL newbie - lemme have it - how to improve this?

Related link: http://www.oreillynet.com/linux/novell/register.csp?doc=getservices&sitesrc=LCHo…

Occasionally I skim a whitepaper. I feel a little bit guilty saying that;
I’m not a CIO or CTO — I think of myself as a developer and author. Still,
there are sometimes a few really good pieces of data to extract from even
press releases masquerading as tech marketing whitepapers.

(Vendors, beware. This isn’t a plea to e-mail me more whitepapers. I said
occasionally for a reason!)

One of the benefits from a liberal arts education ought to be the ability
to analyze a piece of writing and to understand the point of view of the
author. It’s interesting doing this with whitepapers because of how it can
reveal a vendor’s strategy. For example, take the linked Novell whitepaper
(registration required, sorry).

The paper starts by assuming that its readers already believe that
Linux-based systems have a place on the edges of their network, running web
servers or DNS, for example. It also assumes that the readers are considering
using Linux for more important services, such as in data centers or for
“mission critical” services.

That’s a big difference from seven years ago, when I had to scrounge a
spare PC to run a custom program I wrote to collect statistics on Tier 2
customer support requests for high-end laser printers. (A couple of years
later, the company started its own dedicated Linux strategy and floundered for
a while. I stopped paying attention.) The question is not “Can I run Linux
on a box under my desk for something I need?” or “Is LAMP a good option for
our internal or even external web sites?” but “What do I need to do to migrate
our most important internal systems to run on Linux?”

Of course, it’s always helpful to consider the audience. My guess is CIOs
and CTOs, based on the arguments about interoperability and vendor support.
Keep that in mind.

Another point is the quick assertion — as if to believers already — that
capable vendor support is as important as the openness of Linux-based systems.
(Robert Lefkowitz recently argued that the
true cost of software is a small part of its price tag.) At first, these
might seem to be at odds, but the paper then goes on to compare Novell’s
support for heterogenous networks, running multiple Linux distributions, with
that of other vendors. While there are many free software business models
built around the idea of providing support, it’s interesting to see a vendor
take the idea of interoperability and offer support for competing products
without necessarily encouraging migration.

Interoperability may be the most important concept in the latter half of
the paper. While someone like me might argue that openness and freedom are
important reasons to choose free software and an open source person might
argue for improved development processes and quality, someone managing a large
existing network has more practical considerations — such as not replacing
everything all at once, or even much of anything in the near future.

That’s the image that the whitepaper tries to paint. I can imagine how
reassuring readers that this is not (necessarily) a migration path is very
important to keeping their trust.

The rest of the paper is as you might expect; here’s what Novell does and
can do for you. There are useful tidbits for trend watchers: the debate over
the purpose and goals of vendor certification and training, remote management
of multiple distributions from the same tool, and legal indemnification.

Maybe tracing through a whitepaper to uncover a vendor’s strategy and to
learn a few things about the intended audience isn’t the most fun way to spend
a Friday afternoon, but it can be enlightening. We have a few more
whitepapers in our Novell Learning Channel that I’ll skim and analyze. In the
meantime, you might pass them along to your CxOs and IT managers.

Linux-based OSes — fated to be one-of-many or open systems eventually render proprietary platforms irrelevant?

Being the owner of a company, I constantly get people coming to me saying they’ve got the “solution” to some problem.

But… I *LOVE* programming! I love creating. I love the process as much as (or more than) the result! I’d rather DO something than have it DONE.

In other words:

I don’t want the solution … I want the problem.

But hey - I’m weird like that.

I am a musician. I spend a reasonably large proportion of my life creating and recording music. My home studio has all the marks of a musician - guitars, drums, mics, mixers and of course a computer. Despite the fact that pretty much all of my computers run Linux, the studio box is running Windows 2000 so I can use my sound recording tool of choice; Cubase.

From a Open Source consultant and advocates perspective, that computer is obviously a chink in the armour. To replace it with a Linux box and achieve the same results is a real challenge though. There are simply no multi-tracking applications on Linux that provide a comparative experience in terms of functionality and integration. Don’t get me wrong, there are certainly efforts going in to this area and applications such as Ardour, Wired and Rosegarden, but these tools face a number of uphill battles in winning me over. The interesting point is that the challenge is not focused so much on features but on usability and integration.

It is fair to say that the requirements for audio engineering are fairly complex. The need to record audio at different levels of quality, layer on further tracks, mix them, apply effects, edit waves, perform overdubs and mix down are all essential requirements for the sound engineer and musician. Each of these features is no or less important than the other, and they all play a key role in creating a quality recording. When you read through the feature list for many of these tools, they offer the kind of features that I am talking about above. They certainly allow you to record tracks, cut them, adjust their volume and EQ in the mixer, apply some effects and mix them down. On a hard technical level, the feature list is largely satisfied - it is the ’soft’ requirements that are the issue.

When you are identifying the requirements in any kind of software development, it is always essential to prioritise both the hard technical issues and also the ’soft’ social issues. As much as supporting the above features is essential, it is also essential to match the mental mode of operation that the user operates in. When people are recording music, this mode is creative, and technology is typically relegated to unimportance - it should just work. When I am making music, I don’t care for technology. I don’t care about the spec of amps and guitars, I don’t care about the technical characteristics of the mixer; I just want to plug in and record. The time between the birth of a song and getting it down on disk must be short - the creative mind is hampered tiny technical issues, and these issues are unacceptable. As such, any technical barrier in front of creativity is a real issue, and this where the Open Source solution really needs focus. The problems here are not just for those who create the multi-track software applications, but for the entire software stack from the kernel up.

The two flaws; integration and usability

Integration is a key problem in the current Open Source offering, and this is a responsibility of both the application developers and the distributors. If you try to run one of the many multi-track applications, it will need to talk to one of any number of sound systems. This not only requires me to understand what a sound system is, but I also need to dig through the documentation and determine which one it is, how I run it and in which mode. As I am sat there with my guitar resting on my lap, this is one of those frustrating technical barriers.

The integration issue is proportionate throughout the entire system. If I plug in a USB sound card, I want all of my applications to make use of it. Not only that, but I want to be able to configure the sound card from within my application. If you have a simple single channel sound card, the audio mixer will suffice, but if you have a complex card with 10 ins and 10 outs and multiple recording modes, you need an application to manage this. This is where cards such as the M-Audio Delta range fly on Windows - they come with a little control application to manage these parameters. You can certainly control levels with the ALSA mixer, but it will not allow you to deal with the many other options for the card.

The components in the system that do not affect the production of a recording from an interaction perspective need to be fundamentally invisible from the user.

Usability

The second issue is usability. Multi-track tools are renowned for being complex to use. This complexity is not necessarily an issue with the concept of recording audio into tracks, but the issue of having the requisite knowledge to spit shine the track with EQ, dynamics an effects to get the best out if it. This knowledge sits outside of the application. The same can be said for IDE’s - creating a project in an IDE is fairly straightforward; the challenge lies with understanding the code - an entirely separate issue.

The solution to this problem is presets. The vast majority of users who record music are recording within the established remit of a genre. As an example, I record a lot of metal. This genre has some common traits when recording - the guitars are present and fairly scooped, the vocals are up front but slightly recessed in the mix, the bass drum plays a prominent role and requires a ‘clicky’ tone with high mid-range. I also record the entirely opposite ambient/classical style, which also has modes of practise - warm acoustic stringed instruments that are layered and panned throughout the stereo field, very present and up front vocals, plenty of reverb and delay etc.

Each of these modes of practise can be reasonably implemented in sensible defaults throughout the entire application. This not only applies to effects, but to other areas. Some ideas:

• You could create a new song based on genre. As an example, for a rock band there are typically two guitars, a bass, vocals and guitar. This feature would create the tracks, name them and apply the default effects and panning.
• All effects need sensible defaults. The common effects such as chorus, reverb, compression, limiting, wah, flanger and others can all have reasonable defaults, and tools such as Cubase do include some impressive defaults.
• Mixing can also have reasonable defaults. EQ is a science that many don’t understand, and a solid set of defaults can satisfy both common mixing needs and special effects such as simulating AM radio and phone lines.

Many of the issues of usability can be easily solved by identifying the kind of steps required to achieve a common goal. For many people who record, they are often stood up holding an instrument in a small room filled by a band. Interactions with the computer need to be kept to a minimum. The kind of visual interface requirements for recording and the requirements for mixing are entirely different. Recording is simple - you need to manage the stream of audio coming into the computer and assign it to a track, with some minimum level management. Mixing is entirely different beast in which the entire range of features in the tool need to be readily at hand. Mixing is a process that you conduct on your own with a beer, recording is a process you conduct with amps, guitars and band members to contend with.

The application should also hook into the desktop be intuitive. Although Ardour has been touted as one of the tools with the greatest potential, a real sticking issue is the fact that it looks so drastically different from the rest of my GNOME driven desktop (Ardour uses GTK) and is rather unintuitive. With some experience behind me of using Cubase, Cakewalk and Magix Audio Studio, I suspected Ardour with be a cinch to pick up - unfortunately I found it impossible to be productive straight away. If I can’t use it, how is someone with no knowledge of audio recording supposed to use it? Ardour is certainly not the only offender here, and this seems to apply with a number of tools.

Finding a solution

The solution to the problem is integrating key, predictable components and making them work flawlessly. In all honestly, if I cannot download the software and make it work straight away without tinkering around with sound servers and such, it will not get a look in. Period. When you download and use Firefox it just works, when you use OpenOffice.org it just works, when you use The GIMP it just works - when you use Cubase it just works.

Part of this challenge is using comprehensive frameworks for building applications. It seems that GStreamer is becoming a very prominent framework with good support from a range of different applications for different desktops. In addition to this, HAL and DBUS are becoming the de-facto solution for managing hardware. with this in mind, hardware specific issues should really be directed to the kernel and HAL/DBUS teams. This will ensure that changes will propagate upwards through the stack and ease integration. From some discussions with the GStreamer and HAL teams, it seems that the kind of plug and play philosophy regarding hardware and software is becoming reality. With GStreamer and HAL shipping with all distributions, there is the opportunity for the application to just work. The work can then concentrate on being a great multi-tracker.

I am convinced the the problems discussed here have readily available solutions, but I think opening some dialog with the providers of different parts of the stack needs to happen to allow the solution to develop. Creating an integrated and usable system for audio engineering is something that will require cooperation from different parts of the community. This has worked elsewhere with other problem domains, and I see no reason why it cannot work here. Lets see how the story pans out…

What are your thoughts and experiences? Can audio production on Linux get easier? Can we achieve the simplicity experienced on other systems? Are there any interesting developments occurring that will solve these problems? Share your thoughts below…

Related link: http://perlcast.com/2005/07/21/perlcast-interview-with-brian-foy/

Josh McAdams from Perlcast interviews me about The Perl Review. We recorded the interview late one night at YAPC::NA, so I hope I don’t sound too sleepy.

Related link: http://www.perlmonks.org/index.pl?node_id=476730

A comment to an entry in Spidering Hacks says:

It is hopeless. This code never really worked and now Amazon has made it imposible to access their pages with Perl.

Oh really! It’s not so much that the poster thinks Perl can’t do it that he implies a computer program can’t do it.

So I wrote the program (in Perl) to do it.

I could probably use some Web Services API, but WWW::Mechanize turned out to be easier.

Related link: http://www.linuxsymposium.org/2005/

I’ve finished the first day of the Ottawa Linux Symposium, and already
feel I’ve had enough conference to last a long while. The last time I
got to an OLS was four years ago. In the intervening time they’ve
grown a great deal, learned a lot, and become even more
professional. Some 800 attendees from 37 countries are here.

Already, two speakers have made wisecracks about OpenOffice.org,
tagging it as a bloated memory hog. I have the suspicion that some
attendees see Linux as something to run for its own intrinsic value,
rather than as a platform for useful applications that can actually
help people accomplish something.

The keynote speaker was Jonathan Corbet of the highly respected
LWN.net news site,
and lead author on the two latest editions of
Linux Device Drivers.
A few months ago, he ruminated at his LWN.net site on the critique
that Linux doesn’t have a road map. His retort was that
there is plenty of information in full view about where Linux is
heading, for those with the time and knowledge to put it together.

Jonathan’s keynote built on this assertion by providing some
predictions (with suitable caveats) about the upcoming course of
Linux. These predictions include the entrenched use of git for Linux
source control, more support for clustering filesystems, consensus on
the use of SELinux for security, and various enhancements to improve
real-time response (including more preemption and I-pipe interrupt
handling). What was interesting about Jonathan’s talk, of course, was
not the laundry list of features, but his explanations of what
motivated them and why they seemed good solutions.

Another talk I enjoyed was the report by X Window System/Project
Athena leader Jim Gettys (just laid off by Hewlett Packard when they
closed his lab) on his gedankenexperiment about creating a
more responsive computer environment. His vision fits with the
pervasive computing movement that gave rise to Project Oxygen and
other experiments. In this world, we would all be able to look
virtually over a coworker’s or spouse’s shoulder at a remote location
and scribble on their screens. We could switch a remote control from a
TV screen to an audio speaker when a phone call comes in, and then
pick up a keyboard if the phone call demanded text input–all these
devices capable of operating independently and connecting with other
devices on the drop of a dime.

What was most interesting about Jim’s talk was not the grand vision
(they come a dime a dozen) but the set of stepping stones he drew
between the technologies we have now and the capabilities of the
future. He endorsed Zigbee, for instance, as a wireless protocol more
appropriate for the kind of flexible environment he put forward than
either Bluetooth or 802.11. He praised Zeroconf and distributed
caching filesystems such as Coda. On the other hand, he lamented the
lack of convenient systems for retrieving geographic information and
employing it to shape interactions.

Authentication is key to opening up information sharing, so that
people can share just what they want and protect the rest. Thus, Jim
wants X to have a concept of a user, just as the underlying operating
system does. Like Corbet, he sees an important role for
SELinux. Unlike Corbet (who is not enamored of SELinux) Gettys wants
it used by X for its security. He also thinks there’s potential in
SASL, which I find a rather heavyweight spec.

Another interesting aspect of Jim’s ambitions is that they are
imminent, not something that we’ll have to live a long time before we
see. He thinks the way open-source software allows everyone to
approach and play with the internals of all the components can bring
us a brave new world within a couple years–if the will is there.

One advantage of the close examination that a conference like this one
gives to its subject matter is that you see the unsavory
underside. Marcel Holtmann zipped expertly through a comprehensive
assessment of the state of Bluetooth on Linux (the BlueZ project) and
how far each protocol had come. Martin J. Bligh reported the
frustrations of making memory management robust on Linux. Even though
millions of sites are comfortably and reliably running Linux, the
basic operating system task of memory management has a way to go.

There are still ways that users (not even maliciously) can occupy
memory until there’s not enough left to keep the system going. Two
major memory crimes that contribute to this situation are pinning
memory (that is, leaving around references that make it impossible for
the kernel to free some routine house-keeping data) and creating large
numbers of dirty pages (memory that has been changed but that the
kernel hasn’t had a chance to write back safely to disk yet).

The former problem is deeply embedded in the kernel’s design. The
latter occurs in situations the kernel developers know how to check
for, but checking for it would add a substantial burden to routine
memory allocation. In fact, even instrumenting the kernel to help
developers track memory usage would add a substantial burden to
routine memory allocation. And since a program can often cause a
memory allocation simply by calling a different function or reading in
a new line of data, allocation cannot be burdened that way. The most
desperate act that the kernel engages in for self-rescue in low-memory
situations–killing a user process–often chooses a process that’s not
important or not particularly wasteful of memory.

A sign of this conference’s appeal is that a talk such as Bligh’s on
memory management could attrack hundreds of people, and that audience
members came up with several suggestions and specific requests. As I
pointed out, millions of users rely on Linux 24/7 and never have a
problem with memory management. But there are dozens of developers who
want Linux to be even better.

Related link: http://moon.google.com/

In honor of the first manned Moon landing, which took place on July 20, 1969, Google Moon is now open and showing NASA imagery. Google Moon uses the Google Maps interface to help you take a close up and personal tour of the moon.

Here’s the Google Moon FAQ containing an unusual announcement of Google’s future plans: “We usually don’t announce future products in advance, but in this case, yes, we can confirm that on July 20th, 2069, in honor of the 100th anniversary of mankind’s first manned lunar landing, Google will fully integrate Google Local search capabilities into Google Moon, which will allow our users to quickly find lunar business addresses, numbers and hours of operation, among other valuable forms of Moon-oriented local information.”

Zoom in too close, and what do you think you see? (Closer, that is, than the resolution provided by the NASA maps which are the underlying data for NASA Moon). Proof that the moon *is* really made of cheese!

Is this cheesy, or what?

Related link: http://dereksivers.com/rails-shared-controller.html

In Ruby on Rails, I was doing a lot of similar actions/views/methods on a bunch of different tables. So I put them all in one controller at the end of application.rb

Now the similar controllers just inherit from that one. Just tell the shared-controller what Model to use.

EVEN MY VIEWS could be shared, if I could just let the model tell the shared-view what form-bits to include. Kinda like Rails’ generator does.

To do this I came up with the form “Nub” (one form entry) - which can give the info unique to that form field. An array of Nubs is created in the model.

Anyway, I put the gist of it here, for the taking:

http://dereksivers.com/rails-shared-controller.html

Sorry there’s not more explanation. Lots of work I’m excited about, and no time to stop and teach. (Even putting that page together took an hour, so I hope someone can use it!)

:-)

- Derek

As I was 37,000 feet in the air flying across the country, I need a Perl module to extract information from a cooked template. If I was going to create that module myself, I’d call it Template::Extract.

The Comprehensive Perl Archive Network (CPAN) has so many modules that most things probably already exist. I figure out what I would call the module (and, since I’m a PAUSE admin, that’s often what they are called ;), then try to install it. I just assume it exists.

prompt$ cpan Template::Extract

The CPAN.pm module finds Template::Extract in my minicpan on my laptop (so I can install modules while on an airplane) and starts to install it, and it’s exactly what I want.

This happens quite a bit, actually, and I still think this is one of the reasons that Perl is popular. I’d be really nervous about Ruby and Python if they had the same thing (and I think they will, eventually).

Related link: http://conferences.oreillynet.com/cs/os2005/view/e_sess/7118

When someone mentioned the Leveraging Open Source for SOX Compliance session in the OSCON press call this morning, my first reaction was “Hey, I do that!”

At my previous company a couple years ago, a few of us began to recognise the need for serious change control within our IT department. Better still, we had a plan for implementing the tools needed to do it, without paying 6-8 figures to an outsourcing vendor to manage the design, rollout, and maintenance of a proprietary solution.

I was the lead programmer on the project, so I chose to take advantage of free tools — Apache, mod_perl, and HTML::Mason to handle the web serving and templating; an old intranet codebase I had written to handle calendaring, table widgets, and other miscellaneous tasks; and a fresh install of Debian to give me a nice development environment (and allow me to reuse an old x86 server that was literally pulled out of the scrap heap).

Sometime during the development of the project, Sarbanes-Oxley compliance became the big thing in the IT department, and strong change control went from “sure, that would be nice” to “we need that NOW”. Since there was no way to even complete vendor selection on a proprietary product in the timeframe we needed, my pseudo-skunkworks project suddenly became a key part of the company’s SOX compliance story.

A few months after go-live, I decided to switch from employee to consultant to pursue my graphics interests and gain more time to contribute to open source projects. I still spend a fair percentage of my time consulting back to the old company to add new features to the change control system as usage grows. With thousands of RFCs in the system at this point, and quite a few hours billed, I’m told the total project cost is still a fraction of just the license fees (let alone consulting, training, etc.) for the leading proprietary options. Better still, the company knows that they can get almost any new feature they want, at a price and in a timeframe they can live with. If they don’t need any new features, there’s no continuing cost beyond (minimal) hardware to keep the service up. There’s no vendor lockin, because all of the code is open, built with open source tools, and uses open data formats.

And that’s an open source SOX success story if I ever heard one.

How have you used open source technologies to address Sarbanes-Oxley?

I’ve been programming computers for twenty years. I’ve finally come to terms with the fact that I only know three things:

1. your code will change
2. no matter how simple you think your code is, it is complete nonsense to almost everyone else
3. if you haven’t tested your code then it’s probably wrong

Do you have your own equivalent to the “Diggins Laws of Programming”?

Related link: http://conferences.oreillynet.com/os2005/

The first week of August I’ll be in Portland for OSCON 2005. I haven’t attended an OSCON since my employer paid for the team to go to OSCON in Monterey. (When was that? 1999? 2000?) This time, instead of just attending, I’ll be blogging each day’s sessions here. I’ll be mostly following the Perl track, with a few other interesting bits thrown in for good measure.

OSCON these days is pretty big, with over a dozen tracks and far too many timeslots in which I wish I could be in several places at once. I’m sure there are other OSCON bloggers out there, and it occurs to me we could all live vicariously through each other by getting some good crosslinking going. Even when we have overlapping sessions, there’s a decent chance we’ll fill in each other’s gaps.

I’ve only picked my tutorials so far (I’d welcome advice for the sessions):

Monday

• Morning: Presentation Aikido (Damian Conway)
• Afternoon: Law for Geeks (Lawrence Rosen)

Tuesday

• Morning: Perl Best Practices (Damian Conway)
• Afternoon: Creating Passionate Users (Kathy Sierra)

What tracks/tutorials/sessions will you be blogging?

Related link: http://it.slashdot.org/comments.pl?sid=155776&cid=13060451

In response to this article about legal download music services, I think this comment is the funniest I’ve ever seen (IF you know the typical Slashdot comments on any story to do with digital music.)

There are people in this community who will continue to lobby against legal downloads no matter what the terms or what technology is used. I swear, sometimes I think that if Linus himself started a company that sold no-DRM OGG Vorbis songs for a penny a piece and you got a free blowjob from Natalie Portman with every 10 purchased tracks that we’d still see posts on slashdot justifying P2P piracy because we didn’t get to pick out Natalie’s outfit when she showed up at our parent’s basement to deliver.

Related link: https://adwords.google.com/select/

How does the value of a conversion relate to the return on investment of a Google ad campaign? This is pretty straightforward Business School 101. If you understand what a conversion is worth to you, and the percentage of CPC (cost per click) AdWords visitors who do convert (the conversion ratio), then it is easy to calculate your return on investment (ROI) for an AdWords campaign. If the amount each conversion is worth multiplied times the conversion ratio is greater than your average CPC, then your AdWords campaign is producing a positive ROI—and probably makes sense.

You could put this as an equation. For an AdWords campaign to make sense, then the following should be true:

Conversion amount * Conversion Ratio > Average CPC

Google’s underlying conversion-tracking mechanism bears a striking resemblence to the way Google AdSense works (AdSense is the program used to put Google contextual ads on your and my web sites):

• You add some special Google conversion tracking code to a results page on your site.
• You make sure that the results page will be opened when a visitor is converted, for example, by buying something (in the case of a purchase, the results page usually doubles as an order confirmation).
• When a user clicks your AdWords ad, Google adds a cookie to the user’s computer to track the user.
• When a user with the Google AdWords cookie on their computer opens the results page, a conversion is logged, and a special tracking message displayed to the user.

An interesting, and somewhat controversial, feature of Google AdWords conversion tracking is that as part of the tracking, Google notifies users that they are being tracked. This notification is produced by the Google-supplied code you add to the results page. A tracked user sees a message titled Google Site Stats with a “send feedback” link when the results page is opened.

Google explains that they prefer to be above board about their actions, and that the send feedback link is chance for users to understand Google’s privacy policies, and indeed to reject the Google tracking cookie if they wish.

However, most major advertising programs do provide conversion tracking options, and other advertising programs that track users and conversions do not “brand” the process. Users who click through ads in these other programs never know they are being tracked.

To summarize, Google tracks users coming through AdWords to your site by giving them a cookie. You decide when a conversion has occured by opening a page for your visitor (for example, to confirm an order - but the choice is yours!). When the two match (the cookie and the confirmation page) a conversion is recorded and reported in AdWords.

Cross-channel conversion tracking is a nifty feature within AdWords that also allows you to do conversion traffic coming in to your web properties from other advertising networks such as Overture. Taking advantage of this feature, if you are deploying ads across multiple advertising venures, means that you can use the powerful AdWords reporting facilities to aggregate your information about conversions in one place.

Do you advertise using Google AdWords? How do you decide if your ad campaign is working?

Related link: http://pragmaticprogrammer.com/titles/rails/

What the Pragmatic Programmers did with their new book on Rails is wonderful: they started selling it as a PDF when still in BETA - to please the eager, and get feedback/typos.

Now that I’m going through the book, this appeals to me on a few different levels:

#1 - I had a strong practical need for this book NOW - not in 5 months, but now now now. THANK YOU to the authors for making this available early. It has helped me immensely.

#2 - They have a wonderful error-submitting page that they respond to daily. I found a few typos as I was going through the examples, submitted them, and got a reply that they were fixed the following day. THIS IS BRILLIANT! Why wait until it’s on the bookshelves to find out that there are typos?

#3 - I prefer technical books on PDF anyway.

Releasing books in beta-format takes advantage of the fact that there are different kinds of readers. Some, like me, need the info sooner, even if it’s not “perfect” yet. We’re avid fans of the technology. We’ll hear on the mailing list that you are making this available. We’ll be right there giving feedback daily, which will improve the book for when it’s released to the much-larger public.

I hope more authors and publishers do this.

Related link: http://www.cdbaby.com/

It’s been over two months since my last post, here. Besides being the programmer, I’m still president of my company. It mostly runs itself without me, but for the first time in 3 years I felt things had gotten off-course, culture-wise, inside the company. I had to make a VERY tough decision to STOP programming for a couple months, while I turned my full attention to the company itself and the people that work there.

I put aside most of the last two months to sit with each of the 50 employees for at least an hour, talk about what they do and don’t like, take suggestions, and generally just being a part of the day-to-day workings of the company. Put my finger back on the pulse.

Spent lots of deep-thinking about the direction of the company. In some ways, we had become the inflexible beast that I was rebelling against when I started CD Baby in 1998. We got set in our ways about the CD itself, when the real point was to make a company that helps individual musicians sell their music. Many of the excuses we were making to people, about what we can’t do for them, came down to simple technical problems that I hadn’t put aside the time to solve yet.

Anyway - all of this was DEFINITELY a drain. Good for the company and probably good for my soul, but VERY hard. Not “exciting”, that’s for sure. Necessary.

Now I’m finally back at a lovely FreeBSD Konsole terminal and working in PostgreSQL, Ruby, Rails, PHP, XML, and all of those things that for me are “exciting”.

A couple books to recommend, that really got me through the last few months of trying-times…

For big thoughts about company direction:
Good to Great, by Jim Collins
http://isbn.nu/0066620996

One of the best books on managing people:
First Break All the Rules by Buckingham/Coffman
http://isbn.nu/0684852861

Above: Entrance to the Hynes Convention Center

Above: The Quark booth

Above: The Apple Specialist/HP booth

Above: On the floor

Above: The Berklee College of Music Dream Studio

DAN WOODS: Why do you favor the BSD license over other open source options for the IT context?

MARK BREWER: I should clarify that when I say I believe that BSD-based license technology is better for IT departments, I mean IT departments that are taking in code and potentially changing it, enhancing it for their own business use or incorporating it into other products or other software development projects. IT departments who are just taking in GPL code as is, and don’t intend to make any modifications, or go back to the community that owns that code and tell them or give those changes back, then they’re fine. I think there’s definitely a place for GPL-based software and I think there’s definitely a place for BSD-based software.
I am very pro-BSD for a number of reasons. One reason is that when you look at the successful projects that Covalent happens to be involved in, they have not forked, and they have not died like so many of the early advocates of the GPL predicted. Apache or BSD-based technologies have done extremely well, and the communities around them have done very well at enhancing and maintaining them, and not not forking them. Moreover, we’ve seen projects like Apache Tomcat flourish and grow, it’s in version 5.5 now and it is the most widely used application server out there. Another reason I’m an advocate of BSD is that I think software vendors as a whole should be able to take advantage of open source technologies without having to deal with all the issues that the GPL presents. In fact, the GPL is frequently, not compatible with other licenses software vendors may have, or other licenses they wish to use within their product. BSD-based software lets you do whatever you want.

DW: What are the advantages of using BSD for building applications and toolkits?
MB: If I’m a software vendor and I want to take some open source code and make it work for my customers — if I’m selling to a financial institution, for example — then I’m writing some specific functionality that’s going to address their business needs. Clearly, the BSD is going to be a lot more acceptable to that customer because they don’t even have to think about any enhancements to that open source code being given back to the community and getting into their competitors’ hands. If you’re writing applications, you need to pay very special attention to what license it is that you’re using. And, in cases like this, I believe that a BSD type of license is going to be much more appropriate to you than something that is GPL-based.

DW: But in the cases where you are using an open source project but not modifying it, aren’t all licenses basically equivalent?
MB: No. If it’s GPL-based code, you have to be more aware of how it’s being used, and how it might be included or compiled into another product. So you definitely have to pay more attention to exactly what you are doing with GPL code versus BSD code. The BSD license is pretty open, you can do whatever you want, however you want. The only requirement, generally, is that you need to acknowledge who the author is and who has copyrights.

DW: Do you think that the provisions of the GPL license are having a negative effect on adoption of open source because they’ve been used by opponents of open source to spread fear, uncertainty and doubt?
MB: Yes. Microsoft’s done a good job of that. I think even Sun’s done a good job of that. A lot of the big vendors who didn’t want projects that were out there under a GPL code to be successful used that FUD.
We are fortunate enough to have a large number of Fortune 1000 customers and I can name a dozen who specifically said to us, “We don’t allow our developers to download any GPL source code,” because they’re afraid of the consequences. There are very few lawsuits that have been filed against the GPL. But those that have been filed were because somebody simply forgot, or didn’t know that they had included a GPL-based piece of software in a product, and didn’t realize they were in any sort of violation.

DW: And this has been extended by the distorters to make it seem that if you even use GPL code all of a sudden your intellectual property is at risk.
MB: Exactly. The GPL is a much more confusing license agreement. So it’s easy to have FUD when somebody can’t interpret or understand the terms.

DW: But if someone wants to develop an open source platform that is immune to legal risk it requires ignoring a huge amount of open source that is under GPL. How would you advise people to make the tradeoff between re-creating the functionality of GPL and coping with the legal difficulties of GPL?
MB: If the functionality that you’re building your framework around is largely already done under a GPL license then by all means you should go with the GPL. Because, otherwise, you are re-creating all of that work and not taking advantage of what the whole open source community is about. The negative side is that you really don’t have a choice unless you want to start from scratch.

DW: If you were designing the next version of GPL, how would you recommend that it be evolved in order to make itself friendlier to IT adoption?
MB: One recommendation, and I believe this is going to be part of the 3.0 revision of the GPL, is to clearly address patents and copyrights. Be explicit about allowing people to still have patents on software that might be included in GPL code. Right now that is prohibitive. It’s not clear enough in the current version that you cannot have any patented software.
The second thing is that the viral nature of the GPL needs to be made explicit. The GPL license, which everybody interprets differently, says that if you take GPL-based source code and modify it or enhance it, then the derivative work or new product that you deploy must be licensed under the GPL. And some interpretations are even broader than that. This needs to be clarified. What is a derivative work? What does it mean to include GPL software inside of another application? And does that in due course mean that it has to be licensed under the GPL also? One of the big fears of software vendors—including even a company like Covalent who does everything on open source—is that I might take in some GPL code, modify it or include it into some other product or software package and not license it appropriately. There’s a lot of confusion out there, and even I have been confused at times when I read a software license as to whether this is supposed to be licensed under the GPL or not.

DW: Let’s say someone in an IT department recognizes an opportunity for open source use — they have the skills in-house, they are able to institutionalize them, they understand the software and how to use it, support it operationally and to support it based on the community — and then they run into a roadblock from a legal department or a risk management department that is ignorant of the real nature of open source licensing. What would your recommendation be in order to overcome those barriers and educate people about the risks associated with using open source?
MB: This comes up all the time. Somebody within an IT department decides to use Apache Tomcat or some other product that we happen to support and their legal department finds out about it and says, “Wait a minute, we can’t be allowing this open source software in-house.” They argue that there must be a way to either find a proprietary solution that does the same thing. The other option, and this is what’s happened with Covalent and other companies, is they find a vendor who takes that open source technology and provides enterprise software type of support, indemnification insurance and all the protections that go with it. Clearly, that’s one of the reasons we’re in business. A few years back a very large and well-known software house was considering using Apache in a product and their legal department said flat out no. They contracted us to come help them evaluate the risks of bringing Apache into their product and offering it to their customers. So it was mostly a legal discussion between our lawyers and theirs about the Apache license and what you have to do if you’re going to include it in your products. The BSD or Apache license is so straight forward that it wasn’t difficult to explain, and was, therefore, pretty acceptable to them. There isn’t a lot of risk around an Apache-based technology.
Now, if somebody asked me the same question about using a GPL or LGPL code, I would encourage him or her to take a close look at how that product is being used. Are they modifying the source code? And are they involved with the community? Ideally, they should have somebody who is doing development with that code involved in that community so they can keep up to date on fixes and so forth. And further to assure that any changes they might make get attributed back so that they’re not violating the GPL or LGPL.

DW: Why is the LGPL insufficient to solve this problem?
Brewer: Well, it’s better at solving the viral nature that we were talking about earlier. But it doesn’t change the fact that you have to license it under the same license if you’ve made changes to the code. It just makes it easier to include that software in another product and not be a derivative work. Additionally,it’s supposed to be used only for libraries, but we’ve seen the LGPL used for much more than just libraries.

DW: How do you think the alternative indemnification mechanisms, like open source risk management, have been accepted by IT departments?
MB: Not very well yet. The reason is that it’s still not the same as calling on a vendor. A vendor is a true throat-to-choke, right? If you feel like there is a problem, whether it’s an infringement claim or anything else, going to a vendor is a lot easier and they have incentive to address your problem.

DW: What are the five biggest myths in IT departments about the legal and IP risks of using open source that you and your sales staff run across when trying to sell Covalent products and services?
MB: The biggest legal risk that people are afraid of is that they have no indemnification coverage. And that isn’t a myth, it’s absolute reality — I don’t know of an open source license that explicitly indemnifies you or protects you in any way. So that’s why a vendor’s important. If there is some infringement claim made and they’ve taken in this open source software and used it in an application or used it somewhere in their company, then they are protected.
I think the second myth is similar to what we’ve been talking about. What are the license requirements? Not necessarily restrictions on use but requirements like including acknowledgement of who owns the copyrights or who authored the software. These are the two things that if you’re going to use open source you really must understand: your comfort level with the indemnification issues and a clear understanding of the license.
Another myth is that that if you just use GPL software that somehow you lose your right to intellectual property. And that one also needs to be clarified in the next version of the GPL.
And there are other myths, like that if you’ve made a code change of any type and didn’t distribute that code within an LGPL or GPL licensed product you still need to commit it back to the community. People think that they have to commit back changes under all circumstances and that isn’t true. In some cases you don’t have to. For instance, if you’ve made modifications just for yourself and you’re not distributing the software.
A few years ago we certainly heard a lot more of, “How do we get fixes?” If something was broken they didn’t feel that it could be addressed. Many of these IT departments don’t have the access to the community. They don’t have any way to get a hold of them and say, “Hey, help me fix this problem.” That’s less of an issue today.
Another thing that we used to have to protect a lot more against was the potential introduction of viruses. You know, big companies would look at open source technology and say, “It seems a lot more risky.” Because the source code is available to anybody so some hacker could write a virus and stick it inside the source code… We don’t hear that one as much, but it’s still out there.
And then of course there is the myth that open source has no quality control. I think that the last few years have shown that’s not true. Apache clearly is more secure, more reliable than Microsoft’s own web server, or Netscape’s web server. Why else are 78% of the web servers out there running Apache? As far as quality control and quality assurance or QA testing, it’s something that a vendor like Covalent provides on top of the community. But the community does a pretty good job, honestly. They do a good job with basic testing because the people who develop that code are very proud of what they’ve written. They really care about what it is that’s being used by the world and their community, and their cohorts in the community pay attention. When they see something wrong with a piece of code, they’re not going to be shy about saying you just developed or delivered a bad piece of code.

Is licensing really chooseable? Most of the time IT doesn’t have an alternative and so much is in GPL the it must be accepted.

Related link: http://www.desktopcon.org/2005/

The skilled organizers of the highly respected
Ottawa Linux Symposium
have tagged on another high-octane conference, the
Desktop Developers’ Conference.
The desktop conference will be held in Ottawa, Canada on July 18th and
19th, right before the main symposium.

I was very impressed with the caliber of the attendees at the Linux
Symposium I attended (and I will attend again this year), so I expect
that the Desktop Developers’ Conference, too, will be a place where
lots of real work gets done. The list of topics reminds me of those
discussed at the X Developer’s Conference (see
earlier blog),
which has been hosted a few times by Hewlett Packard’s labs. (And
Hewlett Packard is one of the sponsors of the Ottawa conference as
well.) People who want to contribute to the X Window System, to a
desktop, or to an application that runs on these projects, will
probably get a lot out of both the presentations and the informal
interactions at this conference.

The barriers to Linux adoption are probably more in the areas of
conversion costs, corporate politics, and training issues than in the
quality of the desktop. So forums for planners and business people,
such as the Desktop Linux Conference (see
earlier blog),
are helpful to increase use of Linux in that area. But lots of people
in X.org,
Freedesktop.org,
(another sponsor of the Ottawa conference), the desktop projects, and
Linux vendors, are trying to work together to make the desktop respond
faster, work on more hardware, support 3D better, and act more
predicatbly from one application and desktop to another. Conferences
such as this one and the X Developer’s Conference could remove a lot
of friction and speed up progress.

I have written about Tor before. In my opinion, the Tor project is an excellent effort towards protecting online privacy. I routinely use the Tor network, and it works well for me.

I believe that the merits of a project like Tor outweigh the channels of abuse it may grant malicious users. However, these channels of abuse do exist, and they cannot be ignored: if a malicious entity wants to scan or launch your network via the Internet, he or she can do this via the Tor network. This will make it incredibly difficult for you to track down the source of the attacks.

In order to demonstrate this, I setup a host on the Internet that I wanted to scan from my home network using the Nessus vulnerability scanner. Before an attacker can exploit a specific vulnerability, he or she will want to test the presence of the vulnerability using a scanner such as Nessus. Here are the steps I followed to launch the Nessus scan via Tor:

1) Install and Setup Tor.

2) Download desproxysocat (Thanks Chris!). This tool will allow us to setup a local TCP listener that will tunnel connections via the Tor SOCKS server (listening on port 9050).

Let us assume that the IP address of the host I wanted to scan was 10.0.0.1 (yes I know this is non-routable over the Internet, but I don’t want to publish the real IP address of my host). I invoked socat like this:

./socat TCP4-LISTEN:8080,fork SOCKS4:127.0.0.1:10.0.0.1:80,
socksport=9050

The above command causes socat to listen on port 8080, and tunnel all incoming connections to 10.0.0.1 (port 80) via the Tor SOCKS server.

[Updated July 12, 2005. Step 3 is not applicable now].
3) Configure privoxy to allow HTTP CONNECT requests via port 80. By default, only port 443 is allowed. To do this, configure your web browser to use privoxy as the HTTP proxy (127.0.0.1 81118) and browse to http://config.privoxy.org/show-status. Click on the “Edit” button next to the applicable “default.action” file, and choose the “Enable” radio button on the left side of “limit-connect”. Enter “80, 443” in the edit box and click on “Submit”.

4) Install and configure Nessus.

5) Launch a Nessus scan against 127.0.0.1 port 8080. Configure Nessus to limit the scan to port 8080 in the “Scan Options” tab.

Here are some of the entries in my Apache log that were a result of the scan:


192.168.1.1 - - [10/Jul/2005:17:29:56 -0700] "GET /Agents/ HTTP/1.1" 404 205 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:56 -0700] "GET /cgi-bin/viewpic.php?id=7&conversation_id=<script>foo</script>&btopage=0 HTTP/1.1" 404 217 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:57 -0700] "GET /index.php?err=3&email=<script>foo</script> HTTP/1.1" 404 207 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:57 -0700] "GET /scripts/fom/fom.cgi?cmd=<script>foo</script>&file=1&keywords=nessus HTTP/1.1" 404 217 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:58 -0700] "GET /scripts/viewpic.php?id=7&conversation_id=<script>foo</script>&btopage=0 HTTP/1.1" 404 217 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:58 -0700] "GET /Album/ HTTP/1.1" 404 204 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:59 -0700] "GET /fom/fom.cgi?cmd=<script>foo</script>&file=1&keywords=nessus HTTP/1.1" 404 209 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"
192.168.1.1 - - [10/Jul/2005:17:29:59 -0700] "GET /cgi-bin/wiki.pl?<script>foo</script> HTTP/1.1" 404 213 "-" "Mozilla/4.75 [en] (X11, U; Nessus)"

The 192.168.1.1 IP address represents the host that is the last onion router in the random circuit that was setup by the Tor software (NOTE: I don’t want to publish the actual IP address of the last onion router that I noticed in my logs, so 192.168.1.1 is a place holder to serve as an example). Given the design of Tor, it would be extremely difficult (if not impossible) to determine the source IP address (i.e. my IP address - and not the IP address of the last onion router). The above instructions can also be used to exploit software flaws (using tools such as Metasploit) in order to anonymously execute arbitrary commands on vulnerable hosts.

I am a big fan of the Tor project. However, potential attackers who may want to hide their tracks may abuse the anonymity granted by Tor. The aim of this post is to shed light on this fact, and I hope it is helpful to administrators are effected by attacks that seem to originate from a source that is a onion router in reality.

Related link: http://www.oreillynet.com/pub/a/oreilly/ask_tim/2004/perl_0707.html?page=last#th…

In this week’s “Ask Tim”, Tim O’Reilly answers “Is Perl Still Relevant?” He thinks it is. He’s also backing that up with major additions and updates his the Perl catalog, which I go through in my reply to his answer.

When I was a kid, I played with Lego. I didn’t go all crazy about the little bricks like some people, but I did enjoy building pointless little contraptions. As I grew out of Lego and my rather embarrassing childhood haircut, I have turned to sticking together other types of blocks. Technology are blocks I now play with.

After spending some time wandering round an incredible Lego shop in Birmingham, my memories of Lego got me thinking about how things fit together these days. Lego fits together because the blocks are the right size, the sticky bits fit in the other sticky bits, and of course, it is all made by the same company. It is pretty amusing how so many Open Source hackers were raised on possibly the first and most prevalent proprietary technology that we were exposed too.

The ability to put things together is everywhere. All my electrical appliances fit into the same plug sockets, my computer devices mostly come with USB leads, the nozzle on the petrol pump fits my car, my guitars fit my amp. It all just works. It seems that in the physical world, plugs are very important. Without plugs we would be, er, well…disconnected.

In the IT world, we are reasonably well connected too. Sure, many of you will balk at the huge range of Operating Systems, networking protocols, window managers, wireless standards and suchlike. Although the application level layer often provides a range of competing frameworks, it is still important to remember that the underlying structure is usually pretty good at doing the predictable heavy lifting. All windows managers use the X plug, all Linux applications use the kernel plug, all Operating Systems for PCs will run on a PC plug. At this level of plugging, the results are fairly predictable and rather unexciting. So, a window manager runs on X? Whatever. An application uses the kernel? Whatever. This is the nuts and bolts of the computing world. There is nothing special here, please move along.

The area in which my spidy-senses flutter is when you try to step beyond the predictable - when you put the instruction book back in the box, and you are left with some bricks and just your imagination. This is where the fun happens, and this is where the innovation begins.

Open plugging

In the last few years, Open Source development has gone on to become more integrated and more expansive. Only five years ago, the culture was more akin to passing the buck between a range of isolated islands. If a challenge confronted a specific project, the responsibility was often passed down the direction of the dependency chain. A specific feature in GNOME or KDE would often be directed onto the XFree86 team, and at that point the chain died. The lethargy and bureaucracy that riddled the XFree86 project denied a solution, and each side of the dependency became more isolated. The GNOME and KDE guys hacked on their code, the XFree86 guys hacked on their code, the kernel guys hacked on their code, and the users discovered continuity holes in the natural inclination to plug things together. The assumption that a floppy disk drive could plug into the desktop was simple, but the reality was different. The bureaucracy of the plug makers meant that the user ultimately lost out.

In recent times, this has changed. Further innovation in GNOME and KDE, the birth of the far more open X.org project, the proliferation of freedesktop.org and technologies such as HAL/DBUS/udev/inotify/gstreamer/cairo/glitz and others have meant that things do now fit together. The once difficult premise of graphically mounting drives, plugging in USB sticks, switching screen resolutions, configuring printers automatically and other common-all-garden IT chores have become second fiddle on the modern Linux desktop.

Facing the web

Although the Linux Operating System is making leaps and bounds in plugability, the same can’t really be said for web applications. With the humble web browser generally staying open and used for the vast majority of the day, web applications have become a big deal. Tim O’Reilly’s incisive thoughts on web applications as the new computer applications we all use is becoming reality. Amazon, eBay, PayPal, Google; they are stalwarts of regular computer users.

Aside from online hosted applications such as the heavyweights above, in-house hosted web applications are rapidly on the rise too. As a professional Open Source consultant, no end of people are interested in Customer Relationship Management (CRM), Content Management Systems (CMS), project management, resource planning, groupware and more. With a large and impressive stable of Open Source solutions such as EGS, Sugar, dotProject, Mambo, Wordpress, Hula, OpenGroupware and more, there is plenty of choice. The ease of use, choice and stability is not the problem here though, it is the plugability.

A typical SME will want a CRM, a CMS and possible some groupware. How do you fit them together? How do you make the CRM talk to the CMS? How can the events on a website hook in with a delegate list in the CRM? How can interactions with the CRM be reflected in groupware with shared calendars, email and TODOs? How can membership information on the website relate to real world CRM leads and statistics? How can an intranet Wiki pull in data from the other web applications?

These are not niche or edge case questions. These are the real questions that are asked on a daily basis by many of my clients. For the casual observer, a web application is seen in exactly the same light as any other application. Sure, you interact with all the web applications via the same interface (the web browser), but then again, all those games you play on your Playstation 2 are controlled and viewed via the same interface, all you electronic devices get electricity by the same interface. The consistency of plugs in other parts of our world makes light of the glaring inconsistencies of plugs in these other areas.

Some of you may suggest web services as the solution, but this is the equivalent of the plastic casing of a physical plug; it is merely a component inside a bigger picture. Web services provide a transport for web applications to talk to each other, but they don’t map the real world ins and outs of one application to another. If I were to plug my laptop into the wall, I don’t want to have to think about current and voltage - I want to plug it in and have it work. In the same way, I don’t want to think about RPC, XML, methods and state - I want to think about making this contact on my CRM talk to that web page on my CMS. I want to map the reality of one application with the reality of another.

Looking forward

My assumption is that much of this will happen in the future, but it will happen in a similar way to the Open Source desktop. As flag waving standards have come and gone, dominance and popularity will win out. In Open Source development it is incredible difficult to define a new standard or framework and get everyone to buy into it. It happens very, very rarely. This is no different in the web applications space. The challenge is in the perception.

Bolting in plugs and bridges between these different applications is only half the battle. Web applications are intended largely as centralised systems; systems in which you focus a specific aspect of your life on. All the contacts and activity tracking goes in the CRM. All of the public content goes in the CMS. The problem with these kinds of centralised systems is that developers often forget how people interact with the information as opposed to the system. People use email clients, desktops, desktop sticky notes, PDAs, watches, music players, SatNav systems, cars, phones, TVs and more. There is so much opportunity to plug these different disparate mediums together in the software world that makes sense in terms of usably streaming information between different systems.

The challenge is in developing good technology. The evolution of the Open Source desktop is a telling reminder that having the most hands up doesn’t win; its having the right hands up. Create good technology that developers are psyched about, and there is a real opportunity for innovation and change. The bricks are there - we just need to fit them together.

What do you think? Can this work or is it all pie in the sky? Share your thoughts here.

Related link: http://www.perlfoundation.org/index.html#googlesoc2

The Perl Foundation recently announced the list of projects accepted through Google’s Summer of Code program. There’s a nice split between Perl 5 and Perl 6/Parrot projects. Congratulations go to the acceptees and their mentors. (I am sad that someone didn’t pick up P5NCI, though.)

June in BSD. When the sun starts shining, and people start working on the summer projects; or spend time talk about each other.

Worth reading

• The Power of Words
• The Importance of RSS
• Network and System Security - and why products still suck.

• What drives software.
• The Worst Mistake in the History of the Human Race

Fun

• Infinite Cat Project
• Ten things I learned about the future at the Wired NextFest

Technical Recruiting and similar

(worth reading, for humour as well as information)

• The Talent Landscape at Microsoft

• Myth of the Chief Marketing Officer

When Employers turn against you

(downloading more from CVS than you upload means reappropriating trade secrets? Then your employer gets a court to impound all your computers)

• Health Market Science uses the methods of spammers?
• Legal Defence Fund for Perl developer criminally accused after whistleblowing

Misc

• Classic Shell Scripting - book from O’Reilly
• Collaboration at Dreamworks

• Steve Job’s very moving commencement speech at Stanford
• Microsoft Office 12 For Mac Will Support Open XML

• Damian Conway on his new book “Perl Best Practices”

• Tacit knowledge and cortical algorithms
• Google Earth

Linux and BSD compared

• by Linus

• by Matt Dillon

• by Theo de Raadt and also Christos Zoulas

Summer of Code projects

• “Google’s program designed to introduce students to the world of open source software development.”
• Groklaw interview with Google Project lead

• 28 BSD projects in total

• NetBSD’s projects announced

OpenBSD

Intelligent Infrastructure

• Is Linux For Losers? (Interview with Theo)
• OpenBSD’s ports framework - Interview with Marc Espie

post-Hackathon

• Henning’s travels through Canada

• TV Coverage

FreeBSD and NetBSD

FreeBSD 6 and beyond

• Interview with FreeBSD developers

Misc

• TrustedBSD Status Report from Robert Watson

• OpenBSD’s dhclient imported to FreeBSD
• NetBSD gets support for special strings in symlinks

Apple

Intel Inside

• How it began

• Macslash’s take

• The Rules Of Being A Mac User No Longer Apply
• Rosetta came from Transitive

• Some history.

iMusic

• iTunes supports podcasting
• New ipods

Apple opens up WebKit

(the engine behind Safari)

• Webpage
• comment