And it was yet another action-packed day. “Yet another” might be a significant phrase for the day, since Yet Another Society’s Kevin Lenzo appeared on stage with Larry Wall, Damian Conway, and Nat Torkington after the day’s keynote speech, and Kevin announced that the Perl Mongers and Perlmonks were to be merged into YAS. This, combined with Larry’s blessing and Damian Conway’s funding for another year, establishes YAS firmly as the central location for those wishing to fund Perl and other open source languages.

The rest of the day saw two real Perl tracks: Damian Conway, and everyone else. “The Conway Channel” was a three-hour presentation of all the modules that Damian had created so far during his indenture–from the sublime (Filter::Simple, NEXT) to the ridiculous (Acme::Bleach). Damian continued after lunch, with a talk on the skin-crawling horror of SelfGOL. Thankfully, I was spared the worst of the presentation by having to deliver my own talk about my mail filtering module, Mail::Audit. I got into Damian’s class to see him explaining how to create a loop by having a string of code perform a regular expression substitution on itself and then reevaluate itself. The use of y=[====y=]== to test whether an input had matched square brackets was particularly unpleasant.

His next talk described the mechanics of how Perl’s OO system despatches methods, and how you can change it with modules like his NEXT. Finally came the lightning talks. This year as ever, we had an excellent mix of the serious, the thought-provoking, and the merely insane. Jukka Juslin told us how to use Perl and graphing modules to make it easy to detect network intrusions; Yves Steyt told us how Perl manages Europe’s air-traffic control; Michel Rodriguez told us we need to get our own back on Java for inflicting us with XML; Carlos Ramirez told us about his perldoc.com. Schuyler Erle, a fellow O’Reilly Networker, told us why “pseudohashes must die”–a little redundant considering that Larry had already pronounced them dead in Perl 6 and they had been deprecated in Perl 5, but it was still and entertaining talk. Then came the first of Damian’s talks: “I Am The Very Model of a Poster to CLPM”, or “Everything I Needed To Know About The Perl Community I Learned In Kindergarten”. Enough said.

Jeff Pinyan braved the wrath of his peers by proudly demonstrating his use of Powerpoint, then went on to tell us how to write loops in Perl, and Nat Torkington reminded us of people in the Perl community doing hard work without necessarily taking the big lights.

After the break came my absolute favourite talk of the whole conference, by Jon Bjornstad. Jon’s friend is a quadriplegic who cannot speak by herself. Jon wrote her a Perl/Tk application that she can use by moving a mouse pointer with her head; she can select letters and words and put them together into a sentence, with speech software producing the sentence for her. Jon’s application also allows her to look at pictures and read books on the screen. I could not imagine a better example of Perl used to improve the quality of life.

Autrijus Tang told us about the use of BBSes in China and how Perl is helping to build up the BBS network while restoring privacy to its users. Scott Penrose gave two short talks on two modules he’d written, and Charles Engelke, a self-confessed manager, told us how Perl got him coding again. Dave Cross reminded us–jokingly, I hope–that if we get more people programming Perl, we get more stupid people programming Perl. Geoffery Avery advocated the deprecation of h2xs in favour of a module of his own devising, and once again Damian Conway stole the show with a Shakespearean dialogue with Brian Ingerson on the provenance of Inline::Files.

And suddenly it was over. I think it’s a good sign that a conference leaves you wanting more, and certainly if it makes you feel more inspired about Perl rather than less. The Perl Conference 5 and the Open Source Convention 2001 certainly fired me up on so many levels; there’s a lot of important work to be done on Perl 6, and there are so many people around me doing great and interesting things. I think the highlights for me were Larry’s State of the Onion since it gave us so many new directions; the perl5-porters party, as it’s always enjoyable to see the Perl community at play as well as at work; Mark-Jason’s tutorials, which proved to me there’s always more to learn about Perl, and finally, Jon Bjornstad’s lightning talk, which for me summed up what Perl was about: not just improving things on a technical level, but on a deeply personal level as well.

I had an excellent time at TPC, and I’m looking forward to the next conference, as well as to getting the time to play with the ideas I’ve picked up from here. But now, it’s time for me to continue my tour of the U.S., heading on to New York, Washington D.C., and Philadelphia. Until I get back to the U.K., happy hacking!

Look out, firewall administrators. Get ready to update all your filter tables, because TCP and UDP have been joined by the Stream Control Transmission Protocol (SCTP).

SCTP presentation

SCTP can deliver multiple streams of short messages between sockets. One of its “big wins” is that it permits multiple IP addresses to be associated with each connection, and if one connection fails the streams will automatically switch to the next.

An admirably lucid presentation on Friday explained how the protocol works and how to program it in a pending Linux implementation. Testing has shown that the protocol involves less overhead than comparable applications using TCP (at least at the user level and in CPU time; kernel performance is expected to be better too); this is attributed to its careful alignment of data on 4-byte word boundaries.

A single new call has to be added to the standard socket API: bindx, which works like bind but adds or removes an address to the same connection. Another bit of a kludge is an extra data structure that contains the number of the stream used by each message; this structure must be queried through getsocketopt. Recommended web sites are http://www.sctp.org/ and http://www.sctp.de/; the Linux implementation is the lksctp project on SourceForge.

Netfilter (iptables) BOF

Speaking of firewalls, there’s a lot of new features on the way from the team developing Netfilter. Some of these seem to me to be reasonable extensions that bring the treatment of various network parameters up to the level of the ones currently recognized. Others scream “Bloat! Bloat!” Harald Welte, who presented the changes at Friday’s BOF, distanced himself from many of the changes while defending others. The audience vociferously recommended that the team devote itself to creating a robust test framework and recruiting testers. This sounds like good advice to me, but coding new features is fun while testing is merely indispensable.

The upcoming features that are certain to be released soon include:

• Extending the “expectation” feature (which looks inside packets to determine the state of the connection in regard to the application running) to support multiple expectations. This is useful for some complex applications like IRC.

• Stateful failover: if the firewall machine fails, the rules will be all ready and up to date on the box that takes over.

• Allowing the tracking of connections to be restricted to particular interfaces, so you don’t suffer the overhead of tracking an interface where you haven’t installed any rules.

• Greater efficiency when a rule is changed: the kernel won’t have to reload the entire table containing all the rules.

Of the other changes, the only one that sounds both imminent and of widespread interest is a logging facility called ulog. Currently logging involves an expensive formatting and writing of a message at the kernel level. Ulog, by contrast, will perform initial filtering and send data to a user process, which can then do any kind of time-consuming or complex processing desired.

Elsewhere in the conference

I had a chance to hear a talk by Miguel de Icaza, who is always good for an outrageous statement or three. If I heard him right, he said that intellectual property issues “killed Sun and Java.” He reiterated his comments from the Open Source conference that Microsoft is encumbering .NET with patents, but that the Mono project will work around them and still manage to produce a complete implementation.

Richard Gooch presented his Linux Device File System (devfs), which solves the crisis in device numbers. (Traditional UNIX is limited to 256 major device numbers, and Linux is fast approaching the limit; Linus Torvalds has called for no new major device numbers.) Gooch was sensitive to the considerable flack he’s received, and spent a good deal of time arguing that the system was not bloated and provided adequate security. Regardless of problems in devfs, the idea looks quite elegant to me and I can tell that something like it is sorely needed. When it’s configured into the kernel, devices occupy a flexible, hierarchy space like the /proc directory and drivers are responsible for creating their own devices when loaded, a considerable saving of trouble for the system administrator.

The Ottawa Linux Symposium is still the place where things happen, but it has its ups and downs just like any other conference. Just because somebody has a novel idea does not mean he’s got a viable innovation–I reserve “innovation” for something that has widespread use over a long term. Some of the projects that looked exciting on paper turn out to be just clever hacks when I hear the presentation. “Linux contains lots of hacks,” said a fellow attendee, but I believe there’s a qualitative difference between the technologies we’ve come to depend on and some of the new things being proposed. A couple presentations got so dull I was left deciphering the code on the T-shirt of the person sitting in front of me.

My exposure here has convinced me that the peer-to-peer meme I’ve been researching is relevant to our era. For instance, a presentation I saw on fail-over servers created (without calling it peer-to-peer) a configuration of peers listening to each other and trying to jump in as soon as one failed. The issues explored in the Peer-to-Peer book were largely relevant here: authentication, presence, avoiding DoS attacks, and so on.

Speaking of peer-to-peer and innovation, I learned some interesting background about Napster from system administrator and free software contributor Pierre Phaneuf last night. He claimed that good old Internet Relay Chat contained all the features of Napster, and that Napster introduced no technological innovations but simply provided those features in a manner that was trivial to use. IRC had bots that let people announce they had certain files, and at any time a chatter could invoke the Direct Client to Client (DCC) protocol to obtain the files. It was just a pain to get working. Napster provided an interface–and not a pretty interface, either–to the file-transfer feature allowed anybody to get the job done. And thus does the world progress, supposedly. Neither Pierre nor I felt that the demise of Napster was a great loss to the public, although we deplore the legal machinations that brought it about.

The morning started with the eagerly-anticipated Craig Mundie versus Michael Tiemann keynote talks and debate. When I first heard that Craig Mundie had been invited to speak here, I was a little sceptical of Tim O’Reilly’s intentions. (And a little concerned for his sanity) But the gamble most definitely paid off. While the audience was, for the most part, civil - yet still not afraid to recognise soft soap and disingenuity for what it was - Craig Mundie put across Microsoft’s case with discretion and care. One could almost be justified in accusing his opponent of playing to the crowd, but on the other hand, the temptation of getting a couple of cheap laughs from the home crowd was a little too great to resist. The panel debate afterwards was particularly engaging, with Craig Shirkey managing to tie down David Schultz to a simple “yes” to the question of whether the Hailstorm architecture would allow him to have two non-Microsoft operating systems take part in both ends of a Hailstorm transaction. Of course, there was a catch - Hailstorm instances would “want” to use Microsoft’s services. I wonder what that means.

After the panel debate, the Perl talks began in ernest. I heard Elaine Ashton tell us how to grok the CPAN, which neatly segued into my talk about “10 Perl Modules I Wouldn’t Go Anywhere Without”. After that, I took some time to hang out, get chatting with people, and prepare for my talks after lunch - a demonstration of how to port other languages to the Perl Virtual Machine, and a two-hander with Kirrily Robert, one of my collaborators on the Reefknot project. Dan then continued the Perl 6 theme which has dominated this conference with a description of the Perl 6 internals. As if to demonstrate that Perl 5 had not been forgotten, Jarkko outlined his plans for 5.8.0 and its maintainance.

Julian Cash from SuperSnail was on hand to take some pretty dynamic pictures of us all, including a great group series from the #perl IRC channel.
Right now, it’s time for me to get myself over to the perl5-porters party - even us hard-working hackers need to let their hair down at times!

Topics at the Ottawa Linux Symposium span the many layers of Linux–from hardware issues like cache mechanisms, Flash memory chips, and hot pluggable devices (known to the rest of the world as Plug-and-Play) at the bottom to futuristic GNOME applications at the GUI level.

A barrel of Ximians

Ximian programmers working on the GNOME architecture and applications are getting a pretty enthusiastic reception here (as well as the expectable grilling about standards support and reliability). They have triumphantly picked up Miguel de Icaza’s crusade to transform the expectations of the Unix community, and implicitly (or sometimes gleefully) criticize the usability of old applications from the X Consortium or the GNU project.

Down with ASCII–long live rich, displayable, typed objects! Down with rc files–long live run-time discovery of interchangeable components!

In Evolution, which is both a platform and an application suite and attempts to provide all the applications that office staff need to use daily, the object model extends down to a single column in a table. Each column knows whether it’s a number, a string, a date, or whatever, and can be displayed and sorted appropriately.

Speakers from Ximian insist that applications should be “consistent and pretty” (to quote GNOME hacker Federico Mena Quintero), that it’s ridiculous in this day and age for anyone to use a mailer that doesn’t display HTML or handle MIME types, and that basic activities like printing should be offered with a full-featured interface such as paper selection. Mena Quintero, to begin his presentation on GNOME, declared that, “People who come from a proprietary world come to expect certain features in their applications.” To underscore his rejection of the old free software world, when discussing the development of the GNOME printing system he said pointedly, “The GhostScript code was completely unusable.”

What’s good about GNOME, however, despite their unabashed admiration for the look and feel of Microsoft Office and the success of its component architecture, is that they are doing things that should go far beyond what is imagined by Microsoft. The Bonobo CORBA interface, along with a sophisticated storage component called Wombat that is shared by many applications, should enable rich collaborative applications. There are some interesting side features, such as the index used by several applications to speed up searching and demonstrated by Ettore Perazzoli during his Evolution presentation. Thanks to pre-indexing, a word search on a huge mail folder or an address book finished so fast I didn’t even have a chance to see the screen update itself.

The social implications

Every conference on free software this days is turning into a political event. It’s happened at O’Reilly’s Open Source and Peer-to-Peer conferences, and it happened at a reception last night where Hugh Daniel of the FreeS/WAN project spoke. He covered the travesties of the DeCSS case and the arrest of Dmitry Sklyarov, urging people to vote, to discuss issues with their friends, to follow new bills introduced in their governments, and to protest like hell against bad bills. (Significantly, I didn’t hear Daniel say we should argue in favor of good bills.) Most of all, he ordered the audience, “Do not stop writing code that annoys my government” (U.S.).

The Ottawa Linux Symposium is the primary forum for discussions about the future of Linux. If you think of most boating conferences as being for yachtsmen, the Ottawa Linux
Symposium is like a conference for the people who design and build hulls.

Right now, most people (those not mesmerized by their own laptops) are huddled in small groups in the common area, murmuring informally about obscure technical topics. It seems appropriate that the conference should start, not with the keynote, but with Working Groups, which are small groups in conference rooms murmuring informally about obscure technical topics.

Yet I gave up the chance to attend this year’s O’Reilly conference on Open Source to be here, and as the days go by I will be trying (hampered by a buggy Mozilla interface) to derive some meaning for the wider public in the creations that emerge here.

People seem to be relaxing here in Ottawa in relief from the heat found in the U.S., of both the meteorological and the legal kind. I have already held an intense discussion with an O’Reilly author and given my card to someone who wants to be one. The first author, Karim Yaghmour, explained to me why Linux has succeeded where Windows failed in spanning the range of systems from a wristwatch to a high-availability server. Essentially, the very aspect of Linux and Open Source that creates a frustrating exprience for current users–the semi-random agglomeration of software components from many different projects–also gives it the beautiful flexibility that enables different system integrators to assemble the precise system they need. Furthermore, Karim said, Linux was pulled into new platforms and domains by people who wanted to make it work, rather than being pushed into such domains as Windows was pushed by Microsoft.

I have also arranged a massive give-away of O’Reilly books (including our new edition of Linux Device Drivers) as part of a raffle held by AMD at a welcome reception this evening.

Well, I arrived at the Perl Conference 5 on Sunday night after a long night and day travelling. It wasn’t long before I’d got together with people I’d known from previous conferences as well as people whose names I’d never been able to put to faces.

Sunday evening, I had dinner with Eric Raymond and Chris di Bona, two people I’d been meaning to meet up with for a long time; we had a good chat and an excellent Mexican meal down in San Diego old town. However, I had to get back early to finish off my preparation for my tutorials in the morning.

Monday was a very busy day for me - I was giving two tutorials, which meant six hours of talking. My first tutorial was about Unicode and Perl; around 30 people came to hear what Unicode was and how Perl supports it - I was excited to hear that a fair number of them were already using Unicode or were planning to do so immediately.

The afternoon session was a lot more fun - I was teaching Perl internals, with a class which really wanted to listen. This meant we could make the tutorial pretty interactive - I got asked a couple of surprising questions I didn’t know the answer to, so I dived into the Perl sources and together we figured it out. Some people came away with ideas for things they wanted to hack on, and everyone came away with more confidence about diving into Perl’s guts and knowing where to find what they’re looking for.

In the evening, we had the perl5-porters meeting, a fairly busy town-hall style meeting of the Perl developers. People got up to talk about various things: Jarkko talked about what was new in 5.7.x and where 5.8 is going, and Sarathy talked about the maintainance of 5.6. (Once again trying - and failing - to give up the pumpkin!) Hugo, who Jarkko had nominated the 5.10 pumpking, talked about what was going on with the regular expression engine. Dan Sugalski and Nat both talked about Perl 6 - Nat from the project status point of view, and Dan from the internals design. Jeff Pinyan talked about proposed new regular expression features, and Jeff Okamoto asked us how we would like to support IPv6.

Finally, Adam Turoff kicked off the big debate - how we should “recommend” a set of modules for Perl users, and how we should define what goes into the Perl core. The main consensus should be that Perl should allow the user to select, just before installing itself, a set of “personalities” or “tasks”, which define modules which it will then download from CPAN. Although we were pretty much agreed on what to do, there wasn’t really an agreement on how to do it or on the details, so we decided to push back discussion onto perl5-porters.

It was a busy day for me, but definitely an enjoyable one. It’s such an excellent feeling to sit down and talk face to face with people you’ve been working with for years; it’s also great fun to spot your Open Source heros as they wander around and relax. I’m very much looking forward to tomorrow, when Mark-Jason Dominus is teaching all day on advanced Perl topics, and Larry Wall will be giving us his State of the Onion presentation - where we can expect something hopefully not quite as mind-blowing as last year, but pretty mind-blowing nevertheless…

Many of the talks at ISMB 2001 on Monday and Tuesday focused on networks and modeling — serious science for serious scientists. “Inferring subnetworks from perturbed expression profiles” and “Rich probabalistic models for gene expression” were just two examples. It’s fascinating to see the cross-disciplinary side of molecular biology, but I have to admit, the science and math were over my head. Like a well-spread smorgasbord, however, there was still something for everyone in the schedule — including undereducated molecular biologists like myself. So I attended several sessions on the web’s role in research, and on bioethics.

The Web’s Role. Bernardo Huberman, a fellow at Hewlett-Packard, director of the Information Dynamics Laboratory at HP, and one of the discoverers of chaos theory, gave a very interesting talk about the phenomenon of the web. As most of us are well aware, the web is a revolution in communication. We can encode data and pass it around the world at essentially no cost. Thanks to the web, bioinformatics is not only a fast-growing discipline, but it is also a remarkably international one. A scientist in the U.S. or Japan with a fast internet connection can now download hundreds of megabytes of data from from a public database in Switzerland in a few minutes for nearly no cost. This was not the case even five to ten years ago.

But the ability to download information and mine databases over the web also raises issues of privacy. At the moment, much of this information is freely available, but should it be? What kinds of uses can DNA and protein sequence data be put to? The answers aren’t clear yet.

One of the most interesting concepts I took away from Bernardo’s talk was how the theories of cryptography that we now apply to computing may be applied to biology. DNA and proteins are incredibly complex. We only understand a fraction of what they do and how they work, but our cells know how to decipher these complex codes, and synthesize the compounds to keep our bodies running.

Bioethics. Later in the day on Monday I attended a workshop on bioethics and “fiction science” organized by Pierre Baldi from U.C. Irvine. Fiction science, in Pierre’s words, is “the rational extension of current technology.” What happens if you extend an idea such as cloning through a rational pathway? You enter the realm of bioethics.

We’ve passed a number of thresholds in the last few years and are on our pay to passing several more. In two decades (if Moore’s law holds) we’ll have computers that exceed the capacity of the human brain. This will shatter our notions of artificial intelligence.

Though we’ve been tinkering with evolution and genetics for generations, we now have the capacity to clone ourselves, to produce babies in vitro, to “operate” on proteins, and genomes as if they were software, and put DNA from one species into another. For example, scientists have inserted a gene from fireflies into tobacco to produce a plant that glows in the dark. The results of this sort of genetic tinkering may be very beneficial — perhaps a cure for cancer will be found. Perhaps not.

What are the issues of identity if you clone someone? How would you feel if you entered a room with a thousand clones of yourself? Who should be allowed to clone whom? Should we limit how many times a genome can be cloned? The panel of scientists at this workshop explored many questions, and as you might have guessed, there were no answers.

The ramifications of this session, and bioethics in general, are too large and too complex to sum up as I stand here typing at one of the conference terminals. I’ll need to think about them and do some research before I feel confident enough to speculate more deeply on what I heard.

Many people attended this session and expressed concern about how scientists in general should approach these issues. I was relieved to see that researchers in molecular biology are as worried about the outcome of cloning and stem cell research as any other member of the general public. Hopefully, those worries will drive them to continue exploring ethics and morality as fiction science becomes reality science.

			Developing Bioinformatics Computer Skills By Cynthia Gibas & Per Jambeck April 2001 1-56592-664-1, Order Number: 6641 446 pages, $34.95

ISMB 2001 — the Ninth International Conference on Intelligent Systems for Molecular Biology — is in full swing this week at Tivoli Gardens in Copenhagen, Denmark.
For those of you who’ve never been to Copenhagen, Tivoli is a combination of amusement park, public gardens, and food hall. The park was constructed in the mid-nineteenth century, so it has a kind of other-world, other-time feel to it. At quiet moments during the talks you can hear kids screaming as the roller coaster rumbles by. If you step outside the conference hall you see young couples, and mothers pushing baby prams (the old-fashioned kind with four big wheels and a spring suspension), and families, complete with kids and grandparents. All of Copenhagen is here enjoying a summer’s day, unaware of the 1,300 slightly geeky biologists talking about proteomics and gene sequence analysis not 20 yards away.

The first keynote was given on Sunday by Christopher Dobson, a decidely un-geeky researcher from Oxford University in the U.K. Chris’s eloquent and thoughtful presentation on protein folding and its potential impact on the disease process was, in my opinion, the epitome of a keynote. Hours later I’m still thinking about what he said, what it means, and why biology is such a fascinating science. At a meeting like ISMB it’s easy to get caught up in the scientific detailia and lose sight of the big picture. Why does bioinformatics exist? Not for its own sake, but to help answer such questions as, why do proteins fold and what happens when they don’t?

Here’s an example. After a protein molecule is synthesized by the cell’s ribosomal machinery, it folds up into a very complex configuration. The chemistry of the molecule affects how it folds, but exactly why it folds is one of the big, open questions in science. So what happens when genetic coding for a protein goes a little wrong? Let’s say a genetic disorder affects how your body codes for and synthesizes a protein called alpha-1 antitrypsin. The protein is miscoded, it folds incorrectly, and eventually, you wind up with liver disease. But how does it happen?

As a result of research in his lab, Christopher has found that the folding patterns of certain proteins that are left to age in high concentrations are somehow changed. They begin to aggregate and form a new kind of molecular structure. As it turns out, this new structure is toxic, and often causes the cell to die. The implication is that as a protein ages, it begins to break down. It’s an interesting parallel to the fact that as our bodies age, our biological systems begin to break down.

So why might some proteins cause disease by aggregation? It’s likely the biology of the system that’s to blame, not the gene sequence. As the body produces proteins in higher and higher concentrations due to genetic misfire, the proteins themselves trigger failure of the biological control mechanisms that prevent aggregation in the first place. The cytotoxic (cell death) effect of aggregated proteins could be a bodily defense mechanism (the errant cell dies before it can reproduce), but if enough cells die, you have another problem. I’ve simplified the science (and I hope I’ve simplified it correctly), but the end result is a fascinating glimpse into how a biological system works, and how it breaks down.

Christopher Dobson’s resarch may give us some new insights into how genetics, protein chemistry, and aging are related, but much more examination is needed to prove his hypotheses. And if they are proven, yet more effort will go into looking for a solution to the problems they uncover. The tools and techniques of bioinformatics may help us handle all the data, and perhaps offer some insights into possible solutions, but not before basic biological research shows us how systems such as protein folding work.

The point of this long story is is that bioinformatics and biological research exist together. They’re two sides of the same coin, and one is of no use without the other.

BioPerl

			Developing Bioinformatics Computer Skills By Cynthia Gibas & Per Jambeck April 2001 1-56592-664-1, Order Number: 6641 446 pages, $34.95

Since Perl is such a great language for manipulating biological data, it stands to reason that the bioinformaterati have banded together and created their own set of Perl modules for biological applications. Collectively these modules are known as BioPerl. Volunteers from both BioPerl.org and the biological community maintain the existing code base, and contribute to it as need arises. As Hilmar Lapp, one of the core BioPerl developer/coordinators said at the meeting, “BioPerl is not a religion. The modules exist because they solve problems.” Given the rate that BioPerl has grown (The core development team expects to release version 1.0 later this year), it’s clear that the modules have made bioinformatics research much easier.

One of the most popular BioPerl modules allows you to do automated parsing and conversion of the major biological database formats such as GenBank, EMBL, and Fasta. As is often the case with resources that have developed over time, biological data formats are not consistent or interchangeable. For example, you must convert sequence data from GenBank format to Fasta in order to use BLAST (a sequence alignment tool) to compare your data with GenBank’s. The conversion itself is easy, if you’re only comparing one sequence to another. It becomes incredibly tedious and time consuming when you’re comparing hundreds or thousands of sequences, or entire genomes, which is how most biologists approach it. Thankfully, Perl is well suited to this sort of data manipulation. Once you’ve converted your data and completed your BLAST analysis, you can use another BioPerl module to parse the multi-megabyte file full of results. And this is just the tip of the iceberg. BioPerl also includes modules for sequence translation, batch retrieval of records from a public database, and much more.

BioPerl is one of the oldest of the bio projects (its siblings are Biopython,
BioCORBA,
Bioxml,
BioJava,
and the most recent, BioRuby) and it has made great progress since it started as a loose collection of scripts. Right now BioPerl focuses on sequence handling, annotation, and analysis. In the future the core team hopes to add modules on visualization and GUIs, and establish a script repository. As Hilmar said in his closing statements, “There’s still lots more to do, so remember: if you need it, code it.”

This is open source at its best.

I3C

Lunch is the perfect time to make announcements. Everyone is happy, well-fed, and hopefully, open to new ideas. The BOSC organizers have figured this out. Between the sessions on BioPerl, they snuck in Eric Neumann from Beyond Genomics and Maciek Sasinowski from Incogen to tell us about the Interoperable Informatics Infrastructure Consortium or I3C, an organization dedicated to exploring interoperability in life sciences using XML technologies. The I3C will “serve as the international organization for global coordination and cooperation for the convergence of Information Technology in Life Science Research. It will promote and maintaining a broad spectrum of activities focused on the development and availability of standards, solutions as well as associated technologies.” A number of key organizations are involved including Blackstone Technology Group, Incogen, Oracle, Sun Microsystems, TimeLogic, LabBook, IBM, BIO, and many others.

The I3C is taking an open source approach to development, and plans to become a global repository for information and education. They feel that XML is a key component of the current “best approach” to informatics, and they and their members are developing protocols in bioinformatics using ebXML (electronic business). Eventually they plan to expand into other areas of informatics such as metabolomics, pharmacogenomics, proteomics, cheminformatics, and others.

I think that the I3C is an organization that defintitely bears watching.

You know you’re in Scandinavia when the carpeting in your hotel room is electric blue and your towels a complementary hot orange. No hotel in the US would have the nerve to use color the way the Danish do. Everything in Copenhagen works well, from the airport (blissfully uncrowded), to the trains (on time), to the people (who can switch from speaking Danish to perfect English and back again without losing a word).

The same sense of organization has also affected the Bioinformatics Open Source Conference. Everything was up and running on schedule when I dragged my jet-lagged self into the main meeting room yesterday. I sat in on two talks from the biopathways section of the conference and was blown away: both by how well they went and by the subject matter. I’ve been away from molecular biology for too long. I’d never realized that networking principles could be applied to the analysis of genomes, or that a program (called Genies) exists that uses principles of information science and fractal analysis to extract data from scientific literature. From the Genies talk I also learned that cauliflower is a perfect example of a fractal. No matter how small a piece you break off a head of cauliflower, it will always look exactly like a complete head. What a great way to explore science and play with your food at the same time! I’ll definitely check this out the next time I stir-fry.

After dinner I sat in on a BOF (Birds of a Feather) session on open source authoring rights in the biological sciences. More and more researchers at universities and in industry are developing open source tools, or contributing to open source development, however, many of these efforts are being slowed down or hampered by intellectual property rules at host institutions. As it stands now, many researchers have no choice but to hand over not only the technologies they develop while students or employees of a university or corporation, but control over all future development of the tool or technology.

The scientists and developers attending this BOF are looking into new forms of open source licensing, and new ways of working with the IP officers at their institutions. Their goal is to have more freedom to develop their ideas and software tools, and do collaborative development with colleagues from other institutions, but still keep the technology licensing arms of their host institutions happy — all without having to sign over all their rights to ideas they develop.

			Developing Bioinformatics Computer Skills By Cynthia Gibas & Per Jambeck April 2001 1-56592-664-1, Order Number: 6641 446 pages, $34.95

Bioinformatics is the art and science of using computational tools to find answers to biological questions. It’s a hot topic these days, and the 1,300 or so people registered for the Bioinformatics Open Source Conference (BOSC) 2001 and Intelligent Systems for Molecular Biology (ISMB) 2001 are gearing up to look at bioinformatics, and the science behind it, from a number of different angles.

BOSC, a satellite meeting of the larger ISMB, explores open source bioinformatics projects under development, including BioPerl, BioJava, Biopython, BioCORBA, Bioxml, and BioRuby. New this year are “Lightning Talks”, which are turbo-summaries of projects, research, and interesting topics. I can’t wait to hear “Mining gene expression information using a controlled hierarchical vocabulary” summed up in five minutes.

This will be my first time attending ISMB. The program looks to be an interesting smorgasbord of the latest topics in Protein Structure and Modeling; Sequence Motifs (patterns in DNA); Networks and Modeling; Gene Structure, Regulation, and Modeling; Methods; and Sequence and Phylogeny (evolutionary development).

So stay tuned as I extend my knowledge of bioinformatics, and of course, sample the wienerbrød, which is Danish for . . . Danish.

Before I begin, I must make a clarification. This is an article about
why I am not a Java programmer, one person’s reasons and thoughts.
It’s not about why you are not a Java programmer and it’s not about
why Perl is better than Java. It is also not about why Java is the
way it is, I (mostly) understand Sun’s design decisions in doing
things the way they’ve done them. But knowing why something is the
way it is doesn’t make it any less irritating. [1]

Please keep this firmly in mind while reading.

Recently, I moved back to Pittsburgh after a stay in New York City for
a few years. [2] In NYC, Perl jobs are plentiful and Perl is fairly
well respected as a language to get things done. In Pittsburgh,
“Perl'’ is a four-letter word while “Java'’ is not. I’m reminded of
this all the time. Since I’ve been gone, most of my programming
friends have switched to Java, even the people who taught me Perl.
“When are you going to learn a real language?'’ I’m asked, and it’s
long past being funny. I’ve gotten tired of parrying the same old
critiques.

All this crystallized for me when a friend of mine, during yet another
Java vs Perl argument, asked me “What is Java missing?'’ This was not
really my thinking at the time, I’m not as concerned with what a
language has as to how it goes about it. Few modern languages are
“missing'’ any really critical features, it’s more a matter of how
easily they can be expressed. However, this does let my brain start
organizing my objections.

I’ll try to avoid incidental things. I won’t complain about the way
Java looks or refer to the mythical “typical Java programmer'’, just as
I’d expect not to hear about Perl’s “line noise'’ and stupid things your
CGI programmer friend did. I’m not going to get into closed source vs
free software, or which one is faster, or one supports some protocol
that the other doesn’t… all this could change tomorrow.

Instead, I’m going to stick to purely technical arguments about
design decisions in the language. I feel these are fundamental design
decisions which aren’t going to change anytime soon in Java, that
I can’t easily work around and would really get in my way. [3]

Without further ado, some reasons why I program in Perl instead of
Java.

“Hello World'’ should be one statement

Yes, I dredged up this old argument. Before your eyes glaze over,
hear me out. This isn’t the cheesy, old argument about which one is
shorter.

Here’s “Hello World'’ in Perl.

    print "Hello, worldn";

One line, one function, one string. What does it do? It prints
“Hello, World'’. Explaining it is almost as simple as writing it.
The trickiest part is possibly the newline.

And here in Java.

    public class HelloWorld {
        public static void main (String[] args) {
            System.out.println("Hello, world!");
        }
    }

This basic of all programs drags in the concepts of class, privacy,
types, methods, the magic “main'’ method, the String class, arrays,
class methods vs object methods and chained method calls. And on top
of all that it, it has to be saved in a file called “HelloWorld.java'’.
One must know all this (or cut & code it) just to get moving.

This makes it difficult to teach Java to a first-time programmer.
Just to get off the ground, you’ve got to get past the whole OO
philosophy. What’s a class? What’s a method? What’s privacy? Why
does it need to be called main? What’s String[] mean? What’s void?
Why is it System.out.println? All this incidental crap is exposed to
the student, someone who’s still struggling with the idea of a text
editor (“What do you mean I can’t program in Word??'’) You can, of
course, hand-wave all this, but that doesn’t settle well with me at
all. I prefer to use as little magic as possible when teaching.

I recently had the pleasure (?) of teaching Perl to a 14-year-old. I
started with “Hello, World'’ (’This is how you print something and run
a program’), moved onto conditional logic (’This is how you print
something if something else happens’) and then to loops (’This is how
you print something a bunch of times’), etc. Each lesson contained
only one or two new concepts. Each concept produced a concrete
result. Each new lesson built off the last. All with a minimum of
hand-waving.

Nit picking? Possibly, but it is a symptom of deeper problems. To do
anything in Java, even simple things, you’ve got to roll out all these
conceptual cannons. You can’t do one-liners in Java (assuming a
sufficiently short line). The upshot of that is if you know Java
you’ll have to learn another language for quick tasks. For some, this
isn’t a problem, but I’m lazy. I like having one language which
handles the vast majority of my daily tasks.

There’s more to life than OO

OO is currently vogue. Someday it will not be. Can Java adapt? I
know Perl can, it demonstrated its flexibility when it made the leap
from structured to object-oriented programming six years ago.

Because Java is selling you a philosophy of programming, anything
which does not fit into that philosophy rapidly becomes awkward. This
is typified by “Hello, World'’. It is not a data-centric task,
therefore the OO solution is clunky.

There’s nothing you can’t do with objects. There’s nothing you
can’t do with any Turing Complete language. But that’s not the
point. The point is how easily, elegant and maintainable can you do
it? When you try to shoehorn every programming task into one style,
things get ugly. I’d rather have a tool-box full of lots of different
tools than one with 57 kinds of hammers.

And what of other styles? Structured programming has not breathed its
last, and functional programming is just coming into its own, just to
mention two alternatives. I’ve recently started playing with
object-inheritance in Perl and find it useful. Can a strict,
class-based OO language be adapted to new styles or will they become
the COBOLers of the future? I know Perl can adapt.

No source for core classes and utilities

I wrote a quick Perl program to automate a set of remote benchmarks
(ironically, they were benchmarks for a Java program). Very quick:
log in, run a few commands, log out. Unfortunately, Net::Telnet had
some built-in assumptions about what a password prompt should look
like that this system didn’t follow. After peeking at the source I
found the problem, but there was no clean way to change the behavior,
I’d have to override the whole login method. So, I wrote a quick
subclass by copying the 160 line login routine from Net::Telnet and
modifying the password regex. Cut and code at its finest, but it got
the job done.

I caught some ribbing from my friend about this, “Such a powerful,
concise language Perl. It only took 160 lines!'’ However, could I
have done this in Java? No. Why? No source code for its core
classes! Faced with a similar situation (Java doesn’t prevent bad OO
design), a Java programmer would have either had to find an alternate
telnet class to use, or rewrite the entire login routine. Finding the
problem would have also been harder, lacking the ability to step
through the login() routine in a debugger to pinpoint the failure.

My solution was messy, but it was a messy problem. Perl kept a hard
thing possible.

No CPAN

CPAN is half the power of Perl (along with five or six other things).
Having a single repository of Free [4] high-quality, well-documented
and well-tested software (for the most part) with a common test and
installation suite. The ability to need code to do X, finding it in
one place, knowing it’s probably going to be something more than Joe
Hacker’s little one off (not to say CPAN doesn’t have its share of
those) and not having to worry about license fees, can slash
development times drastically.

Java, and most every language, lacks this vital, centralized resource.[5]
There will typically be several equally popular repositories, varying
in quality and openness (ie. Microsoft might provide a lot of Java
classes, but what’s the license like?) supplemented by smaller
repositories and individuals. What you need is out there, but how
hard is it to find it, know you’ve found it when you see it and use it
once you’ve got it? And how much will you have to pay for it?

No function pointers means no closures

Not having function pointers in an all-OO language isn’t in itself so
bad, since a similar effect can be had by passing in the name of a
method along with a class or object. The real problem is that without
function pointers you cannot have closures. No closures makes lazy
evaluation, and other parts of functional programming, difficult.

Some of what closures do can be simulated with inner and anonymous
classes, but their conciseness is lost.

A closure can be thought of as OO inside out. Instead of data with
code attached, it’s code with data attached. Is this OO? Sure! In
fact, several of Perl’s OO systems have been implemented using
closures [6]. It’s a different approach to the same problem, and it’s
good to have lots of tools in your tool-box.

No symbol table manipulation

99% of the time, mucking with the symbol table is wrong. And most
programmers will get by just fine without ever even being aware of its
existence. The Devil is in that last 1%. When, damn-it, you just
really need to generate a whole bunch of variables and to do it any
other way would require hours more work.

This is what I call a 1% feature. 99% of the time, it’s not needed by
99% of the people, but for that last 1% it can really make life a lot
easier.

Often, symbol table spelunking allows very surprising code libraries
with interesting and unexpected effects. Language designers can’t
anticipate everything, and it’s often good to be able to take a
language in a totally unexpected direction. Symbol table manipulation
is one that allows this sort of hackery.

Of course, this feature is very often abused, most of the time things
are much better accomplished by better data structures. However,
abuse by novices does not mean it should be denied to everyone. The
impact of such odd features can be lessened by properly structuring
documentation and books to deemphasize such 1% features. A Perl
tutorial shouldn’t even bring up symbol table hacking (alas, many do).

Is symbol table hacking nasty? Yes. Is it messy? Yes. Should it be
avoided when possible? Yes. But remember, any sufficiently
encapsulated hack is no longer a hack. [7]

Much more tragically for an OO language is that a closed symbol table
means…

No dynamic method generation

Every OO programmer reaches the point in their understanding when they
grok encapsulation. I mean truly realize the benefits of wrapping
everything up in methods. The class is thy sword, and the accessor
method is thy shield and micro-optimizations be damned!

Roughly five minutes later you never want to see another accessor
again.

Computers are basically well-trained monkeys, and your typical
accessor is monkey code. getName() returns a name. setName() sets
the name. getAddress() returns an address. setAddress() sets an
address. Repeat until carpal tunnel. Let the computer handle this
busy-work for you! Not only is it lazier, but it’s easier to maintain.
Basic laziness tells us that if you have twenty methods which all
basically do the same thing, you should consolidate them into one
method. A single point of change.

Perl has lots and lots of ways to do this. Class::Struct (distributed
with Perl) and Class::Class to name two. This is so important that I
wrote one myself, Class::Accessor. 60 lines of code, 5 methods.
Fairly simple both inside and out.

All rely on one or more of function references, eval and symbol table
hacking to generate methods on the fly. Alas, there’s no easy way to
do any of this in Java.

Ah ha! What about writing generic get() and set() methods?
obj.get('Name'); obj.set('Address', '32 Yarrow Way'); The problem
with this approach is three-fold.

First, inheritance is defeated. If I wish to alter the way
obj.getName() works, I can simply override it. To change how
obj.get('Name') behaves get() must be overridden and with that all
the parameter’s behaviors must be taken into account. Encapsulation
is blown.

Second, you tend to wind up with a big hairy case statement.
Polymorphism helps break it down somewhat, but you still wind up
handling multiple fields in a single method.

Third, the issue of how to get the types right. You wind up abusing
polymorphism, each group of types must be handled by a separate
polymorphic method, scattering the accessor code around and defeating
the purpose of writing “single'’ get() and set() methods.

What about using editor macros to generate the accessor code? Problem
there is the next person to maintain your code won’t have your editor
or your macros. Count on it. [8]

Preprocessors. C uses a preprocessor to get its work done, why not
Java? When C came about, preprocessors were a good idea. Then again,
so was paper tape readers and pet rocks. The problems with
preprocessors are so well documented that it’s probably redundant to go
over them here, but history is often forgotten. They make debugging a
pain in the ass [9], when your source code doesn’t match the running
code.

No eval

eval is another one of those 1% features. The ability to add new code
to a program while it’s running is a colossal amount of power. I
actually find eval to be a messy solution and usually find a better
way, but they usually involve other techniques Java can’t do (closures
and symbol table mucking).

Unfortunately, a free form eval in Java would violate all sorts of
compile-time checks, so it’s not going to happen. You can create
and load new classes on the fly (simply write a class file to disk and
call javac) but it’s not quite the same as being able to alter an
existing class to fit. And what are the odds the user of your program
has Java compiler installed? Pretty slim.

It’s odd that an interpreted language (yes, Java is interpreted. A
virtual machine is just a fancy name for an interpreter) would fail to
have some way of eval’ing new code. [10] With no eval, the whole idea
of dynamically-generated and self-altering code becomes very
difficult, if not impossible.

No multiple inheritance

A lot of people knock multiple inheritance, but there’s really nothing
wrong with it unless you’re the one who has to design a language
around it. Personally, I use MI extensively in my class hierarchies,
fully aware of what I’m getting myself into. If you don’t like MI,
don’t use it. But don’t tell me I can’t use it.

Java’s answer to MI is interfaces. Interfaces are not multiple
inheritance, it’s really just a very strict virtual class. They
address the first reason for inheritance, common interface, but
completely miss the second, code reuse. Interfaces force you to
reimplement the interface, for every class! What a bunch of busy
work!

Aggregation and delegation are other work arounds, but they both
involve writing wrapper methods. A situation further exasperated by
Java’s lack of dynamic method generation.

And, let’s not forget, multiple inheritance still allows single
inheritance if that’s the way you like to do things.

Interpolation is difficult

Ok, Perl uses lots of funny characters. $this, @that, %other_things.
People like to knock this bit of line noise. One thing it does make
very easy, however, is variable interpolation:

        print "I like $food, but only on $day.";

This is probably one of the most common constructs in Perl, and is
possible because of the funny characters in front of variables

Pulling off something similar in Java is not nearly as simple:

        StringBuffer buf = new StringBuffer();
        buf.append("I like ").append(food).append(", but only on ");
        buf.append(day);

Ick. Do I really have to go through all that work just to print a few
variables in a string?

There are a few implementations of printf() in Java [11], but
they’re almost as bad:

        Format.printf("I like %s, but only on %s.",
                      new Parameters(food).add(day));

There, wasn’t that easy? ;) [12]

No here-docs

Sometimes you need to have big blocks of text in your code. This
comes up very often with HTML, XML and SQL:

        $sql = <<'SQL';
        INSERT
        INTO    Stuff
                (This, That, Other_Thing)
        VALUES  (?,    ?,    ?          )
        SQL

It’s clear, it’s formatted, it’s unencumbered by quotes, appending
characters, etc. Means I can just paste text into my program.
Contrast this with how it’s done in Java: [13]

        "INSERTn" +
        "INTO   Stuffn" +
        "       (This, That, Other_Thing)n" +
        "VALUES (?,    ?,    ?          )n";

I think that speaks for itself. Wow. [14]

Bureaucratic privacy rules

I started programming by learning C++. I got up to the part about
public, private, protected, friend, etc… got very annoyed by the
unnecessary bureaucracy of it all and dropped the language, switched
to Perl. If I can’t trust the programmers around me not to muck
around in my guts without good reason, I can’t trust them at all. And
if they’re willing to perform that bad practice, they’ll probably do
more anyway. It’s not worth worrying about.

This is not to say there aren’t times I’d like to enforce privacy,
just don’t make me have to do it all the time. [15] To paraphrase Doug
Gwyn on Unix, Perl was not designed to stop you from doing stupid
things, because that would also stop you from doing clever things.

Mandatory strong types

Right up there with the privacy rules comes the compulsion to enforce
strict type checking. Don’t get me wrong, types are nice if you’re
into that sort of thing, but most times I find it just another set of
unnecessary hoops to jump through. Sometimes types are appropriate,
sometimes they’re not. Let me choose.

David Nicol, in a recent perl6-language@perl.org thread, put it
nicely, “The draw of the PMAW [Perl’s Magical Autoconverting
Wondervariable] is why we’re all here'’. We sling data all over the
place in Perl programs without hardly worrying, it’s wonderful! Many
people cringe at this sort of free-wheeling approach to data
integrity, but in my experience it’s not a problem.

People complain about the dizzying variety of interfaces to Perl
modules and functions, but we are saved from one thing: type casting.
You don’t have to worry about what special String subclass it
requires, or that instead of taking a normal list it requires you use
a specially crafted ParameterList object. No matter how weird the
interface, it ultimately boils down to scalars, arrays and hashes. [16]

If you like types, use them. I’ll even admit to wanting a strong
typing system in Perl. But don’t make me use them for everything.

Conclusion

Perl is a concise language, designed to make it quick and easy to turn
thought into code. Java is a syntactically simple and consistent
language, designed to encourage good style [17] and be easy to embed.
Each has their strengths and weaknesses, but people rarely agree on
which is which. People are funny that way. Some want to save you
from yourself by restricting you. Some want to let you be yourself by
removing as many restrictions as possible. Both are fraught with
peril. I happen to like the latter peril better.

Bjarne Stroustrup has this to say:

  "The connection between the language in which we think/program and the
   problems and solutions we can imagine is very close.  For this reason
   restricting language features with the intent of eliminating
   programmer errors is at best dangerous."

But I think Larry Wall sums it up best: [18]

  "The very fact that it's possible to write messy programs in Perl is
   also what makes it possible to write programs that are cleaner in Perl
   than they could ever be in a language that attempts to enforce
   cleanliness.  The potential for greater good goes right along with the
   potential for greater evil."

---

[1] I’ve found the first reaction most people have to this article is
immediately defending Java’s design. While it’s interesting to know
why, it doesn’t make it any easier to write code.

[2] Since originally writing this I’ve bounced back to New York.

[3] Along the same path, JWZ has written a
rant about Java picking apart annoying details that
could only come from someone intimately familiar with the language.

[4] That’s no-cost, open source, and unencumbered. Not this Community
License “we’ll grant you the privilege to peak at our code” nonsense.

[5] Java sort of gets around this by distributing heaps more classes in
its JDK than Perl does (maybe not for long at the rate Jarko is
going), but you can only push this so far. As Adam Turoff recently
lamented on advocacy@perl.org, the Java2 SDK tarball is pushing 20
megs making Perl 5.7.1 look positively anemic at 6.7.

[6] Class::Accessor and Class::Data::Inheritable to name just two.
Damian Conway goes into gut-blowing contortions about closures and OO
in his book “Object-Oriented Perl”.

[7] From the user’s perspective, anyway. But that’s what encapsulation
is all about, perspective.

[8] I’ve had more than a few “code building IDEs” suggested to solve
this, effectively glorified editor macros. One,
StructureBuilder, costs a mere
$1000 and works only on Windows.

[9] Trust me, I know. The perl source code is mostly macros.

[10] There are rumors of a Java library in the works which gives you
access to the compiler, but has all the problems of calling javac.

[11] Henrik Bengtsson’s is the one I’m looking at: http://www.braju.com/.
But printf() at its best isn’t nearly as elegant as variable
interpolation.

[12] Interestingly enough, Java’s inability to handle variable numbers
of arguments makes the obvious Format.printf("%s %s", foo, bar)
syntax impossible.

[13] Two individuals showed me examples of how to do multi-line strings
in Java. Both forgot the newlines. Perhaps anecdotal evidence that
this sort of unnecessary complexity leads to bugs?

[14] In all fairness, you can use preprocessors to get a here-doc-ish
behavior. One such widget is MLS.
But preprocessors, like filters in Perl, have their own set of
problems.

[15] Contrary to popular belief, you can enforce privacy in Perl.
Closures are a simply way to get private methods and several modules
provide privacy, Class::Contract in particular is air-tight!

[16] Yes, I’m oversimplifying.

[17] I’m being a gracious host here. ;)

What do you think? How would you compare Java and Perl’s different language designs?

Remember Parrot, the April Fool’s Joke, where I had Perl and Python
joining forces to create a new language? Well, there was a serious side
to that little prank. I recognised that the developer communities in
both languages came up against the same problems and discussed a lot of
the same issues; this would become especially important as people
started developing Perl 6 and Python 3000.

As an example, right now on python-dev, there’s a big thread about how
to handle Unicode characters; a couple of weeks ago on perl6-internals,
there was a big thread about how to handle Unicode characters. Every so
often on perl5-porters, there’s a big thread about how to handle Unicode
characters. Doubtless there’s similar threads on Tcl and Ruby
development lists. Maybe we should all talk about it together.

I also realised how much implementation experience there is in the
various lists. If your language is about to do something that’s
potentially non-portable, it might be useful to have Jarkko perk up and
say “You can’t make that assumption on the Cray” or whatever.

So I got talking with Nat Torkington
and Guido van Rossum about how we could encourage collaboration between
the developer communities.

As a result, I set up the language-dev mailing list to encourage collaboration between implementors of similar programming languages: we have representatives from Perl, Python, Ruby and Tcl subscribed.

Watch this space for news of how the collaboration pans out!