July 2002 Archives

Michael Morris

AddThis Social Bookmark Button

Related link: http://www.usatoday.com/tech/news/techinnovations/2002-07-29-fingerprint_x.htm

In a scary story from USA Today, biometrics are here helping us become more efficient consumers. In the name of convenience, consumers are unwittingly helping to create the Brave New World.

The best line from the story:

“If we’re able to handle people speedier, then they’re likely to come to our restaurants,” says Lisa Howard of McDonald’s.

Just think what they’d do if they served decent food.

Michael Morris

AddThis Social Bookmark Button

Related link: http://www.bio-itworld.com/archive/071102/source.html

This article discusses the Mitre report in support of government use of open source and a lot from bioinformatics folk hero Lincoln Stein. The article basically covers (again) licenses and what should be done with the product of open source tools. I love this from Stein,

“Surprisingly, I find the single major advantage of open source is its stability,” he explains. “Not in the sense of ‘bug-free,’ but in the sense that I can rely on its being around for the long haul. I have been burned in the past by relying on proprietary software libraries and tools, only to see the company that provided [them] go under or be purchased by another company. Just ask any developer who relied on the Netscape Live Wire series of development tools.”

Andy Oram

AddThis Social Bookmark Button

Related link: http://icannwatch.org/article.php?sid=877

Justice could not be evaded; lame excuses, obfuscation, and even
oodles and oodles of legal maneuvering were not (at least in this case) enough. Karl Auerbach gets to do what every board member has
a right to do, and check the operation of the organization under his
charge.

Auerbach was elected to the board of ICANN by popular North American vote well over a year ago. He has been trying to see essential documents about ICANN’s financial and organizational status since that time, but officers kept putting barriers in his way and stretching out the responses to his request in the hope that he (along with all publicly elected members of the board) would eventually go away. Now we’ll get to see who will outlast the other.

Did the officers of ICANN have something really shameful to hide, or
were they just blocking Auerbach out of spite for his championing the
public interest? Soon we should find out. Auerbach is both a lawyer
and an engineer, and if anybody can figure out what ICANN is doing,
it’s he.

Schuyler Erle

AddThis Social Bookmark Button

Without fail, the Perl Lightning Talks at TPC never fail to amaze and amuse, and this year’s round were no exception. Conceived several years ago by Mark-Jason Dominus as a way of letting attendees air ideas and projects unworthy of a full-on conference session, the Lightning Talks are comprised of a rapid-fire series of presentations, each precisely five minutes, and not a second longer. They have in only a couple of years blossomed into one of the conference’s most perennially popular events. Hosted this year by perl.org’s Robert Spier, the talks covered a broad swath of topic including:

  • Dan Brian on the “New Opinion,” which may be summarized as “For every opinion, there is someone who holds it.” Using Google’s SOAP interface and his own Lingua::LinkParser, Dan located and counted the Internet’s most frequent references of the form “x sucks.” The winner? “Life sucks.”
  • Brian Ingerson’s attempts to get his shell to allow him to specify a non-binary executable on the hashbang line, in order to permit /usr/bin/ingy, the interpreter for the hypothetical Ingy language, to be implemented in Perl. Ingy gave example code, and noted that probably only Damian Conway himself could implement Ingy.
  • Joe McMahon talked about how Perl’s highly useful debugger is mired in its Perl 4 origins. He issued a call to action for developers to set about documenting and re-architecting the debugger for the future.
  • Casey West announced the existence of the Perl Documentation Project, and begged for contributions of docs and support.
  • Tim Bunce presented the latest improvements to the DBI, Perl’s database interface. If you use the DBI, you will love what Tim and friends have added.
  • Chip Salzenburg raised doubts about Larry Wall’s origins. How did a mild-mannered amateur programmer working for the government invent something with the awesome power of Perl? Is Larry like unto one of the archetypal wizards of Middle Earth, appearing mysteriously on the western shores to arm and fortify freedom-loving peoples against the forces of domination and control? Is he secretly leading a spiritual revolution of the faithful, a sort of Larry-Wall-Muad’Dib? Whatever his origins, he has certainly succeeded in teaching us at least one deep moral truth — There’s More Than One Way To Do It.
  • Allison Randal gave a reading of her Dr. Seuss inspired epic, “On Passing Perl V”.
  • After the break, Matt Sergeant gave a talk entitled “101 Damnations” on the how and why of the fifty-plus modules in his CPAN directory. Matt cited great tools, a great community, and a deep and abiding laziness as the primary inspirations for his CPAN contributions.
  • Casey West (again) gave a shout-out to some of the people whose hard work has made Perl great, complete with portraits built with the South Park Studio. “Oh my god! He killed Larry, Damian, Nat, Jarkko, Elaine, Schwern, and dha! You bastard!
  • Ken Williams announced the inception of the Perl Apprenticeship Site, intended to connect people who have cool ideas with people who want to learn how to work on them.
  • Tom Phoenix shared some of the Perl community’s most colorful euphemisms. Not all Perl hackers like to sniff someone else’s packets or put their socket in promiscuous mode, but doesn’t everyone always eagerly anticipate the upcoming major release?
  • Adam “Ziggy” Turoff shared the top ten reasons you need an iMac. Anti-aliased fonts in your terminal window, anyone?
  • Pierre Denis presented Vx, a slick SOAP-driven integrated multimedia exchange program.
  • Graham Barr introduced the shiny new search-beta.cpan.org, intended to replace the aging search.cpan.org site. It’s attractive, it’s functional, and it’s becoming the best way to browse the CPAN. Check it out.
  • Michael Schwern took us on a trip down memory lane, by compiling Perl 1.0.14 right before our very eyes, in under five minutes. Wow.
  • Damian Conway took the stage to demonstrate the brand-new /usr/bin/ingy by running Ingy’s sample code with it.

As cool as these presentations were, digital video stole the show at this year’s Lightning Talks.

  • Rael Dornfest presented a set of dead-on parodies of the recent Apple “Switch” commercials, featuring the likes of Chris DiBona, Dave Ascher, Ken Williams (in his cinematic debut), Nat Torkington, and Sarah Burcham, who switches to Windows XP because of “that button thingy”.
  • Infamous auteur Nat Torkington got in another screening of his screwball epic A Python Programmer Learns Perl From the Masters. (Warning: Not suitable for children or people who pride themselves on their good taste.) :-)
  • Leon Brocard introduced a fun promotional film about the Perl Foundation. Summary: Give TPF your money so that we can give you Perl 6.

By now, I’m guessing you’re probably sorry you missed the Perl Lightning Talks, if you indeed had the misfortune of missing them. Don’t worry, though, we’ll be seeing them - and hopefully you - at next summer’s Perl Conference 7.

Andy Oram

AddThis Social Bookmark Button

Related link: http://conferences.oreillynet.com/os2002/

We are constantly reminded how little intellectual or cultural capital
we’d have without a public domain or commons. We wouldn’t have the
technology that is currently most exciting to computer users, for
instance: Mac OS X. Its history (partly retold by Jason Evans today at
the

O’Reilly Open Source Convention

in his talk on

Porting Open Source Applications to Mac OS X
)
is nothing but a string of projects whose intellectual property was
taken over wholesale by other projects; the whole operating system is
as mongrelized as Yiddish or Spanglish. The Mach team at
Carnegie-Mellon adopted BSD as their interface. NextSTEP adopted Mach
and BSD. When NextSTEP was taken over by Apple, it went back once
again to newer versions of Mach and BSD to create Darwin. The system’s
toolchain, like the one on Linux, consists almost entirely of GNU
utilities such as gcc. I could go on and on.

Nowadays, the use of Open Source software is firmly on government
agendas in several continents. Incentives are financial, functional,
and strategic. Representatives of Germany and Peru
joined several U.S. experts in and outside of government
in the panel

Open Source in Government Panel
,

Here we learned that Open Source is widely used throughout the federal
government, including by the military in mission-critical projects
(where they really mean “mission-critical”). But the atmosphere
created by high-level officials, as well as by Microsoft’s pressure,
can be characterized as “Don’t ask, don’t tell.” Therefore, nobody can
tell how widespread it is.

Dr. Edgar David Villanueva Núñez, a congressman in Peru
who has introduced a bill mandating the use of Open Source software by
government, raised the financial issue. He pointed out that people in
Peru go to jail for illegally installing proprietary software–while
the prosecutors and judges who jail them are also illegally using
proprietary software.

Functional considerations are the same ones driving the adoption of
Open Source in industry. One of the great advantages here of Open
Source software is its interoperability. “The ability to collaborate
is critical,” said Mary Ann Fisher of IBM, especially between
government and universities.

Strategic issues include knowing how the software works and giving the
public confidence in government functions.

As Open Source enters the mainstream, in the session

Legal Issues for Open Source Projects

Lawrence Rosen showed how young and unformed the legal regime for Open
Source is. The GPL is quite sophisticated, but Rosen took exception
with clauses even in that. Many other licenses seem embarrassingly
insufficient from a lawyer’s point of view. Ironically, UCITA (which
would let commercial software developers wantonly abandon their
responsibilities to consumers) would place burdens on free software
that its developers could not afford by requiring implied warranties
of merchantability and fitness. (According to Rosen, Maryland put an
exemption for free software into its version of UCITA.)

When Richard Stallman, a conference keynoter whose speech I summarized
in
yesterday’s weblog,
chose as his life’s mission to make software free, many of us saw it
as rather quixotic and obsessive even if they sympathized with the
cause. And even if we recognized that the future would be based on
the miniaturized and the digital, we did not grasp how totally software
would become the metaphor and then the model for all things
intellectual and cultural. The more change is framed in terms of
intellectual property, the more RMS’s place in history is assured.

Copyrights are by no means the only battleground, or even the most
important one. Thus, the greatest health battle of the past two
hundred years, AIDS, and the success of drug companies in producing
treatments, becomes framed in social terms as a patent issue. So do
controversies over new crops. In the two keynotes this morning, by
biologists Ewan Birney and Jim Kent, questions of intellectual
property were posed over and over. For instance, the issue came up
several times of the difficulties in opening up research results, as
well as software developed during these projects, because of the
Bayh-Dole Act of 1980.

So there’s a lot of work left to do to put Open Source on a firm
footing: legal work to produce licenses good for the long term,
educational work work to promote its use in and outside government,
and political work to combat the laws and policies that could crush
it.

AddThis Social Bookmark Button

Related link: http://mt.sixapart.com/oscon/

Mena and Ben Trott have created a Trackback at OSCON site to aggregate OSCON 2002 related postings. Trackback is a peer-to-peer communications framework between weblogs/websites.

AddThis Social Bookmark Button

Whose Code is it Anyway? Game Show BOF

Host: Casey West

Comedy Troupe: Damian Conway, Allison Randal, Chip Salzenberg, Nathan Torkington

Casey West hosted the hilarious parody of the British improv comedy show “Whose Line is it Anyway.” It’s been revamped, regeeked, and recast with some of the sharpest wits in the Open Source Community. This brainchild originated from a dinner discussion during one of O’Reilly’s Perl Tour Sessions. Nathan Torkington, Damian Conway, Dave Adler, Mark-Jason Dominus, and Casey West improvised this idea over dinner while discussing some of the quirks of the p5p contigent of the community.

The format of the game show parallels that of the original:
the troupe plays games and acts out various scenes based on information that the host and/or audience make up. The audience contributed quotes, punchlines, and entire scene at the beginning of the session. What better way to see Damian as a track star doing laps around the conference hall in order to win a date with Allison Randal?

Games included 90 Second Alphabet - Each actor says one phrase on their turn, each subsequent phrase must begin with the next letter of the alphabet from the prior one. Actors must complete the 26 phrases within 90 seconds and bonus points for cohesiveness and humour. Other games included Questions, Weird Newscasters, Superheroes, Press Conference, Quick Change, The Dating Game, Worlds Worst, Whose Line is it Anyway, and Scenes from a Hat.

An audience favourite, talk is already looking forwards to next year’s follow-up. The word on the conference room floor is that an attendee audition will be held and the charismatic winner will be added to the mix of the comedy group. Want to make Nathan Torkington bark like a dog
in order to win the heart of Damian Conway? Be sure to attend this popular BOF next year. Or, better yet, searching for that perfect public unveiling of your late night hacking repartee? Start sharpening that wit for auditions at next year’s OSCON 2003!

Andy Oram

AddThis Social Bookmark Button

Related link: http://conferences.oreillynet.com/os2002

Free software (which is the term I tend to use in casual conversation)
is flexing its muscles and showing good health here at the

O’Reilly Open Source Convention
.
For the moment, no one seems concerned that the stock market was
tumbling for the past three days straight, that the way things are going we may all end up selling apples for a nickel on
street corners (and no, I don’t mean Apple computers), or even that two-foot-long squid have been reported in today’s USA Today to be washing up on San Diego’s beaches by the thousands.

The halls are
jammed with people, and the men’s room even more so. Predicted registration has risen to about 1475 people. All the key
projects in Open Source are developing in interesting directions and
showing strong community support. Presentations
today
on upcoming features of Linux 2.5, Perl 6, PHP, Python, MySQL,
PostgreSQL, and Zope showed an industry that is bold and able.

In
yesterday’s weblog
I said Perl 6 would lead to simplification; now I’m not so sure.
Certainly, the developers have done some clever rationalization (which
one can do when stepping back and seeing how usage has developed over
the years). But the bulk of the effort seems to have gone into adding
new features, without letting them get all tangled up like dogs
leashed together in the yard. Despite Larry Wall’s comparison of Perl
to natural languages, I don’t know if it ever was truly intuitive
(where “works like awk” is not the same thing as
“intuitive”).

The difference in connotation between “free” and “Open Source” came up
during the keynotes, as part of the discussion that has taken place at
every recent O’Reilly conference about the social and legal challenges
to innovation.

Lawrence Lessig sternly, and perhaps despondently, took the audience
to task for “doing nothing” while the studios and other big content
industries chipped away at our freedoms. This talk is one of the last
that he plans to give on this subject; he needs to return to his
regular legal work (which includes arguing the Eldred case) and his
ambitious attempt to define a new “creative commons.” (Unfortunately,
he had little to say about that initiative, which needs a lot of
definition.)

One of his contributions in this keynote was to point out that the
largest and most frequent uses of creative material always used to be
unregulated: reading, giving a book to a friend, and so on. Only in
the past decade (notably, since the Clinton Administration released a
white paper formalizing the notion that any access to material on a
computer was a “copy”) have the content producers tried to bring this
huge unregulated area under their control. “Nobody likes Washington,”
he said, “but if you don’t do something now, all the freedom you’ve
spent your lives coding will be taken away.”

The notion of freedom was shaped a more concretely in the keynote by
Richard M. Stallman that followed, which contained his traditional
history of the GNU project and ended with his endorsement of a
“two-step” education whereby people learn to appreciate free software
for practical reasons (which he characterized, with some accuracy, as the goal of the Open Source movement) and then
for moral reasons (the FSF’s goal). He asked people to help him drive
the second step, since most people don’t know about it and get stuck
before taking it.

Legal and corporate machinations have definitely emerged, I think, as
the threats to Open Source. In practical terms, it has already proven
itself. This was illustrated by the brief talk given by Tim O’Reilly
and Rob Glaser (chairman and CEO of RealNetworks) concerning Monday’s
announcement that RealNetworks would distribute its software through a
combination of an Open Source and Community Source license. Glaser
pointed out that Open Source gives the public confidence, because it
denotes quality and stability.

I don’t quite know why RealNetworks wants to follow the path emblazoned
by Netscape with Mozilla, but they definitely think they’ll get better
software by inviting in the talents of the programming community. And
today they started off the process by announcing work on support for
Ogg Vorbis.

Schuyler Erle

AddThis Social Bookmark Button

Damian Conway commenced his joint presentation with Larry Wall on the state of Perl 6 Wednesday morning at OSCon by giving due thanks to the hundreds of companies and individuals who have supported the design of Perl 6 through financial contributions to the Perl Foundation. Damian then pointedly noted that the title of their presentation was not in fact “Perl 6: Where Is It?” but rather “Perl 6: Where Is It At?”

Where it’s at is a place that is markedly different from Perl 5, but it still looks and feels like home. So far, Damian reported, Larry has already laid out the design for all kinds of goodies, including implicit subroutine parameters, higher order functions, a fully object-oriented exception mechanism, a switch statement, a hugely powerful new regexp engine, and a whole raft of new and improved operators. They are still working on, among other things, a new object-orientation model, multiple dispatch, method delgation, multi-dimensional data structs, and much more. “You will walk out of here more excited about Perl than you walked in,” Damian assured the crowd of perhaps a couple hundred hackers, “I want to show you some really cool stuff.”

And show us he did. I’ll spare you all the gory details, since you can read all the latest news in Damian’s Synopsis 5, and his earlier articles on perl.com and dev.perl.org. I will share one particularly juicy tidbit though: Variable aliasing. Suppose you have two arrays whose contents you want to swap. In Perl 5, you might try (@a, @b) = (@b, @a), but that won’t work, since the right side will flatten and you’ll end up with both lists slurped into @a, and nothing in @b. So you’re left having to introduce a temporary third list, which sucks. Not so in Perl 6 — you’ll be able to use :=, the new variable binding operator, which aliases the left side to the right, just like *foo = \$bar does in Perl 5. But in Perl 6, you can go one better and swap the contents of both lists by simply asserting (@a, @b) := (@b, @a), and it will just work. How cool is that?

Some of the new things are a little on the weird side. A lot of people seem dubious about the new use of the underscore as the concatenation operator, but Damian noted that “you don’t have to like it, you just have to accept it. We tried everything else.” I’m also a little leery of the added responsibilities laid on =~, the Perl 5 regexp binding operator. In Perl 6, it will become the catch-call “smart-match” binding operator, and automagically seek some kind of logical match between its right hand and left hand sides by analyzing their respective data types at runtime and doing whatever makes the most sense. Now, this sounds great in theory, but since Perl determines variable type at runtime, this seems like it has the possibility for all kinds of incomprehensible hell to break loose just when it’s least expected. Perhaps that’s the price to be paid for so much power at your fingertips, which, come to think of it, may not be so different from Perl 5, after all.

Some things really will change, though, like the regexp engine, which is up for a major overhaul. “Never mind that other languages followed Perl 5’s lead in regexp languages,” Damian observed, “They will follow again!” When all this will come to pass, precisely, is still unclear. Asked about the schedule for the remaining design of Perl 6, Larry inquired, “What is this ’schedule’ thing? We have a plan, but we don’t have a schedule.” Damian added that “it’s very important to us that we get it right, and not just ‘right now’. We’re planning a twenty-year lifetime for this thing, so another year or even two probably won’t seem like so much in the long run.”

Andy Oram

AddThis Social Bookmark Button

Related link: http://conferences.oreillynet.com/os2002

One of the most gratifying tidbits I heard today at the
O’Reilly Open Source Convention
was not in the many tutorials but over breakfast, when I asked several
Jabber
developers where the applications are that could catapult it
into popularity. (This was a follow-up to a conversation I had two days ago
and
weblogged
at that time.)

Jabber is a lightweight protocol based on XML that potentially lets
people exchange structured data and bring automated programs to bear
upon it as easily as they now do instant messaging. Jabber is open, simple,
and wildly extendible–all winning qualities–and it could be the
basis of new, collaborative activities of great benefit to businesses
and online communities alike. But no one tags data just for the hell
of it; Jabber requires a killer app to show its power. I haven’t seen
announcements of anything promising.

The Jabber developers explained that interesting applications were
being developed by business in-house for their own use rather than
publicly. Enterprises are the target of many companies currently
pushing proprietary software and protocols; perhaps the simple
but flexible Jabber protocol will ultimately prove to be the solution
of choice.

Simplicity was also the virtue claimed for Python by Keyton Weissinger
in his
Python and XML
tutorial.
It seemed to me that the interface for various XML activities was
pretty similar to the interfaces offered by other languages, although
I did notice some nice convenience functions. Weissinger’s cocky
demonstrations of interactive programming, which often required a
couple rounds of corrections, showed that the interface
was not as simple as all that.

It was only later that I learned about Python’s value from Simon
St. Laurent, a fellow editor who edits books on XML and the
Web. (Simon operates his own
weblog,
but gave me permission to quote him on this.) According to Simon,
Python’s object-oriented foundation fits neatly with XML, while its
loose typing makes programming less cumbersome than with a
strongly-typed language like Java.

I would think that one’s judgment of simple vs. complex depends less
on the programming interface that on one’s view of XML itself–if you
feel the specifications make sense, you probably feel comfortable with
programming it–and perhaps most of all with the fit between XML and your
particular data and application. Furthermore, if the text within a tag
requires a lot of fancy processing, your XML interface may be less
important than the text processing capabilities of your language.

I learned that structure can’t banish complexity in system
administration, either, when I stopped in on the session

OpenNMS: Managing your Networks, Systems, and Applications
by
Shane O’Donnell. This impressive system provides a lot of choices for
monitoring, logging, and notification. But throw in an ill-considered
configuration option and you could be up all night. OpenNMS could act
like the guards in Monty Python and the Holy Grail who
refused to generalize or apply common sense to their master’s
directions. Used carefully, it looked like a big time-saver.

Nobody called Perl simple, but Larry Wall did promise in his annual
State of the Onion address that it would get simpler in Perl 6. His
half-hour talk was admirably designed and delivered, considering that
it must have been prepared in the interval since the appearance of the

August 2002 issue of Scientific American

on which Larry based it.

The evening ended with Jon Orwant’s notorious Internet Quiz Show, which the defending champions won for the third year running. Perhaps the Internet Quiz Show is popular because the loony randomness of the questions reflects the day-to-day unpredictability of the typical programmer’s or sysadmin’s life. Who can tell what nonconformant server or unrevealed code bug will interrupt one’s work with a crisis at any moment?

AddThis Social Bookmark Button

Mark-Jason Dominus

Iterators and Generators in Perl Tutorial

World Premiere Presentation

I was there. The dawn of an era. Where it all began. Iterator value 0. Generator value ‘rand()’. The unveiling of Mark-Jason Dominus’s “Iterators and Generators in Perl” tutorial was up to MJD’s usual exacting standards, carrying the audience with both his dry humour and generally renowned technical aptitude. What has always amazed me about his presentation style is his ability to both move quickly through content and still present explanations carefully and methodically… in such a way that builds up to a crescendo that is both completely unsurprising, and totally cool.
This contrasts Damian Conway’s mindbending presentation style, which is to start slowly and then suddenly accelerate beyond the speed of light. They both understand the
rule of pedagogy: only present one new concept at a time.
Speaking of concepts, let’s move along.

So…what’s a generator?
A subroutine that returns different values each time is a generator (e.g. rand).

So… what’s an iterator?
An iterator is an object interface to a list. It keeps track of the ‘current position’ in the list. It’s a method for advancing to the next item in the list. Things like
filehandles, dirhandles, perl’s each hash function, the unix yes command. Mark-Jason takes especially twisted delight in using the yes command to iterate no responses with the “yes no command” format. The first half of the tutorial walks us through a case study building up to an iterator version of File::Find. This helps address the problem of a find call
taking a very very very very long time to finish. He shows us iterator implementations using perl object-oriented features, using closures, and a hybrid approach using both. Tricky things such as scope and duration with regard to closures (Why do the variables stay in memory?!), and anonymous subroutine pointers were addressed. MJD explains variable references and garbage collection! What a man. Here’s a look at a simple closure implementation of an iterator creator upto that takes a low integer value $lo, a high integer value $hi, and returns an iterator that will iterate through integer values from $lo to $hi.

sub upto {
  my ($lo, $hi) = @_;
  return sub {
    return if $lo > $hi;
    return $lo++;
  };
}

$iterator->() # NEXTVAL

General strategy: To convert a recursive function to an iterator, maintain a to-do list. What’s a to-do list? A list of things I know I have to look at, but I know I haven’t looked at them yet. MJD pauses and says, “Sometimes also called an agenda. (Latin for a to-do list).” The crowd goes wild with laughter. MJD pauses again, “I wonder how many people laughed because they knew I was telling the truth.” Oooh, roots humour… how base.

When power was accidentally cut by an audience member, Mark-Jason comes up fists: “A saboteur… Damian sent you didn’t he?” When the projection equipment recovers and begins displaying a countdown to power-up “30…
29… 28″, he quips “It’s just like New Year’s Eve.” Definitely in top form

Other in-depth topics covered include: iterators and exception handling (using eval to catch), depth-first vs. breadth first, Genome Sequence Generator example, Iterated database queries, imap (iterated map), igrep (iterated grep), a replacement for WWW:SimpleRobot.
Iterator termination and the semipredicate problem.

All these topics and more are covered extensively in his book Perl Advanced Techniques Handbook, to be published this year by Morgan Kauffman. Will also eventually be available for free at:

http://perl.plover.com/book/

Also, if you sign up now, you can get a draft of chapter 4 covering Iterators for 90 pages of iterators iterators iterators. Send email to: mjd-book-subscribe@plover.com

In the tutorial summary, mjd sums up his talk when someone attempts a question, “One second, I’m ranting.” I’d rather hear MJD rant than most other people talk.

Andy Oram

AddThis Social Bookmark Button

Related link: http://conferences.oreillynet.com/os2002

Databases are the big topic all over the
O’Reilly Open Source Convention.
It’s no surprise that the room would fill up (particularly because it
was on the small side) for the tutorial on
ACID Transactions in MySQL with InnoDB.
Interest runs high concerning the features being added gradually over
the past couple years by MySQL AB to bring MySQL in line with more
heavyweight databases, and transactions is what comes first to
mind. (Another biggie, foreign keys, is offered by the same InnoDB
type that supports transactions.) Furthermore, the talk was given by
some of the leading members of the team, David Axmark and Kaj
S. Ärno, with Monty Widenius chiming in with comments from
time to time.

The pattern was like old blues men performing. David or Kaj would
offer a standard explanation of what goes on, and the thin, wry voice
of Monty would then riff cynically from the rear. Nothing is as simple
as the textbooks would have you believe. Concerns for consistency in
the field of databases spawn concepts that range from the complicated
to the truly spooky, such as phantom reads. And even when you’ve
accounted for the four types of consistency (as David and Kaj did in
their slides and presentation) little anomalies intrude. These were
the contributions Monty made.

While I would like the interruptions and discussions to be planned a
little better, I really thought it was important to hear both sides of
the talk. We need to understand the concepts in a clean, structured
presentation as the folks at the front gave us. And Monty’s
off-the-cuff observations were marvelously educational too. They show
that relational databases are subtle treasures that require skillful
handling, and that there is no end to learning about them.

A similar pattern emerged across the hall where a key developer of
PostgreSQL, Bruce Momjian, offered his expertise on

PostgreSQL Performance Tuning
. Here again, the official
presentation gave way regularly to discussions among Momjian, team
members, and the audience regarding what worked, what was useless or
obsolete, and what had to be better integrated with operating systems
or gain a better interface. While these cadenzas and obbligatos could
have been considered confusing, they showed that databases are always
evolving. It’s a sign of strength that chief developers re-examine
basic issues again and again.

Relational databases are still the stars of the data storage
firmament. After them, the next most popular type of databases are the
even older DBM key/value type. These old stand-bys have been
challenged by companies pushing object-oriented databases, and
recently XML databases have emerged with claims on the future.

Why haven’t the challengers won out? Perhaps it’s because
object-oriented and XML solutions demand that the organization data be
limited for the convenience of the programming language. Organizations
that invest huge sums into collecting data want all the richness and
sophistication they can get in searching that data, and the best we
have at this point is still the relational database. So we’re going to
have to deal for some time to come with painful clashes between the
relational model and the model offered by whatever language we choose
to work in.

This theme also motivated a third talk I attended,
Integrating Databases with Zope.
While there was plenty of Zope in the talk, the underlying concept was
to maintain data in its relational format as much as possible, rather
than creating a competing form of storage, while still cataloging and
presenting the most important data in convenient user displays.

I also had a bit of fun trying out a decidedly old-fashioned
technology, sailing. Although I did little more than tie the fender, I
enjoyed the wonderful trip through San Diego bay and the sight of the
full moon at night over the lit-up skyscrapers of downtown. A little
more light would have been advisable for our boat, whose lights
malfunctioned. So we motored back as fast as could in the dark,
holding a tiny pocket flash out to avoid a collision from which no hash function could rescue us.

AddThis Social Bookmark Button

  • Time tested methods of bonding. No no, not superglue and duct tape. There’s a lot you can learn about people before they’ve had coffee and after they’ve had a drink. Email, irc,and electronic mediums are fabulous for keeping us connected and for making connections we otherwise might never make. But there’s so much about a person that doesn’t click until you’ve met them. This morning I met in the flesh two fellow O’Reilly bloggists:
    Peter Wiggin and chromatic, sporting an amazing
    code monkey tshirt. Which leads me to the next benefit…
  • You can more easily steal their stuff. Now, this is only partly about petty thievery. It’s also about a bigger, better type of theft - Open Source People. Where better to learn the tricks of how to be a wizard than from the wizards themselves. Mark-Jason Dominus had that idea in his Tricks of the Wizards presentation. There’s a lot that makes an alpha geek, mjd can give you the technical side in his talk, but talking to him over a drink can help reveal that je ne sai quoi.
    Buckle your seatbelt.
  • Eat more branez It’s easier to hunt when the prey is in the same building.
  • Andy Oram

    AddThis Social Bookmark Button

    Related link: http://news.com.com/2009-1033-945345.html

    WorldCom’s size double, quadrupled, continued to increase
    geometrically over the decade as it kept buying and buying. Even when
    it first began to talk of acquiring MCI, several observers suggested
    that it was stretching itself too thin. WorldCom wanted to buy Sprint,
    too; the whole telecom field is lucky the European Union put a stop to
    it. But in the dot-com era, this was considered just the accepted
    strategy of gaining market share by any means and at any cost.

    So finally it came down. The tragic flaw was in the original
    strategy. I imagine that the questionable accounting practices were
    just symptoms; they’re the kinds of desparate measures taken by
    managers trying to stretch out an unavertable decline. Very few
    managers, I believe, practice unethical accounting, trading, or
    reporting as a matter of course, and most of these no longer have to
    do so because they have cushy jobs in the Bush administration.

    Why do you think WorldCom went bankrupt?

    AddThis Social Bookmark Button

    Travelling itself could be likened to a video game. Moving along, against all odds: security checkpoints (”Remove your shoes and lift your feet, ma’am.”), lost luggage (”But that was my stuff.”), mangled hotel reservations (”We had you arriving tomorrow.”). Frustrations build and suddenly you’re playing Medal of Honor and you absolutely must find health and additional ammo…

    Once settled at the San Diego Sheraton Hotel, travel-weary attendees, speakers, and organizers found themselves amassing outside on the patio deck overlooking the ocean. The frustrations of travel fade. London.pm looked as if they’d become permanently glued to the patio furniture. The rumour was that they’d planted themselves there and hadn’t left since midday. When questioned on the validity of this, I was assured by Dave Cross that they actually had not been there all afternoon, merely “most of the afternoon.” New arrivals of the Who’s Who in open source and Perl communities started catching up with pint and rocks glasses of various beverages.

    Nathan Torkington flitted about, dealing variously with family, the conference, and documenting Things Which Should Not Be Filmed. Damian Conway and Larry Wall wandered onto the patio, as well as many of the regulars from the #perl irc channel, perlmonks, Dave Adler and brian d foy from The Perl Review and previously The Perl Mongers, Dick Hardt from ActiveState, Jarkko the pumpking, Elaine Ashton of search.cpan.org, various module creators, and book authors. Assuredly a motley crew of increasingly inebriated geeks. Almost exactly a month after YAPC, the gathering had a sense of dejavu.

    Perl.org Business cards (series of 8 — collect them all) made the rounds on the patio. Promoting the merits of Perl, the ones that can be documented here include:

    Perl

    … Because life’s too short to code without interpolation.

    … When the best is good enough.

    … It’s what the kid who replaces your department will code in.

    … Go ahead, use Java. That just leaves more punctuation for us real programmers.

    … It’s like Java, only it lets you deliver on time and under budget.

    Several drinks later, I realized I didn’t have what it takes to compete with the livers and stamina of london.pm. It had been a long day and past time to crash and get ready for the tutorials, presentations, and carousing that would fill the next week and constitute OSCON 2002.

    Schuyler Erle

    AddThis Social Bookmark Button

    Related link: http://www.salon.com/tech/feature/2002/06/11/stalker_tech/index.html?x

    UC San Diego has introduced a pilot program that lets students instantly locate each other anywhere on campus, by triangulating their PDAs via the campus Wi-Fi network. Creepy, cool, or just a sign of the times? (Thanks to coral for finding this article.)

    Schuyler Erle

    AddThis Social Bookmark Button

    Related link: http://www.uclick.com/client/nyt/db/2002/07/21/index.html

    In the latest installment of Garry Trudeau’s Doonesbury comic, a character gets busted for wardriving. Guess the meme has gone mainstream. While wardriving usually really is freeloading, the recent buzz obscures the more exciting developments — such as community networking — that might render the issue of parasitism irrelevant. And, what’s more, the comic totally ignores the key question: What are the telephone companies and record labels really selling us? (Thanks to Roger Weeks for finding this one.)

    Andy Oram

    AddThis Social Bookmark Button

    Related link: http://conferences.oreillynet.com/os2002

    The halls are still quiet as registration starts for the href="http://conferences.oreillynet.com/os2002/">O’Reilly Open Source Convention,
    but at every turn (and there are lots of turns in these halls!) I run
    into someone I know, and often a coworker I’m glad to meet.

    One of the people I got acquainted with has just been hired by a
    company doing Open Source development (so you see, at least one
    company is hiring), and decided to come to the convention to meet the
    leaders of his field. That suggests the convention is still a
    significant force. And the expected registration of 1300, which is
    pretty impressive in times when the stock market keeps slipping, also
    suggests this is a good place to be. (Some of this 1300 number represents press and other non-paying, but
    still valuable, attendees.)

    More pessimistically, I also ran into a developer I had met in the old
    P2P days, and we asked each other where were all the amazing new
    collaborative, community-building applications we had anticipated to
    arise in the movement. We concluded that most people were happy being
    passive consumers after all. Maybe the revolution will arise
    elsewhere.

    AddThis Social Bookmark Button

    Related link: http://slashdot.org/article.pl?sid=02/07/20/1342205&mode=nested&tid=106

    • Take a stand. Don’t back down. It doesn’t matter if you’re right or wrong. People respect confidence. If necessary, repeat yourself until your message becomes conventional wisdom.
    • Use the royal “we”. People like to be told what to do, and they will recognize your strong community ties.
    • Turn your experiences into absolutes. If you’ve seen something once, it must exist everywhere. If you’ve never seen it, it must not exist.
    • Argue that people will fail unless they do as you suggest. No one wants to fail, and everyone wants a strong leader. People are confused by choices, and will welcome your wisdom, confidence, and charisma.
    • Accept only complete agreement. Don’t get caught up in the minutiae of debate. You look weak if you have to change your mind.
    • Point out the stupidity of the commons. Only a few people are lucky enough to be so much more intelligent than the average person. You have to praise yourself before these folks will recognize your superiority.

    There are more rules, but if you stick to these, wise readers will recognize your rhetorical skills. Good luck.

    Have you found any other useful rhetorical devices?

    Michael Morris

    AddThis Social Bookmark Button

    It’s a sad day. Alan Lomax, folk musicologist best known for recording the history of folk music - including a legendary recording of Huddie Ledbetter - has died. The NYTimes has a nice story about Lomax (he punched out Bob Dylan’s manager when Dylan turned to rock for the Newport Folk Festival? Wow). The piece has a nice quote that reveals some of why this was a great man:

    He advocated what he called “cultural equity: the right of every culture to have equal time on the air and equal time in the classroom.”

    I’m surprised he wasn’t an Internet evangelist.

    We’ll have to dedicate something to his passing at the OSCON Music BOF.

    Andy Oram

    AddThis Social Bookmark Button

    Related link: http://www.wired.com/news/culture/0,1284,53704,00.html

    Gene’s chapter for our Peer-to-Peer book brimmed over with sensitivity toward people’s social behavior, insights into how technology reflected such behavior, and concern for supporting people the way they really are. He was well-matched with his fellow contributors to the Peer-to-Peer book, many of whom never met but all of whom combined technical insight with unhackneyed, free-thinking views about society and what was good about life.

    He told me once that the best time of his career lay before Sun, before InfraSearch, before Gnutella, when he was a documentation tools specialist. I realize now that his clinical depression simply prevented him from enjoying the excitement and significance of his peer-to-peer work. All the time he was inspiring thousands of people, he was taking nothing from it himself; very sad.

    Gene’s fame shows the balance that open source projects require. Gene was the point person to go to for news about Gnutella. But Gnutella itself, as a protocol with dozens of implementations, has no leader or even an official spokesperson. We need for people to grab an idea and go off in different directions. But we also need quasi-stable interfaces to the media and to companies that show interest in open source projects. Gene played this role for Gnutella, and I’m sure he didn’t ask for it. It came to him naturally because he was the right kind of person to handle it.

    Nowadays, we know that depression is an illness like any other, often with a chemical basis. This helps me accept Gene’s death. I don’t have to look for a flaw of Greek tragic proportions or ask where he went wrong. I know that the illness just caught up with him before his doctors figured out the appropriate treatment. I can be thankful to Gene for what he did for me, and continue to hold him up as a model for anyone who desires to bring technology into harmony with humanity.

    AddThis Social Bookmark Button

    Related link: http://wgz.org/chromatic/perl/brencrypt.pl.gz

    I wanted to pick a symbolic target. If I could use a symbol of US pride or history as a mechanism to bypass access controls, I’d put the DMCA enforcers in the unenviable position of claiming that the symbol was illegal.

    I settled on the text of the Bill of Rights. It’s plain text. It’s short, it’s easy to get into a canonical form, and it’s legal for me to distribute otherwise. It’s also the enumerator of the most important rights recognized by the US government.

    Using a symmetric cryptography algorithm (CipherSaber 1, a variant of RSA 4) and a program that writes programs, I’m able to encrypt any file, creating a self-decrypting file that requires a secret key. That key, of course, is the text of the Bill of Rights.

    If I were to encode an MP3 (to which I do not hold the copyright), a television episode, or even a movie, would it be legal for me to distribute the encoded version? It’s useless without the key. It could arguably be called a derivative version, but that assertion is unprovable without decryption. In fact, I dare you to tell the encrypted data from random noise. (Some might claim the resultant Perl program is random noise.)

    If the powers-that-be are willing to claim that such programs as DeCSS violate the DMCA (even though the stated purpose is to allow fair-use of DVDs and DVD players under free operating systems), why wouldn’t the Bill of Rights also qualify?

    To celebrate US Independence Day, you’ll need a recent version of Perl (5.005 or newer). To break the law and make the Constitution into an access control device, you’ll need to install my Crypt::CipherSaber module from the CPAN. You’ll also need to find the text of the Constitution somewhere. I’d link to it, but I think I’m prohibited from doing that now. :)

    Is this clever or stupid? Is there a better (read, more dangerous or more symbolic) way to demonstrate the absurdity of absurd laws?

    Andy Oram

    AddThis Social Bookmark Button

    Like an infomercial on cable TV, the promise of cable modem service
    strikes one at first as a wonderful package at an impressively low
    price. But the limitations of cable modems would render them
    unattractive if alternatives were available at a reasonable cost. And
    wait–there’s more. Cable modems keep us in the dark ages of Internet
    access, seriously distorting Internet usage, economics, and policy.

    Cable modems are a great advance over dial-up access, no one could
    deny that. People who experience the convenience of always-on
    connections and instantly downloaded Web pages declare, “I could never
    go back to 56K.” But while we can each individually enjoy the
    pleasures of this service, we should not be blind to the social havoc
    it wreaks.

    How cable modems distort Internet usage

    Customers, competitors, and industry analysts alike complain bitterly
    about the terms of service enforced by cable companies. Just last
    Friday, the Washington Post
    reported
    a complaint filed at the Federal Communications Commission by an
    industry consortium that included Microsoft, Dell, IBM, Sun, and
    others. I don’t have a handle on the ax this group is grinding, but
    their points were all valid and familiar: that users are not
    allowed to run servers, that artificial restrictions are placed on
    certain types of high-volume downloads, and that companies often
    intrude into user choices by ruling out home networks and even
    telecommuting.

    Some observers like to talk of conspiracies behind some of these
    impositions. They speculate, for instance, whether the cable
    companies restrict video downloads because these could someday compete
    with cable TV offerings. Well, that day is far off. While I don’t
    place cable companies above such ploys, I don’t think the danger is
    imminent enough to warrant dark machinations.

    Nor do I expect these companies to use Quality of Service features to
    slow down Web pages served by competing sites (another warning issued
    by some observers). I trust that the Internet service components of
    cable companies have the same attitude as Internet service providers
    everywhere: they simply want to get packets to and from their
    customers as efficiently as possible without hampering anybody’s
    enjoyment.

    Cable companies’ terms of service do distort usage. But ultimately
    the causes of the distortion are the asymmetric nature of the cable modem
    connection and the architecture of the network as a shared LAN. These
    are technical features, inherent to the cable system. The terms of
    service derive directly from such technical constraints. And we must
    remember this point later when discussing policy issues.

    There is one common area of complaint–reliability and service–that
    cable companies could ameliorate. But they won’t. These companies grew
    up offering entertainment, and they just aren’t mobilized for the
    serious business of handling people’s data. So when individuals
    justify a subscription to cable Internet service because they use it
    for key work functions, I wonder whether they also get their stock
    tips from the bartenders and their marriage counseling from their
    hairdressers. The reliability of a cable network just doesn’t justify
    this kind of trust.

    While I used the term Internet *usage* in the title to this section, I
    could have just as well specified Internet *expectations*. Cable
    modems promote the glitzy side of the Internet, because they allow
    graphics and animation to work so well. But the most important aspects
    of the future Internet, in my opinion, are precisely the ones
    prohibited by cable companies: the ability to run servers or
    peer-to-peer applications, and bidirectional, real-time audio and
    video. Thus, users think they’re getting something special while
    completely missing out on the real story.

    But instead of berating companies for things they can’t change, I
    would opt for truth in advertising. The Internet was originally a
    symmetric environment where everybody could run a server and, given
    the money, could increase bandwidth as much as they needed. So it’s
    just misleading for cable companies to advertise that they offer
    Internet service. They should advertise email and fast Web access,
    because that’s what they really offer. People who want to get on the
    Internet should understand that they have to look elsewhere. And that
    “elsewhere” is the subject of the next section.

    How cable modems distort Internet economics

    High bandwidth is a costly and complex market problem, and solutions
    will not come easy. But as long as people think cable modems are good
    enough for their needs, solutions will not come at all. That’s how
    cable modems, wonderful as they are for what they offer, distort the
    market for Internet access.

    The current competition to cable modems, of course, is ADSL. It has
    its own set of problems: it is no more symmetric than cable modems, it
    is unavailable to many people who live far from the central office or
    have old wiring, and the phone companies often mangle it during
    marketing, installation, or customer service because their staff is
    undertrained and stretched too thin. Yet ADSL fixes many of the
    problems of cable modems.

    Cable companies can offer access at just enough of a lower cost to
    hamper ADSL adoption (although it does rise from year to year). The
    more consistent bandwidth and greater privacy of ADSL may be worth the
    higher cost, but plenty of people can’t afford it or don’t understand
    the advantages.

    This issue of cost is controversial in itself. Public interest
    advocates are at least as suspicious of phone companies as they are of
    cable companies. It seemed in the late 1990s that incumbent phone
    companies were pricing their ADSL just low enough to undercut
    competitors who were leasing or buying lines from them.

    When a financial plague hit the competitive local phone market in 2000
    and 2001, killing off many promising small providers, and when the
    incumbent phone companies then suddenly hiked their ADSL prices,
    public interest groups called it an obvious case of anti-competitive
    behavior.

    This is quite possible, but there’s another way to look at the same
    history. Perhaps ADSL is naturally higher-priced than cable access.
    Perhaps competition in the 1990s forced the incumbent phone companies
    to price their service lower than their costs, and when the
    competition disappeared they could for the first time let prices
    reflect costs.

    In that scenario, high-speed Internet access ironically
    benefited–briefly–from a kind of subsidy reminiscent of
    the universal service subsidies built into the phone system through
    regulation and tradition. The difference between the subsidy of ADSL
    and the subsidies in the universal service fund was that low ADSL
    costs benefited the most privileged sector of the market: affluent
    consumers with plenty of disposable income.

    At any rate, ADSL now competes poorly with cable modems. We must
    remember that ADSL, like cable modems, was a hack to a system not
    designed for it. As a society, we must look for other solutions. Some
    combination of the following may finally provide the widespread
    high-bandwidth Internet we seek:

    • Consortia of neighbors who come together to string fiber from
      central offices to their local streets, and each of whom pays a couple
      thousand dollars for the last 100 feet to the home

    • Municipal networks that turn high bandwidth into a utility like
      water or electricity

    • Wireless local networks, run by volunteers or innovative companies

    But the urgency of these projects is undercut by the presence of ADSL
    and cable modems alike. A critical swath of potential pressure groups
    are just satisfied enough that they don’t raise the kind of ruckus
    necessary to push forward a radically better Internet.

    A few optimists will say, “Don’t expect the Internet develop in an
    ideal manner. Cable modems are low-cost, low-performing, and offered
    by companies outside the traditional Internet service providers–well,
    those are the hallmarks of a disruptive technology.” But that
    reasoning is flawed. Cable modems masquerade as a disruptive
    technology, but in certain key traits they fail the definition.

    For instance, new companies that successfully champion disruptive
    technologies are risk-takers who have burnt their bridges and put all
    their bets on the new technology. The cable companies have a
    diametrically opposed attitude, using Internet access as a loss leader
    to acquaint users with their more lucrative offerings. Furthermore,
    disruptive technologies contain the potential for upward mobility.
    But the limits of the lines used by cable companies are well-known and
    fully explored; in the pursuit of better variants they would have no
    head start over other companies.

    The question is whether the demand for servers and real-time
    high-speed media will grow to the point where it breaks through the
    inertia created by cable modems and ADSL. That I cannot predict. We
    shall have to wait to see whether a killer app emerges. (I’m waiting
    for something with a firmer business model and legal justification
    than file-sharing, which just fills a gap left by technologically
    backward music and movie studios.)

    But there’s a serious question as to whether people could exercise
    their market power even if they did want something better than they
    have now. And that’s because cable modems are leading policy-makers to
    enforce bad choices through laws and regulations.

    How cable modems distort Internet policy

    Competitors and critics of cable companies pursue an “open access”
    movement, whereby they want to force these companies to allow
    competing providers on their networks. Open access is fair in theory,
    but there’s not much it would actually offer consumers.

    Remember that the most serious drawbacks of cable Internet service are
    part of the architecture; no amount of competition can remove them.
    Another layer of drawbacks lies with the unreliable operation of the
    physical network, which would remain the province of the cable company
    even if competitors could offer their services on that network. So
    cable modems will always be what they are today, and people who ask
    for competition will be disappointed with what they get.

    The pressure of the open access movement has waned along with the
    existence of actual competitors who could take advantage of it, but
    the monopoly position of cable companies has created problems
    elsewhere. It creates the impression in the minds of the public and
    of technically unsophisticated policy makers that phone companies
    deserve the same privileges. This is the most serious distortion that
    cable modems introduce into society: the distortion of law and
    regulation.

    Amply funded Congressional friends of the local phone companies have
    introduced bill after bill, the best-known of which is
    Tauzin-Dingell,
    to remove various competitive requirements from the phone companies. A
    similar effort is underway at the FCC.

    The buzzword for the plans of these Congressmen and regulators is
    “intermodal competition,” which supposedly pits phone monopolies
    against cable monopolies. Essentially, the economic downturn that was
    so lethal to small telecom companies caused policy-makers to give up
    on the possibility of real competition by new, robust innovators.
    Instead, they hope that the behemoths of two industries (both
    financially struggling themselves) will somehow find their way toward
    innovation.

    The choice is profoundly tragic, for neither the cable industry nor the
    local phone industry is capable of the sorts of radical innovation I
    mentioned in the previous section. Both are largely satisfied with
    pursuing traditional markets, and neither would countenance a new type
    of service that could eat into those markets.

    Our only hope is to push for more openness and opportunity, not less.
    We need to make it easier to string fiber, easier to reach customers,
    and easier to interconnect with existing systems.

    If you use cable modems, don’t feel guilty. You have the right to
    enjoy it to the fullest. But look at what’s missing and figure out how
    much it would be worth it to you to have true high bandwidth. Then
    support efforts to create markets for promising alternatives.


    Does cable modem service satisfy the needs of your home or small office?

    Advertisement