On Sunday I begin the California AIDSRide9, that amazing 575 mile bike ride from San Francisco to Los Angeles, to raise money to fight AIDS and to also build awareness that AIDS is still with us and is still very deadly. My challenges are - not necessarily in this order - to sit comfortably, to let my wife know how I’m doing, to keep an eye on the Open Source Convention (I’m the marketing manager for that gig), and uh, to manage that fantasy baseball team. But how to do it all, and do it easily?

One thought is that with my new Gateway 450XL laptop (they will truck our gear for us each day - you really think I’m gonna pack that thing on my back?), with built in wireless along for the ride, I thought I’d check out the wifi network/access points down the coast and through the central valley on the way to LA. I asked around for a bit of advice, but all the wifi experts I know were too busy to help the geek challenged, so I’m on my own.

I found this site listing street addresses of businesses with wireless connectivity, but I’m open to suggestions. If yer reading this and have any tips, send email to M2

Hopefully, I’ll be able to at least post a few blogs about the ride and what I find.

Related link: http://www.vibe.at/aktionen/200205/data_retention_30may2002.html

Related link: http://www.vibe.at/aktionen/200205/data_retention_30may2002.html

The European Parliament, over the protests of dozens of civil liberties organizations and thousands of petitioners, has voted for a proposal that would require ISPs and telephone companies to retain huge amounts of transitory and personal data obtained from users for long periods of time. Conceivably, the radical politician you criticized in email could come back after the next election–now the head of a distinctly anti-democratic regime–to arrest you.

Precisely the fears of such a Nazi-like resurgence originally led European countries from the 1970s onward to pass laws restricting the rights of governments and companies to obtain and keep data on individuals. This culminated in 1995 with a historic Data Protection Directive.

Under the cloak of the same anti-terrorist rhetoric that has led the United States and a dozen other countries to pass laws infringing on civil liberties–a trend documented in the
annual report
recently released by Amnesty International–the European Union is now poised to throw these protections overboard, and to track and record the behavior of residents throughout the continent. Retention periods vary from one to three or even up to seven years.

Even if you don’t live in Europe, of course, you will be caught in this barbed-wire net if you exchange any data with someone in one of the monitoring countries. Material retained could include the actual contents of email, lists of Web sites and chat rooms visited, locations of mobile phones used, and much more.

A large number of organizations (including one that I’m associated with, Computer Professionals for Social Responsibility) signed a
letter to the EP opposing their proposal and pointing out that it flagrantly ran against fundamental conventions, like the European Convention on Human Rights of 1950 and the historic Data Protection Directive mentioned earlier.

The European Parliament proposal is not law, although in theory each member country is required to pass a law conforming to the proposal. A lot could happen between the passage of the proposal (which was widely opposed by leftist parties in several countries) and actual legislation.

What will be the effects of the EP proposal?

What will be the effects of the EP proposal?

A friend I have known for some years recently described via e-mail his attempts to print hardcopies of an encrypted eBook that he had purchased. Disclaimer: I do not advocate or condone the actions described below. This information is provided for educational purposes only.

So, there exists a book that you want to read, and it is only available in eBook format, meaning you need to sit there and read the screen. Blech. Here is the step by step on how I sort of circumnavigated the copy protection of an eBook, and brought it back to the world of paper.

The book in question is David Foster Wallace’s “Up Simba - 7 Days on the Trail of an Anticandidate” on John McCain. BTW, I did actually buy this book, paying $4.95 through amazon.com for the privilege of downloading it.

Part 1: Try to Crack Protection…

1. Try to open the PDF in Acrobat in the vain hope that this would be all you have to do.
2. Open the PDF in Notepad, in a similar vain (sic).
3. Find out what type of copy protection would allow this book to be opened in eBook reader, but not Acrobat. Learn that an unlocking key exists on Adobe’s webserver. Groan.
4. Learn about Advanced eBook Processor, the fabled program that got Dmitri Sklyarov imprisoned, that cracks eBooks.
5. Immediately download this program.
6. Groan as you realize that Adobe has put in a built in foil for this program by this time. If only you had an
   earlier version of eBook Reader.
7. Try to find an earlier version of eBook Reader. Fail.
8. Download the full PDF specifications, available for free on Adobe’s website. Age 10 years looking at the first few
   pages. Give up.
9. Become extremely indignant, seek out a nearby soapbox, and rant about how Adobe is violating your rights for fair
   use of something that you own. Consider donating half your paycheck to the Electronic Frontier Foundation. Burn your bra.
10. Repeat steps 1-6 a few times, not because you think it will work, but because you have no other ideas.
11. Become frustrated and desperate, willing even to consider…

…Part 2: The Brute Force Approach

Ingredients:

• Adobe eBook Reader
• Encrypted eBook
• Adobe Acrobat 5.0 (full version)
• Quite Imposing Plus (an Acrobat plugin that will make you wonder how you ever used Acrobat without it)
• Adobe Photoshop 6.0
• Two hours of free time
• Patience

The method:

1. Load eBook and Photoshop. Change your monitor’s resolution to the absolute highest it will go (in my case, 1280×1024).
2. Set eBook reader to two page view, so two full pages display on screen at one time.
3. Take a deep breath.
4. Hit Print Screen. Switch to Photoshop. Create a new file and paste the clipboard data into it.
5. Repeat step 4, intermingling step 3 every once in a while, until you have copied all the pages to files in Photoshop.
6. Create a script that crops out all of the eBook stuff, saves as PDF and closes. Run script on all open files.
7. Manually concatenate a PDF to include all of the single pages into one PDF (two pages per page).
8. Get a drink.
9. More groaning as you realize that for half of the pages, you have the taskbar captured at the bottom.
10. Go to [company intranet site] and download License key for Quite Imposing.
11. Fix problem in step 9.
12. Crop the right half of the PDF. Save as “odd.pdf”. Crop the left half of the PDF. Save as “even.pdf”.
13. Use Quite Imposing to shuffle even/odd pages together and “create booklet” so that you can print the pages double
    sided and staple them together, like a book.
14. Save.
15. Go to bed, exhausted. Realize that although you can now print the book (at a resolution roughly equivalent to
    150dpi, not bad). Your file size has grown from 900kB to 28MB.
16. Smile!

On a technical note, I subsequently exchanged e-mail with the author of the above, asking “Have you considered using optical character recognition on the JPEGs you screencapped, to get the PDF back to its original size?

The author’s response:

Thought about it. Scrapped the idea for a few reasons. a) I don’t care
about file size. This wasn’t an elegant solution. b) I deal with OCR all
day, and in my experience, you end up needing to spell check and reformat
the entire document anyway, a bit too much brute force for even
me. Besides, with the roughly 150dpi pages, I’m sure its recognition would
be less than stellar. I may as well have just retyped it. I printed the
book out, and it was of sufficient high quality that from normal reading
distance, you don’t notice the lack of vectored text, aside form an odd
blurry quality. I got what I wanted out of it. I can (and just did) read
the book laying on my bed, or at the kitchen table, at my leisure. *shrug*

Frankly, this classic “worse-is-better” solution makes the whole copy protection thing look a little silly. Given how easy it was for this person to print an eBook that he’d purchased lawfully, do digital rights management “features” really and truly protect the rights of the copyright holder? Or do they merely serve as an barrier to fair use by the consumer? While we’re at it, since when did “fair use” suddenly exclude printing a document that you’ve acquired legitimately for your personal consumption?

Does printing an eBook for personal consumption constitute “fair use”? Or does it infringe on the rights of the copyright holder? Discuss.

Related link: http://www.microcontentnews.com/interviews/p2pj.htm

Before a whole slew of ‘blogging books hits the shelves, I’ve been thinking about the whole “phenomenon” from the perspective of an author who finished early. (”Early” being a relative term. Laura Lewin, our tireless editor, might have something different to say.)

The Slash book argues that the march of technology continues to lower the barriers for storytellers to reach wider audiences. Students of history know that these have always been considered disruptive technologies. I’m reminded of John Wycliffe and the Lollards and the attention they received for trying to *translate* a message.

Besides that, for the decade (plus some) I’ve been using the Internet and BBSs, there’s always been an anarchistic, rugged individualistic streak. Maybe that’s why I used to hear Wild West metaphors. It’s hard for me to imagine any new not-quite-completely-commercial development not described somewhere, somehow, in revolutionary terms.

It shouldn’t have surprised me to hear some people claim that Internet news sites would kill their stodgy real-world counterparts. After all, they have the advantages of speed, of no intrinsic editorial bias, and of rapid, uncontrolled distribution. Sometimes, it even works.

I do now believe, though, that there’s nothing intrinsic to weblogging technology that suddenly makes journalism possible. It’s still hard to research stories. It’s still difficult to write well. It’s not any easier to organize your thoughts into a coherent narrative. Let my little essay here be proof of that!

I still believe that weblogs are a good thing (however nebulous that moniker is). I’m convinced that making it easier for people to tell their stories is, on the whole, a wonderful accomplishment. I just doubt that there’s anything endemic to blogging that can regularly lead people beyond journals (not that there’s anything wrong with that), the occasional first-person primary source (there’s a lot right about that), and blepharitic punditry (there’s just a lot of that).

Maybe I’m backing away from my comments in the linked interview. Or maybe I’m becoming more convinced that with weblog popularity, it’s time to tackle more important (and possible) things, like better filtering. Weblogs have already solved the distribution problem (perhaps even a little too incestuously), but the sooner I stop thinking of them as tools to tear down ivory skyscrapers in New York and Los Angeles, maybe the sooner I’ll come up with something interesting to make the tools even more useful.

There’s still a lot wrong with journalism, but the technology of weblogs won’t fix it. Hopefully I’m the last one to have realized this. :)

So, where’s the real revolution? Are weblogs really going to bring down established power structures? Or was I the only person who ever remotely thought that was true?

Related link: http://www.washingtonpost.com/wp-dyn/articles/A60050-2002May22.html

The Washington Post claims that “Microsoft lobbies hard against free
software”–a well-timed revelation. One wonders whether the recent
Mitre report
mentioned in the article, which defended the use of free software, was
written in response to these pressures from Microsoft. When I read the
report I was wondering, “Who would suggest banning free software?” (Thanks to a reporter from C|Net for notifying me about the Post article.)

Microsoft (after the notorious “cancer” remarks reported in the
Washington Post article) has been trying to mend its reputation over
the last few months with the free software and Open Source
communities. Craig Mundie, one of the most aggressive of the anti-OS
wing of Microsoft, agreed to a
debate
at the 2001 O’Reilly Open Source conference.
Microsoft praised BSD as the right kind of license (largely as a snub
to Linux, one presumes, because Linux is a bigger threat) and
volunteered to port sections of .NET to BSD in its
Rotor
project. It is also cooperating with Miguel de Icaza
on Ximian’s
Mono
project, an open-source implementation of .NET.

So why go behind the backs of the free software community and engage
in the take-no-prisoners battle reported by the Post at the DoD? One
could simply call it hypocrisy and insincerity. But I tend to believe
that it reveals a split within Microsoft (several such splits have
been reported in the past, such as between the Windows 95 and the
Windows NT groups, and between pro-Internet and anti-Internet
factions). It’s no surprise that part of the company looks at nothing
but markets for domination and goes after them with well-known
Microsoft zeal. But the other wing had better get it in check; they’ve made fools of themselves plenty already.

Some of the statements in the Post article are a bit on the bizarre
side. Who says that “moneymaking applications should develop from
government-funded research”? Many of us believe the government has
gone much to far in letting companies and universities take over
intellectual property and tie up public research in closed products,
but I agree that it’s good in general for government research to spawn
business-making opportunities. Yet that applies to initiatives
coming out of government. (ARPANET, for instance.) There’s no
reason the government should take in proprietary products if
better ones are available for free. That’s plain old pork.

One can easily see why Mitre comes out in favor of free software. The
licensing costs for proprietary software at an institution as big as
the Department of Defense must be staggering. Support and trained
personnel for free software are sometimes cheaper. And then of course
there are the security issues that Microsoft laughably tries to
exploit. I just hope the lobbyists are wasting their time.

Does Microsoft have a point?

Related link: http://conferences.oreillynet.com/etcon2002/

I attended a press conference on the last day of the
O’Reilly Emerging Technology conference.
The journalists were enthralled by the prospect of ever-connected Web
Services zinging requests from one Internet-connected host to
another. I tried to warn them, as I described in
yesterday’s weblog,
that connectivity could not be taken for granted.

In contrast, the plug was recently pulled on law professor Lawrence
Lessig’s computer by anxious university staffers who detected a Morpheus
server running on it. He had just installed the server so that he
could offer some of his significant and highly desired legal papers to
supporters and researchers. Nothing could better illustrate the
alternatives facing us than the thrall of richly textured Web Services
and the pall of Lessig’s blank monitor.

The thrall

Each of the talks during the day came across to me as a taste of the
cornucopia the technical community is creating out of pervasive
computing.

• At the morning keynote, Adam Bosworth of BEA demo’d his company’s
  WebLogic Workshop, an integrated development environment that wraps
  Java code in all the rich packaging required for a Web service.

  Bosworth indicated that maintainable code should be coarse-grained
  (try to save network traffic by grabbing as much data as possible in a
  single call), asynchronous (allow results to return long after the
  request has been issued), and loosely coupled (so you can change one
  component without breaking others).

  He slipped in a dig at .NET, suggesting it promoted synchronous and
  tightly coupled applications. He declined to explain this claim during
  the following panel, but merely said it was the result of the initial
  implementation and that he has “no doubt that that will be resolved.”
  Dave Stutz of Microsoft confirmed the (still unexplained) accusation
  and said it was a temporary result of creating a product that they
  wanted to be easily demo’d.

• Rohit Khare extended the horizon of SOAP by promoting the notion of
  SOAP routing. When one service forwards data to another, it opens up
  possibilities of much more powerful applications. For instance, a
  publish/subscriber model could be implemented, where an intermediate
  SOAP router returns results not only to the original caller but to
  other interested parties. Recipients of information could also filter
  it and pass certain messages to subordinate servers for special
  processing.

• Dave Winer laid out a world of “250,000 weblogs” and asked how their
  interactions could change the way people form communities. He pointed
  out that a major recent release of a Macromedia product was promoted
  not just by the usual press tour, but by half a dozen weblogs
  published by internal developers. These continuously updated, in-depth
  discussions provided much more coverage than could ever be attained by
  neatly packaged marketing materials.

  Winer suggested that a weblog-heavy Internet would be not only more
  communicative, but nicer. Since a blog is closely associated with its
  poster and reflects back on him or her, posters tend not to “flame” or
  rudely attack targets.

• Ekaterina Chtcherbina reported the results of research she carried on
  at Siemens AB concerning the potential of peer-to-peer for producing
  more highly available wireless networks.

The formal presentations were not the only parts of the conference
that gave a feeling of abundance. Wireless access throughout the
meeting areas gave us first-hand experience of what we have to look
forward to. At first, I feared that the pervasive Internet presence
would cut off all face-to-face conviviality. I was afraid everybody
would be buried in their email all the time. But this proved to be
unfounded. Instead, the Internet presence promoted interaction; any
time we had an interesting document or online discussion to share we
could just flip open a laptop or PDA and gaze at it as we talked.

The pall

In the midst of this ringing paeon to the new Internet potential came
the dismal toll of Lawrence Lessig’s talk on “the future of ideas”
(also the title of his most recent book). Lessig and fellow panelists
conveyed how clueless policy makers are about the value of open
technology and innovation, and how completely they obey their funders
in the entertainment industry.

While Congress extends copyright in to perpetuity temporally, digital
rights management software backed by draconian law threatens to lock
it up technically. Lessig summed up the appropriate response to this
threat as “Write a letter, write a check.” Let Congress know there’s a
counter-movement out here. In addition, David Reed summarized the
strictures on wireless that I described in
yesterday’s weblog.

Exploring my interest in wireless policy, I attended Tim Pozar’s talk,
“FCC Rules and Regulations on 802.11.” Only about 30 people attended,
which I found surprising because Pozar laid out elegantly the various
important regulations concerning wattage, equipment certification,
interference, and exposure to radiation. The only thing I found
lacking in his talk was that it treated the FCC as God. This may be
the only feasible attitude to take when trying to set up a business in
the current climate, but I would like people also to recognize that
the rules are flawed and sometimes both arbitrary and unfair. While
working within the existing rules, we must also work to change them.

The Emerging Technology conference proved that technologically, our
future looks bright. But legally, the clouds have gathered and are
about to shut out the sun. Both the networks we build and the
political actions we undertake will have a radical impact on the
outcome. The choice is between a multi-layered, omnipresent,
high-speed network with agents and infinite customizability, and an
impoverished, crippled environment of captured devices and controlled
interactions. By the time of our next Emerging Technology conference,
I would hope that society chooses the former, but it appears the battle will go on much longer.

Coverage of the conference:

Lightweight speed and power at Emerging Tech conference
(May 13, 2002)

This morning at the Emerging Technology Conference, O’Reilly staffer and author Rob Flickenger explained why we need wireless community networks (the subject of his recent book) and how to make one that is flexible enough to provide great bandwidth to its owners while still offering public Internet access to anyone who happens by. I also saw a way to build Web applications that is just as lightweight and unimposing as Rob’s wireless network.

Emerging Technology Conference goes for reliable and secure
(May 14, 2002)

Many of my articles have enthusiastically promoted the hacker virtues of flexibility and features. But in the real world, reliability and security matter just as much.

O’Reilly Emerging Technology conference: breaking old regulations and old habits
(May 15, 2002)

David Reed: “Under the current regulatory regime, 802.11 would never have been legalized.” Also Schneier, blogging, and file-sharing at the Emerging Technology conference.

Robert Heinlein said it best over 60 years ago, in his first published short story, "Life-Line":

There has grown up in the minds of certain groups in this country the notion that because a man or corporation has made a profit out of the public for a number of years, the government and the courts are charged with the duty of guaranteeing such profit in the future, even in the face of changing circumstances and contrary to public interest. This strange doctrine is not supported by statute or common law. Neither individuals nor corporations have any right to come into court and ask that the clock of history be stopped, or turned back.

As David Henkel-Wallace told the afternoon plenary session at ETech today, "Copyright is dead. It may take ten or thirty years to die, but it’s dead." Would someone please notify the RIAA? The rest of us have work to do, and the "clock of history" is still ticking.

Related link: http://www.oreillynet.com/etcon

Here’s a quote uttered by Internet engineering expert David Reed today that
everyone concerned with emerging technology (and wireless Internet in
particular) should hear: “Under the current regulatory regime, 802.11
would never have been legalized.”

Scary observations like this led me to help organize a
Birds-of-a-Feather session on telecom policy tonight. In several
forums today at the
O’Reilly Emerging Technology conference,
we heard that emerging technologies depend on changing old
regulations, old rules, old habits.

Bruce Schneier lays out a feasible future

Renowned author and cryptography expert Bruce Schneier offered an
extremely lively and surprisingly fun keynote, updating his
Secrets and Lies book and suggesting some concrete steps
toward better security. As readers of Schneier’s book and
Cryptogram
columns know, he moved beyond technical fixes long ago and now looks
for social ways to improve Internet security.

Schneier started by saying that, despite all the technical advances in
computing, security is still a problem because of complexity. I would
rather say–and I believe this to be the true lesson of Schneier’s
talk–that security is still a problem because it is not purely
technical, but involves an ongoing battle between human beings. It is
a matter of sociology and psychology; technology is merely a
vehicle.

As Schneier said, “Security is a people problem, not a technical
problem.” Actually, the people he was referring to at that moment were
not the malicious crackers themselves, but the crowds of negligent
programmers, managers, data centers, and policy-makers who tolerate
weak security.

His proposals for improving the situation included:

• The oft-heard suggestion to make software manufacturers liable for
  defects. This was acknowledged to embody several problems, especially
  concerning small software developers, free software, and innovation in
  general. (Note: software companies are doing their damnedest to move
  in the opposite direction, by pushing
  UCITA
  laws that would let them get away with releasing known defects.)

• Making other companies liable for their own systems as well. They’d
  act differently if they knew they could be sued when their customers’
  social security numbers were released.

• Requiring insurance for data protection and integrity. Insurance
  companies, along with regulations, push companies to make all manner
  of socially beneficial expenditures, from smoke alarms to
  environmental clean-ups.

• Detection and response–rational prosecution and punishment.

“The limits of security,” said Schneier, “are the limits of the
Internet.” He compared the current situation to the rule of warlords,
and concluded, “We need to turn the Internet into a lawful society.” Schneier is a really neat person, despite his illusion that a Gore administration would have protected civil liberties more than the Bush administration does.

Immune to attack

The following talk, as a colleague told me, was an antidote to
Schneier’s pessimism. Steven Hofmeyr described an adaptive, largely
self-regulating system of intrusion detection inspired by the workings
of immune systems in biological organisms. Amazingly enough, in his
tests, it really works.

I have read of research into distributed network intrusion detection
systems, but what Hofmeyr proposed went several steps beyond what I’d
seen. One starts by creating random patterns. Those that match the
expected behavior of the system (as seen in logfiles, etc.) are
discarded; others are kept around for a while to see whether they
succeed in detecting anomalies. When one of them matches something new
on the network, a human administrator is notified. For new patterns,
therefore, some manual intervention is required to determine whether
the anomaly is OK.

But when a certain number of patterns are installed and have proven
their worth, they are remarkably good at detecting intrusions
quickly. The bigger the network you’re monitoring, the more useful and
effective they are. But as one audience member pointed out, the system
is meant for relatively stable and predictable networks with internal
traffic, not for open systems like public Web servers.

Both Schneier and Hofmeyr believe diversity is useful to minimize the
damage of attacks, but Hofmeyr has more faith that diversity is
achievable. As he pointed out, a single patch to an operating system
can change the attacks that work or fail.

Making policy through the back door

I started this weblog with the shocking observation by David Reed
that the wireless networks at the heart of this conference might have
been illegal in the United States, save for some historical luck. The
idea was meant to shake up the typical hacker out of his or her
apolitical bliss.

One wireless provider had enough insight yesterday to ask me for a
report on tonight’s policy BOF, but admitted, “I’d rather just build
networks without having to worry about policy.” And among the fifteen
people who showed up at the BOF, one queried me, “Why are you asking
the government for help?” These questions are reasonable but show the
crying need for education even among practitioners of wireless.
Imagine how much greater is the ignorance among the media, the general
public–and even the policy-makers themselves!

The fact is that wireless rests on very shaky legal ground. There
are no less than four types of devices that are licensed to operate
in the spectrum used by 802.11, and anyone operating one of these
devices has a right to shut down someone who runs an 802.11 network in
the same space. This is not widely known (the manufacturers of 802.11
devices certainly don’t want to talk about it).

Now what do you think? Which is more important to the economy and to
social progress in general: digital broadband or ham radio? The truth
is that ham radio trumps digital broadband, just because ham radio has
been around longer and therefore is sanctified with a license to use
the spectrum. (Yes, it’s happened–a ham radio operator has actually
shut down an 802.11 network.)

The solution is not to license 802.11 providers–that would just
hamper them with bureaucracy–but to find some new common
spectrum where unlicensed operators could put up their networks
without interference. (They could interfere with each other, but if we
get this far we can start to find technical and political solutions to
that problem.)

The FCC has expressed interest in packet radio over the years,
sponsors groups to find ways to make it work (some people who have
worked with these groups came to the meeting tonight), and has even
started proceedings to implement proposals–but both the search for
better spectrum and the potential for ultra-wideband (UWB) are
terminally stalled.

It’s become painfully obvious that, since 802.11 proponents lack a
major commercial presence with millions of dollars to throw around in
lobbying and contributions, neither Congress nor the FCC has
incentives to improve the environment for it. Indeed, the
disincentives are very strong. The lobbying sharks of traditional
telecom companies have smelled 802.11 blood and are beginning to
converge on it.

The apolitical radio operator who approached me today asked, “Can some
policy shift actually shut me down?” It is indeed possible. And even
though it probably won’t get that bad (the public and the media know
enough about wireless to provoke protest) the attention of Congress
and the FCC are focused on pulling the monopoly telephone and cable
companies out of the worst depression they’ve ever had. The last thing
these forces want is a cheap, user-controlled alternative to their
low-quality, overpriced services.

The hope forward may lie in doing what we’re already doing–building
the networks wherever we can–in combination with some creative
digital-divide initiatives. We discussed:

• Using Community Development Block Grant (CDBG) money to install
  wireless networks in disadvantaged neighborhoods and train residents
  to maintain them

• Promoting wireless as a way to save money right now–for instance, to
  replace the expensive police radio systems that suck up huge amounts
  of money in monthly fees

• State-level bills that would bring together municipal leaders so they
  can plan to build hybrid municipal networks of fiber and wireless, and
  that would fund needy communities with the goal of upgrading their
  telecom infrastructure

• Promoting the use of wireless in initiatives started by other federal
  agencies, such as for rural communications

Fiber definitely has a role to play too. Cities could provide neutral
access points where end-users could hook up with ISPs, bypassing the
sclerotic incumbent players. At first, fiber would be an institutional
solution, or a luxury item for the affluent. But a $3,000 dollar
investment for something that will provide shockingly high bandwidth
for 30 years is not an unreasonable investment for individuals.

Meanwhile, there are defensive measures we must take, too. Both
Congress and the FCC are poised to close the regulatory doors that
would let small, competing telephone companies coexist with the local
monopolies in telephone and cable service. (This battle may already be
lost, and in the opinion of some people at tonight’s meeting was not
worth fighting.) The notorious
CBDTPA–which
even Bruce Schneier singled out as a threat in this afternoon’s
keynote–would suppress innovative technology as well as the market
for broadband. Municipalities that try to create networks are
routinely sued by incumbent companies on a variety of pretexts. And a
bevy of regulations could nickel-and-dime wireless Internet providers.

That was about as far as the BOF got by ten o’clock, when people began to rustle around and let conversations drop. I believe they were unconsciusly reacting to my shutting down my laptop. But the only reason I shut down my laptop was that I saw a message telling me my battery was low. I guess that’s what it means to be a technology-driven policy group.

Don’t forget 3G

While analysts are increasing declaring “3G is dead,” it’s actually
been successful in some parts of the world. Nowhere is it more
successful than the Philippines, where wirelines are expensive and use
is metered. This was the subject of the talk by entrepreneur and
journalist Janette Toral.

While only 3 million people have access to fixed Internet access (and
most of these go to Internet cafes to use it) 11 million get the
Internet over cell phones. One of the most popular activities is SMS
messaging, for which a variety of fun and user-friendly applications
have sprung up. It has serious uses, too–farmers can access a service
call B2BPriceNow.com, for instance, to determine fair prices for
their commodities moment-by-moment.

Most U.S. observers are skeptical of SMS, but Toral thinks it could
become more popular here and around the world. She recommends:

• Instituting a caller-pays cell phone policy

• Offering SMS for free to start with, and educating the market as to
  its value

• Encouraging interconnection between providers

Who will blog the bloggers?

I unfortunately missed most of the morning keynote by Steven Johnson,
author of the popular book “Convergence,” because I was busy in a
conference call, saving the world. I made it to some of the following
panel, where weblogs were heavily discussed. Weblogs seemed to be a
theme running through the day. They played a big role in a talk about
new journalism by San Jose Mercury News author
Dan Gillmor,
Most of which I also had to miss. And there was a BOF on blogging in
the evening, but it adjourned to the bar and dissolved into chaos, appropriately enough.

The upshot is that a lot happened today concerning blogs, but I
happened to be away for most of it. Which does not prevent me from
gleefully abusing the medium right now.

File-sharing futures

Kelly Truelove offered a talk about trends in P2P file-sharing
systems. His focus was different from a talk given yesterday on the
same subject by Lucas Gonze. Whereas Gonze traced theoretical work
being done to create new distributed file systems, Truelove dealt with
practical extensions to the two protocols that are currently most
popular: Gnutella and FastTrack. (The latter is the basis of KaZaA and
used to be employed also by Morpheus, which switched to Gnutella this
past February.)

An ideal file-sharing system would be as fast as possible, use a
reputation system to ensure that good data is returned, and would be
massively scalable, anonymous, autonomous (that is, lack centralized
points that could be shut down, and sensitive to different pricing of
ISPs. Many of these goals are conflicting, of course.

In reality, systems that embody guarantees of robustness are also less
efficient that others and impose extra burdens on the user. So those
do not become as popular as systems that are more vulnerable. Even
though today’s systems lack the centralized indexes that Napster used,
there is still a vulnerable centralization in boostrapping (finding
other users to connect to).

Some ISPs forbid file-sharing systems under terms-of-service clauses
that rule out running a server. Few block traffic from file-sharing
systems unless forced to do so by copyright owners. But there is
growing impatience with the loads generated by such systems. There is
little appreciation for the notion that such systems can drive the
adoption of high-bandwidth networking.

After Truelove’s talk, I queried him about initiatives to provide
special classes of low-priority traffic on the Internet. Researchers
are suggesting that applications could voluntarily label themselves as
second-class citizens and slow down when normal applications are
active. If applications and network providers cooperate, a lot of the
pressure on campus networks and other points would be relieved.

What policies are needed for broadband?

Related link: http://conferences.oreillynet.com/etcon2002/

Many of my articles (including
yesterday’s weblog
on the
O’Reilly Emerging Technology Conference)
have enthusiastically promoted the hacker virtues of
flexibility and features. But in the real world,
reliability and security matter just as much. These
were the main themes explored today in the talks I attended at
the conference.

So robust, the system runs itself

The morning keynote on Autonomic Computing was given by leading
Internet researcher Robert Morris, who now works for IBM. Researchers
there have examined each aspect of system operation and tried to find
ways to make computers more like live organisms: “self-configuring,
self-protecting, self-healing, self-optimizing.” In other words, IBM
is trying to create systems so reliable they hardly ever need human
intervention.

Morris listed–as examples of current systems that do a pretty good
job of relieving the administrator of responsibility–RAID, virus
detection filters, and internal database query optimizers. But the
sterling example was the telephone system’s Electronic Switching
System, which works so well most users never experience a failure.

What did Morris suggest for future directions in autonomous computing?

• Expanded RAID disks that do mirroring instead of parity, perhaps with
  enough extra disk space that they never need to be administered even
  after failures take place.

• Database query optimizers that check expected results against actual
  results, effectively learning from their mistakes.

• Massive Web caching that evens out the loads experienced by different
  sites.

A lot of his solutions involved the old trick of “add another level of
indirection.” Thus, multiple operating systems could be run on a
robust base so that an operating-system crash is no worse than an
application crash. Clients could be managed over the network. Most
interestingly, we could replace stock recovery procedures (which are
error-prone) with a system that defines goals–that is, a vision of
what a healthy system should look like–and lets the system find the
procedures to return to health itself.

The question I did not get a chance to ask Morris concerned
security. Designing systems that fix themselves involves recognizing
repeated patterns and defining predictable solutions. What about human
intruders who figure out the patterns and exploit them? IBM’s
solutions may work for acts of God, but not acts of man–and
protection against these attacks take up a lot of system
administrators’ time.

As it happens, a while back I weblogged an IBM response to security called
“self-policing networks”.
It sounded a lot like what Morris was describing, and I was as
suspicious then as I am now.

Full tilt toward pervasive computing

The urge toward adding levels of indirection continued with Michel
Burger’s talk on how to achieve truly pervasive computing. In his
scenario, one could set up multiple sessions, pick them up later on a
different computer, represent oneself with a different identity in
each one, and generally escape what Burger called “the digital
ghetto.” He spoke only of Web sites, but I believe his model could be
extended to any protocol–which is good because I don’t think all
Internet data has to run over HTTP.

The system was extremely complex, involving regular servers (as we now
have), “user servers” that remember what you do across the other
servers (so you can compare all the books you buy, for instance, not
just the books you’ve bought at one site), and “context servers” that
provide your different identities. The full room of attendees seemed
somewhat stunned by the presentation, although a few people were
articulate enough to question aspects of the system.

The search for true security

Three more talks showed where the world is moving in search of
authentication and trust.

Three experts on reputation–Roger Dingledine, who wrote about it in
the O’Reilly
Peer-to-Peer
book, Jim McCoy of MojoNation fame (now with Hivecache), and Bryce
Wilcox-O’Hearn–delivered a mixture of lessons about both the need for
reputation and the difficulties of attaining it.

Basically, reputation lets users rate other users and then decide whom
to trust. The general impression left by the speakers was that it’s
very hard to get working. The best approach is to keep it really
simple: figure out exactly what you want to measure (such as uptime),
stick to as few variables as possible, seed the system with external
information of proven validity, and offer users an idiot-proof
interface.

If nothing else, the speakers were honest–they made it clear they
didn’t have general solutions to offer. While Slashdot and eBay are
famous for their reputation systems, Google may perhaps have the best
one in operation today, despite some known ways to game the system. In
theory, distributed systems are more flexible and robust than
centralized ones, but finding good examples is hard.

I mentioned in
yesterday’s weblog
that this talk would be of interest to builders of community wireless
networks, and indeed McCoy specifically referred to them in a slide
devoted to what he called “ad-hoc networks.” These present special
difficulties too, because when nodes can freely come and go they are
hard to track.

Another talk on security was given by Rima Patel; this concerned the
more conventional Security Assertions Markup Language (SAML). The goal
of SAML is to let users cross between Web sites with single sign-on.

Like many XML initiatives, this adheres to the common approach of
“let’s take known methods, express them in plain text, and wrap them
in angle brackets.” I do not wish to suggest that SAML is
unsophisticated, though. It seems rich enough to be valuable. It
offers, for example, ways to set conditions that apply to security
assertions, such as time limits or restrictions on who can ask for
them. Patel indicated that SAML is flexible enough to be the basis for
other systems, such as Microsoft’s Passport. But it could be run by
any company that wants to get into the authorization game.

Finally, I heard Richard Forno give a spirited critique of current
Public Key Infrastructure (PKI) systems. He is a traditional security
guy, careful in his investigations and brutal in his conclusions. I
have heard most of his points made elsewhere (such as by noted
security expert Bruce Schneier, who speaks tomorrow) but the rigor of
Forno’s thinking and the clarity of his presentation were impressive.

Would you buy a user interface from this man?

Reliability remained the theme of the afternoon keynote, if you define reliability as “doing what the user expects you to do.”

Perhaps the most audacious presentation of the day came from
Richard Rashid, who has quite the distinction of providing core
technologies to the two leading operating systems of today (he created
Mach, the kernel of Mac OS X, and worked extensively on Windows NT).
Despite my respect for Dr. Rashid’s work, I was disturbed by his talk
about the next generation of operating systems.

Rashid’s work at Microsoft is part of an experimental class of systems
known as adaptive interfaces. I like the idea of a system that can
query me intelligently (an example of which I’ll describe later in
this article), but I do not like one that presumes to know what I want
and decides whether or not it should interrupt me by watching my
gestures or checking whether I’m on the phone. The latter, however, is
the vision that Microsoft has.

The technologies Rashid showed were very impressive. He promoted “not
just document retrieval, but information retrieval,” and showed slides
of a technology called MindNet that could accept natural-language
queries and return excellent answers.

I accept the notion that the traditional building blocks of
computers–such as files and processes–are old-fashioned and do not
correspond to the way people think. I certainly would like a computer
that recognized my speech, gestures, and handwriting. But having a
microphone or camera monitor me all the time?

The more modest monitoring suggested by Rashid–such as checking a
user’s mailing habits and online calendar–seemed like reasonable
tools to “augment” (Rashid used an old term from Doug Englebart) the
user’s experience. But they also seemed like an excellent way to
further lock users in to using Microsoft tools. If you want your daily
habits taken into account, you’d better use the integrated system
they provide you for everything.

Furthermore, Rashid’s concept of “user-centered computing” sounds very
individualistic. He didn’t suggest any way to combine the knowledge of
colleagues and peers, which I find much more exciting than having a
computer that tells me which of my mail messages I’ll find important.

Other talks

It was a long day. Tim O’Reilly began it with a discussion of the
trends that came together to produce this conference.For instance, the Internet is assumed to be present, rather than being “an add-on to the PC.” He stressed that
players should not maneuver to control chokepoints and try to set the
rules, but should let everybody contribute–find value in the new community created by new possibilities. Also, new technologies take time
and should be allowed to develop organically.

Overall, he sees the Web as evolving to become a set of
components. For instance, he pointed out that
MapQuest
would be a wonderful service to offer for automated use, and predicted
that if AOL fails to develop it that way, another company will swoop
in and take their business away from them.

I also heard O’Reilly author Brian McConnell discuss his Worldwide
Lexicon, which he recently described in an
article
on our Web site. It is an interesting combination of human and machine
intelligence. It does not attempt to provide machine translation, but
simply accepts definitions of words and phrases from interested users.
It also represents the kind of adaptive interface I said I liked
earlier: it monitors keyboard and mouse behavior in order to figure
out whether it should bother the user by asking questions.

Stayed tuned for tomorrow’s events, particularly a Birds-of-a-Feather
session I’m holding on telecom policy.

Related link: http://conferences.oreilly.com/etcon/

This morning at the
Emerging Technology Conference,
O’Reilly staffer and author Rob Flickenger explained why
we need wireless community networks (the subject of
his recent
book)
and how to make one that is flexible enough to provide great bandwidth
to its owners while still offering public Internet access to anyone
who happens by. I also saw a way to build Web applications that is
just as lightweight and unimposing as Rob’s wireless network.
And I threw in a visit to
Yahoo!
for good measure.

An illustration of the strength of the Emerging Technology Conference

Combining one’s resources with neighbors to provide a local network is
much cheaper and more effective than making each individual buy his or
her own Airport and set up a personal network, particularly when
people share an apartment building and don’t want to trash each
other’s broadcasts. Rob’s
NoCatNet
exemplifies what one can do with cheap, widely accessible equipment.
Response to his talk was impressive; nearly fifty people filled the
room.

Rob laid out an expanding range of configurations:

• Simple “hot spots” (providing public
  access)

• Point-to-point connections (which let people reach friends on
  the other side of town by hopping from one access point to another)

• ParaNets, which involve a mesh of wireless access points and
  effectively provide a network parallel to the telephone wires.

The latter two configurations are particularly interesting (here I am
switching to my personal observations) because they represent a new
movement called “asset-based facilities.” It’s a reaction on the part
of the users to the sluggish and sometimes abusive expansion of the
traditional, large telecom players. Users empower themselves by buying
their own assets instead of leasing them; they can also create their
own point-to-point and mesh networks that permit lots of productive
data exchanges without mucking around on the Internet at all. Rob
pointed out that a key strength of ParaNets is that each access point
is owned by someone who has a vested interest in making it work well.

This movement also illustrates what I pointed out in
yesterday’s weblog
when I wrote that by one could understand each trend promoted by the
conference better by learning about and combining all of
them. Wireless or wired facilities that connect to each other in
hop-to-hop fashion are hardware versions of peer-to-peer networks such
as Gnutella. As such, their architects have a few things to learn from
the peer-to-peer movement.

For instance, users that depend on intermediate sites to carry their
traffic should find ways to recognize rogue sites that drop or corrupt
the traffic. Related reputation system concepts will be presented at
the conference tomorrow by an author who worked on the book

Peer-to-Peer.

Supported by several colleagues, Rob also presented an intriguing
wireless firewall system called NoCatAuth. It cleverly allows
different classes of users (the owners, collaborators of the owners,
and the public at large) get access to the network with predetermined
rights and guarantees of service.

NoCatAuth is basically an SSL server; one can either become one’s own
Certificate Authority and deal out access to known, preregistered
users, or offer service to the general public by getting a standard
certificate from a well-known CA. In either case, users can connect
with the assurance that names and addresses are not being spoofed;
users in turn can be kicked off the network if they abuse it with spam
or other unwanted activities.

To sum up, I find it amazing that wireless networks have succeeded at
all–not to mention having become the new, exciting grass-root
Internet movement. Wireless networks use junk spectrum (because it
does not require an FCC license), can be easily blocked by trees and
other barriers, suffer from security standards that are seriously
flawed (although Rob says they’ll be improved), and get bad press like
“Wireless users steal DSL access.”

But succeed they have. And this is because they are incredibly cheap,
incredibly fast, and almost totally unregulated. Some experts think
users can cooperatively work out continued growth without regulation,
but I think there’s a lot the FCC could do. Rob mentioned his hopes
that ultra-wideband (UWB) would be approved by the FCC in a year or
two. In fact, it occurred to me that the success of wireless may drive
renewed adoption of DSL and other wired technologies–anything that is
free from the crippling service-level agreements of cable modems.

Navigating Yahoo! nonvirtually

I had to miss my colleague Brian Jepson’s talk on .NET Web Services,
where he apparently spun an elegant and powerful programming interface
to a commodity trading system, because I was scheduled to have lunch
at Yahoo! with author Jeffrey Friedl of Mastering Regular Expressions
fame.

Yahoo! has put up a large and attractive modern facility right on the
bay. Their contribution to the environmental movement apparently
includes designing a parking lot that fits only compact cars. The
engineer who started the company still occupies a cubicle like any
other employee, although his is a bit larger than most and faces a
view of the mountains.

I was impressed with the company spirit that apparently holds strong
at one of the companies to maintain its vision and its cash flow (I
consider O’Reilly & Associates another one) through several
generations of fast-moving Internet time. To a large extent,
incidentally, Yahoo! runs on O’Reilly books. You can find quite a
library in nearly any cubicle.

Making Web Services natural

Leaving the site that best exemplifies the 1990s Web to come view the
emerging twenty-first century version, I got back to the conference in
time to see James Duncan Davidson (yet another O’Reilly author) build
an application on top of Brian Jepson’s raw classes. Duncan frankly
admitted a long-time suspicion of attempts to pump up the Web into a
complex application platform, but said “As time goes on, it’s apparent
that something really is there with Web Services.”

Still, for this presentation, he eschewed “huge frameworks or huge
toolboxes” like SOAP and WDSL. Instead, he built a Web service using
human-scale building blocks: XML to hold the data, JDOM to extract
this data easily into Java classes, Mac OS X’s Cocoa to build a
beautiful user interface. And (let us not forget) a simple HTTP GET
for the “heavy work.” Those who disapprove of using GET for this kind
of operation can use POST instead.

JDOM is a recognizably DOM-like interface that deals with the usual
hierarchy of XML tags and contents. Cocoa (based on NextSTEP) has
apparently created a programming system that lets average programmers
use the Model-View-Controller method robustly without having to be
software engineering whizzes.

During the discussion at the end of Duncan’s talk, one of his colleagues defended SOAP. But all participants agreed that proper Web Services were lightweight and simple. Grand schemes like transactions have no place in them.

What technologies will really make it big?

Related link: http://conferences.oreilly.com/etcon

I’m told that 500 people have signed up for this conference (the last registrant today was me, because I literally ran up to the staffer–whom I’ve known for a long time–as she was going to the elevator and begged her to return to the registration desk to sign me up). This number is an achievement, since it’s hard to put your finger on what’s going to happen at the conference. Essentially, this is several conferences in one, each conference representing a trend that is about to break out (we think) and become a major facet of people’s work and life.

But the Emerging Technology conference will (we hope) turn into much more than the sum of its subconferences. These trends all interact in subtle ways. By studying one trend, you understand the others better. That’s the potential strength of the conference.

After registering, I took advantage of another colleague’s last-minute database fixes to stay around a while, boot my Linux laptop, and try out my wireless Ethernet card. I wish I could awe my readers with agonized stories of my configuration struggles with this card, but the simple fact is that I stuck it in the slot, booted, and pinged away. My thanks go to Andreas Neuhaus and anyone else who worked on getting wireless cards operational on Linux. It’s a gratifying experience for me, because it shows me that Linux is staying current with critical modern technologies and is ready for the twenty-first century.

Today I was discussing the exciting potential of wireless for libraries and schools with colleague David Rothman, who promotes the visionary Teleread project. He contacted me, as did a couple librarians and media researchers, following my unkind review of computerized library catologs.

What David and I were discussing was more than an online catalog, but a whole online library. Imagine that instead of (or in addition to) large, clunky PCs in classrooms, students could be given handhelds or other small computers. And that anywhere they sat in school–library, classroom, locker room, cafeteria–they could quickly call up any of several thousand books in an online library.

No more waiting for a reserved book to be read by the other 25 classmates before you can get your hands on it. No more pockets of dead time where you’re waiting around with nothing useful to do. (On the other hand, I’d be reluctant to promote this wireless system if it detracted from kids’ informal face-to-face banter.) No more class projects where you have to leave the scene of hands-on work–like a garden or a lab–and return to the classroom to look up the background literature.

The technology for this library-everywhere-you-go system is all available. Books would have to be converted to XML, but several publishers (including O’Reilly) are doing that now. The server would be the responsibility of the school library. A Wi-Fi 802.11 network would extend the reach of the server throughout the school.

David also suggested that cities with Wi-Fi community networks could offer the school library to people in their homes. This would be true at least where there are neighborhood schools (and might give new reasons to maintain neighborhood schools). Parents could educate themselves as well as help their kids with homework.

The Wi-Li is a variation of Rothman’s Teleread idea, which calls for the development of rugged, low-cost computers for kids and the creation of libraries where books could be downloaded. While there are advantages of keeping a book on one’s personal computer (the original Teleread idea) it should be supplemented by vast virtual libraries.

The problem faced by Teleread and Wi-Li is the same chicken-and-egg problem faced by AOL Time Warner and all the other telecom players. Until the content is present, there’s little incentive to build the networks, and until the networks are built, there’s little reason to convert and upload the content. But vision and public pressure should be able to break the ice. The chance to make these a reality is coming closer and closer.

Can virtual libraries for children be created?

Related link: http://www.democraticmedia.org/news/washingtonwatch/markeyBill.html

New spectrum, and measures to address the digital divide.
I’m proud that my Representative seems to get it.

I love libraries and applaud every step they make into the digital age. But I wish they knew something about interface design and human-computer interaction.

I’ll start on a positive note: I returned recently to my alma mater (Brandeis University) and found that they had put in a comprehensive, Web-based, intuitive application that let me find the Dewey Decimal numbers I wanted in less than a minute. So it can be done. But usually my experience is quite different.

The first problem with computerized library catalogs (as with computers in the schools) is availability. Yesterday I visited the main branch of my home town library (Arlington, Mass.) and found three terminals, one of which was malfunctioning. That means that when two people are looking up something, everybody else has to wait. In the days of card catalogs, hundreds of visitors could search at once. This high-tech bottleneck, in computer terms, could be called the problem of coarse-grained locking.

The standard library catalog interface dates way back; I used to see it running on Vax/VMS systems. Everything about it is counter-intuitive, unforgiving, and obstructive. At some times you need to use the keyboard, at other times the keypad. The interface apparantly predated (or was designed by someone ignorant of) simple conventions such as “Space key moves forward” or “Return key selects current item.” Authors have to be looked up by last name only, and heaven help you if something you want begins with a letter late in the alphabet: there’s no way to sort entries differently or skip past the several dozen you don’t want.

I couldn’t imagine anybody designing anything worse than this venerable database application, but in my home town they managed to do so. The system has a spiffy look and mimics Web browsers, but only as if to mock them. Imagine a hierarchical interface that does not let the user backtrack! Having chosen one path, you cannot try a second–all you can do it choose from the arbitrary options presented later.

Since the books I wanted (I was researching Marshall McLuhan) were either not listed or missing from the shelves, I had to resort to a human interaction that did little better. I had the devil of a time explaining what I wanted to the reference librarian, who seemed to have no familiarity with these classic works. It occurred to me that schools of Library Science probably dislike McLuhan.

Let me end on a positive note once again. Once I get hooked into the interlibrary loan system, its operation is nearly miraculous. In my opinion, Amazon.com has nothing on my area’s Minuteman system, particularly when you consider that the library offers free shipping. And they can even notify me by email when the book has arrived.