Related link: http://fedora.redhat.com/projects/selinux/

SElinux is an impressively designed but notoriously hard-to-configure
set of kernel hooks that enforce Orange Book-style security on Linux.
Full support for SELinux takes effort, but when I first heard about
Fedora’s new targeted policies for SELinux, I was willing to
tell the Red Hat folks “thanks, but no thanks.” A conversation with
their Dan Walsh changed my mind.

The orginal SELinux approach was that anything not expressly permitted
was forbidden. Technically, this meant that every program anybody
would ever run had to be configured with a policy that
indicated what files it could touch, who could run it, and every other
aspect of the program that might present a risk. Practically, this
meant that you’d start your system and find that some obscure daemon
wasn’t running–and the only diagnostic aid you had was a few lines
listing process IDs and inodes. It didn’t help that all the resources
(files and so forth) had to be tagged accurately, along with programs
and users.

(This is the point where I feel it justified to mention that O’Reilly
has a
book
about basic SELinux use.)

Fedora users were getting frustrated and turning SELinux off, so Red
Hat figured they had to take a new tack before making SELinux the
default in Red Hat Enterprise Linux (which they did last week,
announcing RHEL 4 at LinuxWorld).

The concept of targeted policies is a compromise. Certain well-known
targets such as Apache get the full SELinux treatment. Other services
and programs are left with the old Unix security. Over time, more and
more programs will move into the targeted area.

It’s easy to see why I wasn’t impressed by targeted policies. The
program you don’t protect is precisely the one that intruders
attack. I figured that if you care enough about your house to station
someone with a shotgun at the front door, you should also have
somebody peering out the side window.

Dan deepened my thinking about this. We have to look at the solution
as an evolution. SELinux is still hard to configure, and while the
field learns more about how to develop high-level policy languages and
easy-to-use tools, we should take it slow. Some SELinux is better than
none.

Dan pointed out that firewalls went through growing pains too, and at
the beginning many people just turned them off because they were too
restrictive. With SELinux as with firewalls, providers have to refine
the concept over time.

The older SELinux approach was subtractive: install it, see what
breaks over time, and fix the bugs one by one. Targeted policies are
additive: build up security as you go along.

SELinux is based on well-established principles, and it ought to be a
step forward. But to get there, a lot of us have to be patient and try
it out, to give researchers a base for further improvements.

Will you use SELinux?

I’ve said it before:
Content-based spam filtering is a dead-end path. Here’s one big example from my mail this morning:

.,        ,; .R,
@FS      fUD jos
 DN      Gw,
 Fzw    OUn  hdx  DLdknFf:   qgOKPugU  aYkIda  @ygoaQr
  Dj    hN   Sam  xb    tJ. mBT.  fSV  zek    Nw;   @Hf
  dxd  Stk   ALQ    TZFwKw: qR     ol  HJb      EmpiiA@
   sb .Vz    XWw  chY:: Aw, ju     iA  GFk    aHs,c woi
   FsrQua    Gcc pW     kA  IBy   HFd  ZVx   Gsx    SME
    ziyA     riA  UNvhcHbgj  NZaBdunU  TYA    NsaQfMzrRB
              ,    ,:;U   :        Ae   ,       ,;w   .:
                            lze   yrP
                              IegDp.

open( my $fh, '/usr/share/dict/propernames' ) or die $!;
while (<$fh>) {
    chomp;
    push( @{$words{length($_)}}, $_ );
}

while (<DATA>) {
    s/(S+)/replace($1)/ge;
    print;
}

sub replace {
    my $list = $words{length $_[0]} or return $_[0];
    return $list->[rand @$list];
}

__DATA__
.,        ,; .r,
@ln      qly tlg
 nq      aq,
 Brg    iaB  WiW  iqpbduk:   ifcciWvj  Wypdip  @rnoqqS
  lc    st   unx  mm    su. Wyl.  eee  daa    jb;   @kS
  kjt  smp   WkW    8hytct: ih     xd  WiZ      Zlantc@
   tg .vk    WrW  cyW:: hy, vx     bo  WnW    gtx,i 0rW
   SnjsaS    WbW gw     oo  kkZ   rto  WeW   fvB    0qZ
    xbcd     ocg  tfrotxynk  veqWhurb  kdy    wavkuseax0
              ,    ,:;i   :        yr   ,       ,;i   .:
                            Zjc   ugr
                              btfau.

Ed        Al Roy
Amy      Tom Jim
 Ji      Len
 Spy    Lin  Alf  Roderick   Srinivas  Rajeev  Juliane
  Hy    Ti   Tao  Ed    Amy Renu  Fay  Bud    Tom   Jef
  Tim  Kyu   Mat    Nicolas No     No  Hsi      Shannon
   Al Ami    Tai  Judge Hal Al     Hy  Fay    Piete Hsi
   Gregge    Suu Al     Al  Ken   Art  Moe   Lar    Mwa
    Vern     Vic  Stephanie  Teruyuki  Rod    Cristopher
              ,    King   :        Ji   ,       Les   Hy
                            Bob   Dan
                              Dannie

Content-based filters are rags stuffed in the hole of a leaky boat. Water’s still getting in, and they’re not going to hold forever.

What’s the best solution you see that isn’t based on content analysis?

Related link: http://use.perl.org/~jjohn/journal/20761

My old colleague Joe Johnston reflects on why mod_perl hasn’t taken over the world like some of us once expected: “That’s right, stupid CGI + HTML is a kind of universal Microsoft Fundation Class that works for programmers of all lanuages.”

Related link: http://www.oreilly.com/catalog/revolution/index.html

I’ve had Andy Hertzfeld’s Revolution in the Valley sitting around the house for a couple of weeks. One of the O’Reilly people slipped it in with some other review copies, and I was excited to get it even though I didn’t have time to look at it right away.

The book is beautiful. It could be in sanskrit and still be beautiful: it’s a dense hardcover, that sort that you could really use to hurt someone, and its first and last several pages are copies from design notebooks from very, very early Mac prototypes: the time before even the Finder existed. The middle is chock-a-block full of photos: Wozniak’s prototyped circuit boards, office pics, goofy photos of ultra-geek Bill Gates. Everything looks nice, and just the right amount of nice: not too designy like Wired.

The book puts me to sleep though, which might be my fault for reading it before I go to sleep. I’m not a big enough Mac nut to care about the personalities behind it (just like I don’t read movie reviews or actor gossip). For me, it’s like watching other people’s vacation movies or office stories. I’m not in on the joke, or I don’t value the same things.

Andy’s stories, which you can also find on Folklore.org (the proto-version of the book), are short and targeted, and although they are in chronological order, they don’t really seem to connect with each other. To me they seem like little islands in an unmapped sea. If you know the stories already, and how everything fits in, you probably will like the stories more than I do, but Andy didn’t give me much reason to care about these people more than I did already.

I’m not telling you not to buy this book, but you can decide how full of crap you think I am and go from there. It’s a great book if you like the subject matter: I’m just not that into it.

One of my friends got unsolicited email newsletter from his congressperson (he didn’t specify which one), and he doesn’t like it. It’s spam, he says. It is opt-out, but unsolicited nonetheless.

I think that’s missing the point. Although spam is usually defined simply as “unsolicited bulk email”, I tend to think of that in terms of people casting a wide net in hopes of catching a few percent of the recipients. I’d love to get an email newsletter from my representatives.

Still, my friend says it’s all still about sales, and the politician is trying to sell himself. Sure, that’s true, but we’re all selling ourselves every day.

I checked the web sites for my senators and discovered they have opt-in lists, but I wouldn’t have know about them unless I really looked for them. I would have liked a single email saying “You can keep tabs on me by …” or something similar.

I don’t think we should use the spam outrage hysteria to keep our elected officials from reaching us.

What do you think?

Recently, one of the co-located webservers that I help administer developed some fairly typical colo server problems, and it was agreed that the time had come to upgrade from Fedora Core 2 to Fedora Core 3. I volunteered to perform the upgrade remotely - the server is in California and I am in England - using Yum, the package manager that ships with Fedora Core.

I spent some time screwing around trying to find a decent yum.conf for Fedora Core 3 on the Internet, and finally found the Fedora.us wiki’s FedoraSources page. I installed it in /etc, made sure that it pointed to the FC3 repositories, and ran yum upgrade. So far, so good.

Hours later, Yum had downloaded all the packages it thought it needed, and got about a third of the way through installing them, when it hung, with the Python process taking up over 200 megs of RAM, doing absolutely nothing. Control-C didn’t work, and neither did a simple kill - I had to kill -9 the process ID. Then I ran yum upgrade again, hoping it would pick up where it left off.

No such luck. Yum instead started complaining of broken dependencies, and refused to install anything. Well, of course its dependencies were broken - the system was a third of the way between Fedora Core 2 and 3! Not only that, but there were apparently duplicate packages of various important things like glibc installed, each with different version numbers. I was 6,000 miles from a webserver with a now-broken packaging system. I started to sweat a little.

In desperation, I turned to APT to clean up Yum’s mess. Thankfully yum install apt still worked - I think I had to make sure that Fedora Extras were enabled in the yum.conf - and, although the FedoraSources page claims that there are no APT sources for Fedora Core 3, in reality, you can change the revision number in the FC2 sources.list from 2 to 3, and it apparently does find sources for Fedora Core 3 on apt-get update.

APT whined about the tons of duplicate packages, but did clean up the mess when I ran apt-get -f install. What’s more, a subsequent apt-get dist-upgrade actually did finish the upgrade to Fedora Core 3. To be fair, APT didn’t know what to do about the duplicate packages and I had to remove them with this hack:

rpm -qa --qf "%{NAME}t%{NAME}-%{VERSION}-%{RELEASE}n" 
    | sort -r 
    | perl -lne '($pkg, $ver) = split; print $ver if $pkg eq $oldpkg; $oldpkg = $pkg' 
    | sudo xargs -n1 rpm -e

That left a few duplicate devel packages the first time, so I ran it again. The upshot is that the machine rebooted with a shiny new Fedora Core 3 install! (Subsequent note: Okay, I did also have to move /etc/postfix/aliases.rpmsave back to /etc/posfix/aliases to get the MTA working again… But that’s not the packaging system’s fault, per se!)

Last week, I was upbraided by a Fedora developer for running an APT archive of GIS RPMs for Fedora Core. Given that Yum is the packaging system shipped with Fedora Core, he maintained, I should start using it and expect APT to fall by the wayside. He pointed to the halting official support for Fedora Core 3 and the lack of Fedora Extras for FC3 available from APT (which I believe is not actually so). Given my own experience - and we’ll leave aside the ways in which APT provides a superior user interface (e.g. countdown timers for downloads) - I think my next move will be to apt-get remove yum from that webserver.

Was I a fool? Was I right? Did I misuse Yum somehow? Have you had similar problems or no problems at all using either Yum or Apt on Fedora Core?

IBM’s new PHP efforts are described in this press release and this news article. They include:

- A free-download package combing PHP and the Cloudscape DB
- IBM developer resources focused on Web Services in PHP

I suppose this makes (or will make soon) IBM a commercial resource for PHP support.

I’ve felt for a while that PHP is about five years behind Linux on the commercial acceptance/support arc. Maybe that number isn’t exactly on point, but the general trends have been pretty consistent. I don’t remember exactly when the building at 42nd & 8th (across from Port Authority) in Manhattan was splashed with a giant penguin on it as part of IBM’s “we do Linux” ad campaign a while ago, but perhaps a big Elephant (or whatever PHP’s logo is nowadays) isn’t far behind.

Is this good for PHP? Bad for PHP? What do you think?

Related link: http://www.theperlreview.com/?orm

The third print issue of The
Perl Review is now on the presses and will be mailed during the first week of March
(so you have time to href="https://www.theperlreview.com/cgi-bin/subscribe.cgi/orm">get your
name on the subscriber list). TPR is
the only print magazine devoted to Perl.

In the Spring 2005 issue (You can see the first page of most articles for free):

• Hashes with History — Alberto Manuel Simões
• Test::Number::Delta — David Golden
• 9-Block Quilt Patterns in Perl — Daniel Allen
• Packet Sniffing with Perl — Gerry Finkel
• Serious Perl — Henning Koch
• Barcodes from Perl — brian d foy
• plus Perl News, Perl Mongers and Perl Foundation reports, book
  reviews, short notes, and more.

Subscribers get immediate access to the href="http://www.theperlreview.com/Subscribers/?orm">online PDF
versions:

I think I need to create, and distribute, a dictionary of Perl terms. I can’t disagree more with BBEdit’s judgement of Jarkko.

Related link: http://www.usps.com/webtools/technical.htm

I applied for a User-ID and password so I could use the US Post Office’s web services. They have some promising looking tools: zip code lookup, city/state lookup, address verification, and some other things.

A couple of hours after I applied for an account, I got my welcome email. I was on to the next task though, so I filed it. Tonight I wanted to take it for a spin.

I wrote a little program to give it a go. I didn’t follow their technical details because I don’t want to put a long XML string in the query string of the XML. This is an idempotent request, but I’ll put that stuff in the message body anyway and use a POST request.

Here’s the code. Notice I have my ID and password in the environment. The USPS says on just about every other page that I can’t give out those credentials. I can’t share them and I can’t tell anyone else what they are. Fair enough.

Look at the request scheme though! It’s plain ol’ HTTP. That’s plaintext floating across the air, or copper, or whatever. I tried sticking https in there, but it never makes a connection. Every time I test this little application, I’m exposing my credentials. You don’t have to hack ChoicePoint to get that.

use HTTP::Request;
use LWP::UserAgent;

my $content =<<"HERE";
API=Verify&XML=<AddressValidateRequest
	USERID="$ENV{USPS_ID}"
	PASSWORD="$ENV{USPS_PASS}">
<Address ID="0">
<Address1>5250 N. Kenmore Suite 157</Address1>
<City>Chicago</City>
<State>IL</State>
<Zip5>60640</Zip5>
</Address>
</AddressValidateRequest>
HERE

my $ua = LWP::UserAgent->new();

my $request = HTTP::Request->new( POST =>
'http://testing.shippingapis.com/ShippingAPITest.dll' );
$request->content( $content );

print $request->as_string;

my $response = $ua->request( $request );

print $response->as_string;

Okay, it’s their system and a password to their system. Obviously they know what they are doing. They are the government after all.

Not so fast. Check out this response: All I really have is a User-ID and password. I can’t actually use the service, even on the testing service. It turns out that I have to request that separately. Ughh. Not only that, they are using IIS. Oh boy, so this service will down a lot, won’t it? I’ll have to wait to see about that because I need someone to authorize me to use the web service I signed up for two weeks ago.

HTTP/1.1 200 OK
Connection: close
Date: Fri, 25 Feb 2005 11:04:46 GMT
Server: Microsoft-IIS/5.0
Client-Date: Fri, 25 Feb 2005 11:04:40 GMT
Client-Peer: 56.0.134.43:80
Client-Response-Num: 1

<Error>
	<Number>80040b1a</Number>
	<Description>API Authorization failure.
User 931THEPE4647 is not authorized to use API Verify.</Description>
	<Source>UspsCom::DoAuth</Source>
</Error>

Related link: http://mappinghacks.com/

Although Mapping Hacks isn’t slated to hit the shelves until sometime in May, you can start following the latest developments in Open Source GIS and digital cartography on the Mapping Hacks weblog.

In the military they are very big on succession planning. This mentality comes from their heritage — armies exist to go into battle. In combat situations, everyone is vulnerable, so they spend time making sure they have a replacement for any critical function. Even in non-combat units, they share the principle and prevent the single point of failure.

In business, we are aware of this logic, and everyone agrees it makes sense, but unfortunately businesses does not share the same culture as the military. So for many different reasons, sometimes we find critical skills trapped in just one employee. Inevitably she is under-appreciated, overworked, and looking to advance. So the dreaded day comes when she “needs” to talk and before you know it, there on your desk is her letter of resignation. She has been looking for a new opportunity and has found another company to appreciate her and provide the advancement she desires.

Your first thought is typically “How are we going to survive without her?” Which is quickly followed by “We can’t survive without her! I know! We’ll make a counter-offer!” Before you let that train of thought continue, let me say, just say no!

Consider for a moment her fidelity. By the time she drops the letter on your desk, her fidelity is no longer with your company. It took effort on her part to pursue this new opportunity. This effort typically changes a person. In the end, when her mind was made to accept the new offer, she also decided that your company was no longer in her best interest. So briefly consider whether you want an employee who no longer believes there is a future in your company?

I say cut your losses and be done with it. Suck it in, do your cross training during the time given, and start your search for her replacement. I’ve seen the counter offer used on several occasions and in each case, both employees eventually left the company for another new opportunity.

Another twist to consider as well concerns trust. Typically when someone informs their employer they are resigning, they have already accepted the new position. When they accept your counter-offer, they will have to inform the other company that they have changed their mind. So do you want an employee that does not honor their word?

Now of course, there are situations when this advice is not appropriate. I’m also aware that filling a gaping hole left by a key resource will not be easy. Neither am I saying that an employee who resigns and accepts a counter-offer is a bad person. I urge you to consider carefully your decision when this situation occurs. What I’ve found is that a counter-offer will only buy you some time. You’ll end up filling that hole now or later. So unless there is a really good reason, get out your spackling, patch things up, and move on to the next fire fight.

Have you ever accepted a counter-offer and stayed at a company?

Like that FedEx Super Bowl commercial that said every great commercial must have a certain list of elements (a celebrity, some animal, a kick in the groin, a punchline, etc), every great class should have a certain list of elements for its success. Students love examples, hands-on demonstrations, and something to keep and utilize for a long time to come. Because of the increasing use of technology in the classroom, and it is commonplace for most rooms to have a computer and even a projector, instructors want flexibility and consistency of technology. The Knoppix CD satisfies the need of students and instructors.

I am currently teaching my course entitled Security, Privacy, and Politics in the Computer Age, offered at Tufts University by the Experimental College. There is a Dell computer in the room that I teach in, but it is problematic. I have to login using some generic account, thus I do not have many privileges; error messages about the hard drive running low on space, but there actually is plenty of space, constantly pop up; and I can’t run some examples from the SSH terminal that is provided on the computer. To resolve these problems, I boot the computer using the latest Knoppix CD. For instructors who roam to different classrooms, a Knoppix CD is nice because you will have a consistent look-and-feel on different computers, a consistent and flexible set of tools, and you can bypass numerous barriers.

I have twenty students in my class. All of the students are familiar with Microsoft Windows. Only a handful are familiar and have used UNIX or Linux. I have used Linux on numerous occasions: to demonstrate software tools such as John the Ripper (how easy it is to crack passwords), the ping command (although it is installed on almost all Windows systems anyway), how to send forged e-mail via Perl script, and open source packages such as the Gimp, GAIM, OpenOffice during my lecture on open source software. The students were spellbound when they saw a live Linux desktop for the first time.

I gave each of my students a copy of the Knoppix CD. The value of this is tremendous. They can test-drive and learn Linux (especially basic commands) without installing anything on their hard drive. They can use most of the popular open source software packages, can play games, and even browse the Internet. Of course, I also told them that they can use the Knoppix CD in case of emergencies to retrieve and backup files, scan for viruses on their Windows hard drive, and reset Windows passwords. Last Saturday, I received an e-mail from one of my students that he needed to reinstall Windows but the install CD didn’t work. He was locked-out of his system (Windows could even boot, and the hard drive was on the verge of being erased), and feared that some of his critical documents were lost. Fortunately he had the Knoppix CD and was able to read and back-up the files. So everyone was happy with that news.

I’ve received about 20 of these sorts of emails this week, which is up from the usual 0 or 1 a week:

From: ...
To: comdog@panix.com
Subject: Please Remove

I received a medical newsletter from this email address although I am not
sure it is legitimate.  If you are a doctor and your office has a
newsletter, please remove me from your list.

If this message is in error, please ignore.

Thanks

There’s been a surge in worm activity, I think, so all of those Windows computers infested with it have been sending all sorts of mail using whatever names they find in people’s address books.

It’s still depressing that people think that people who are trying to sell snake oil would use their own name.

I was recently looking over the Hula Project, the new calendaring and mail server that Novell recently open sourced.

I was struck by the idea that they wanted to fundamentally rethink what people used a calendaring system for. For example, here’s a whiteboard shot laying out some if the different ideas they’ve been working on.

But these ideas got me thinking. It looked to me as if they were starting with ideas steeped in current technology. That is, I felt they were limiting themselves by beginning with ideas from the current technology — not looking at fundamentally how people could use calendaring technology. This seemed like a limiting approach to me.

I got thinking that if instead of beginning with ideas like ‘notifications’ and ‘aggregation’ they began with common ‘human interactions’, then that may lead to some interesting places.

Here are a couple ideas that look at ‘calendaring’ as “Coordinating meetings and tasks among different spheres of people”.

Common human interactions that are facilitated by calendaring might be:

• Meetings with co-workers
• Meetings with people in other companies
• Social events
• Family activities
• Etc.

In general, I believe this can be generalized as ‘coordinating events and tasks with people in different spheres’. We all have different spheres of people we interact with. My spheres might be:

• Work
• Family
• Friends (different circles/spheres of friends)
• Various professional ’spheres’ of people.

And calendaring facilitates each of these spheres of people by coordinating both 1) meetings, and 2) common, coordinated task scheduling.

One spin on this is that each of these ’spheres’ of people may use different calendaring systems to interact with each other. I may use outlook or evolution at work, but among friends I may use an e-mail list. A central question is essentially, “How do I integrate the common events and tasks/activities of different spheres of people to give users a way to manage them all easily and make the interactions fun and useful?”

Some ideas that make this even more interesting are:

Varied Clients/Media for facilitating interactions.

• Different people will use different clients or prefer different media for interaction. For example, many will use an e-mail client, but others may want to use a browser or a cell phone.
• Using a cell phone as your way to interact with your ’spheres of people’ makes it possible to use audio/voice rather than text/images for interaction. (For example, ORBITZ calls my cell phone with audio-based flight updates based on my flight schedule.)
• Using a cell phone with a camera and rich display as your way to interact with your ’spheres of people’ makes it possible to use pictures/images rather than text/images for interaction.

Archiving/historical records of interactions (i.e., the “memory” of the interaction)

People that meet with each other create ‘memories’ of each interaction that can take various forms. These forms may be:

• Blog entries regarding the event
• Pictures of the event
• Recordings or voice annotations of the event
• New tasks or other meetings that follow from the event.

These ‘memories’ are of value to the spheres of people involved in the event and are likely generated by them. This is the same whether the event is a birthday party for my son or a customer meeting. I’d posit that these various memories should be part of (or at least facilitated by) the calendaring system.

One example might be generating a ‘tag’ for Flickr where images associated with the event can be accessed. The calendaring system (using the Flickr API) should create the tag and provide users of the calendaring system a way to locate images of the event. (Again, this is just one example.)

Given all the latest technologies (like RSS, digital photography, audio/pod casting, super cell phones, etc) it’s almost as if every application we commonly use should be rethought with an eye toward leveraging all the infrastructure that’s being put into place.

What are your ideas?

How can calendating systems leverage new and emerging technologies?

Hunter S. Thompson was a hacker of the first order. Not a hacker in technology terms, but a hacker none the less.

HST’s profession was journalism. Instead of choosing to follow the rules and trying to write the way people are taught in journalism school, he instead chose to hack the entire profession.

He threw out the rules and invented his own style of journalism, which he called “Gonzo Journalism”. His idea was to insert himself (as the journalist) into the middle of the story; to literally become part of the story. Then he related the story to the reader from a participant’s viewpoint.

Being in San Francisco in the 1960’s, he absorbed himself in the culture of that time and place — and that was reflected in a great deal of his work. But if you read beyond his most popular work (Fear & Loathing in Las Vegas), you find that his desire to literally reinvent journalism led to his doing a great deal of other brilliant work.

Like the time, for example, that he paid someone to take him on a boat to a smuggler’s village on a remote beach somewhere in South America. He waded ashore — not speaking a word of the local language — with a few belongings and his heavy typewriter and got a great story. He noted that, even though they lived in a remote village, they all wore rolex’s and had top of the line scotch and rum (spoils of their smuggler’s lifestyle). He spent time with them and got to the heart of the story; for some reason they spared his life and let him go.

He didn’t like the way journalism was being worked, so he hacked the rules and made journalism work the way he wanted it to.

To steal a quote from Fear and Loathing in Las Vegas (via Nat Friedman’s post today):

“There he goes. One of God’s own prototypes. Some kind of high powered mutant never even considered for mass production. Too weird to live, and too rare to die.”

Note: Here’s a page put up by a fan for his admirers to leave their thoughts.

Related link: http://www.google.com/search?q=inurl:blah.php&hl=en&lr=&c2coff=1&start=10&sa=N

Try the following URL:

http://www.google.com/search?q=inurl:blah.php&hl=en&lr=&c2coff=1&start=10&sa=N

You will get this message:

In other words, Google won’t let you look beyond the first 10 results if you request it to find URLs that point to PHP resources. This is most likely Google’s response to recent worms that have been using Google to look for vulnerable versions of PHP code.

In this case, Google is attempting to help thwart the propagation of malicious worms, and that is a good thing. To some degree, it is also in Google’s self interest to help out: thousands (perhaps more?) infected machines were consuming Google’s resources.

Although the number of users who may want to search for PHP resources (in the URL) may be small, this concerns me. Where will Google draw the line in the future? Should Google continue to forbid more types of queries, it will become less useful to legitimate users.

In my Google Your Site For Security Vulnerabilities article, I mention various queries that can be used to find vulnerabilities using Google. The ideas in that article derive from the excellent resources at Jonny Long’s website which contains a huge database of such queries. What if a worm in the future were to use an entire database of such queries to help it find vulnerable targets? At what point will Google decide against banning a certain query because doing so would interfere with legitimate use?

On the other hand, imagine a few hundred thousand PCs infected with worms hammering away Google for additional attack targets, thus causing a denial of service attack on Google’s network. In such a case, Google will be left with little choice but to ban a bigger list of queries. It will be interesting to see how this plays out in the future.

Related link: http://www.sugarcrm.com/home/SugarCRM_Series_B/234/

As I mentioned in “Follow the Money!” these venture capital deals involving open source companies are important. I believe this deal is of particular importance. First, this deal is exactly the kind that should worry Oracle/Peoplesoft and its shareholders, as I mentioned here. Although each venture capital firm has their own metrics and guidelines, most stick close to the industry standard. They look for ventures that are poised for rapid growth. VCs are typically looking for companies that expect at least 20% growth for the next five consecutive years. Remember, they expect AT LEAST 20% growth, it is usually much more.

Proprietary CRM software is not cheap. A typical Siebel installation will easily cost above $3 Million if you include license fees, installation services, and technical support. salesforce.com has done well because it provides a low entry alternative, however the application services model is not for everyone. Therefore, SugarCRM is poised to bring CRM functionality to millions of small and medium sized enterprises (SME) that previously could not afford the entry price and were not interested in the ASP approach. So in the short term, salesforce.com is SugarCRM’s biggest competition. I’m sure their strategy is to dominate the SME market and stay out of reach of bigger competition. This will allow SugarCRM to grow quickly. Once they are big enough, they will set their sights on the markets and customers were Oracle/Peoplesoft is under-performing. Maybe then Oracle/Peoplesoft will regret their support of open source software as a marketing strategy.

The second reason this deal is important is by marking a change in the open source industry. Up until now open source has thrived in the server room. With this deal it emerges onto the desktop. Web servers, database servers, and file and print servers are transparent to most workers — as they should be. SugarCRM, however, is an application and it runs on client computers. A whole new set of users will now interact with open source generated applications on a daily basis. Accounts Receivable, Accounting, Customer Service, and Sales personnel will now begin to hear about open source software. This is a clear example of what Geoffery Moore will discuss in his keynote speech “Open Source Has Crossed the Chasm — Now What?” at this years Open Source Business Conference (OSBC). Once SugarCRM and other open source software becomes accepted in the these user communities, it will much easier for others to follow. None other than Larry Augustin, CEO of MedSphere will offer his own vision of what I’m predicting here in his session from this year OSBC, The Next Wave of Open Source: Applications.

Who will be the next Open Source Application to get financing?

Related link: http://linuxworldexpo.com/live/12/events/12BOS05A

Last night I was discussing with friends what could be the biggest
barrier (or at least the biggest technical barrier) to Linux
desktop adoption: the refusal of consumer device manufacturers to
release specifications that allow the community to develop drivers.

It’s clear that people are increasingly seeing computers as just tools
to interact with a colorful and sublimely noisy world; they love their
digital cameras and scanners and videos and music files. So for Linux
to move out of the basic black storage cabinet in the air-conditioned
back room and onto the desktop, drivers for all these devices have to
appear.

So the four of us batted around several strategies for forcing
manufacturers to open up, which became more and more extreme and
unprintable as the drinks went down in our glasses. We talked about
manifestos and boycotts and appealing to the European Union. But I
like best a campaign that I suggested: asking the stores and web sites
that sell consumer devices to post lists of devices that have open
source specifications. This is like putting nutritional information
(or in Europe, notification of genetic engineering) on food packages,
except that I wouldn’t expect governments to require it.

If enthusiasts for open source operating systems wrote a bunch of
retailers and just asked for signs and web pages saying “Open source
specifications available: Foo-Device 808A, 1001D, 1022X…” it would
accomplish several things. It would show the extent of public support
for opening up the specs, make retailers aware of the issue (and aware
that they could quiet people down by making a minimal concession),
reward cooperating device manufacturers with publicity, and–not least
in importance–bring media attention.

We shouldn’t compromise on the definition of open source here: no
binary-only portions should be tolerated, and every advertised feature
should be available to open drivers.

Why aren’t the manufacturers already releasing their specs?
Occasionally they say it’s because somebody could abuse the device and
do something dangerous, particularly where wireless spectrum is
concerned. Personally, I feel that the Supreme Court “Betamax”
philosophy should hold: manufacturers should give the public powerful
and useful instruments and the public should be held responsible for
their use. (After all, nobody has suggested banning lasers, even
though their effect on air travel is much more dangerous than what
people could do with wireless devices.)

In addition to the concern for misuse, manufacturers may harbor vague
worries about releasing trade secrets. They have to be persuaded that
secrets hurt sales.

People like to set Microsoft up as the big nemesis for open source,
but I think it’s more positive-minded and ultimately productive to
look at the companies the open source has to deal with–and that could
be our allies.

A few reports on companies

I found out a bit more today about
MySQL Network,
which naturally makes one think of Red Hat Network and shows some of
the same concern for offering something extra to those willing to
license open source software. Without changing a single character of
the source code, MySQL AB offers its licensed customers a faster and
more robust product than they could get through a download (unless
they employ their own experts to recompile it). Through tweaking
compile-time options, testing on various operating systems, and
certifying results, MySQL AB can reassure customers that their
database engines will run fast and stay up. This goes along with a
Knowledge Base, an advisory system for security alerts, and various
other standard elements of software support.

A head’s up: the next MySQL conference (which O’Reilly is running)
takes place in Santa Clara, CA this coming April, and early
registration ends February 28.

Emic Networks
continues to grow and roar ahead with a clustering solution for MySQL
and Apache. They point out that you can use their clustering solution
without switching to a different database storage engine as MySQL’s
clustering solution does. (I gave my own review of MySQL Cluster in a
blog
from last year’s MySQL Conference.)
Emic now offers a console for convenient management of their clusters,
and plans to move in the direction of integrating the management with
other logging and monitoring subsystems so you can use the same
familiar tools that you use for the rest of your system
administration.

Radiant Data
is also in the clustering business, offering an interesting filesystem
called PeerFS. Any system–even the laptop they were using as a demo
at the show–can participate as a peer in a continuously synchronized
network running Ethernet, Fibre Channel, or iSCSI. If somebody updates
a file on one system, it is quickly updated on all the others. PeerFS
exchanges all the locking information as well as incremental data
changes to keep files up-to-date and uncorrupted. Their demo shows two
systems running autoincrements on a MySQL table, and properly sharing
the index. Kids, this is something you can try at home.

While many companies tried to find niches in the MySQL ecosystem at
LinuxWorld, Oracle and Sybase of course were present too.
Sybase
is building on their established customer base in Wall Street,
government, and Asia, and is offering migration tools to companies
trying to get off various systems that Sybase regards as obsolete (but
that I will be too polite to list here). Also, a
Standish Group report (PDF)
promoting Linux mentioned last year that Sybase delivers “the most
economic performance” on both Linux and Windows.

Speaking of migration, I dropped by the
Alacos
booth to find out why their Linux Migration Agent won the Best
Integration Solution award at LinuxWorld. What’s cool seems to be that
you don’t have to load software on any desktop to migrate bookmarks,
address lists, and other user preferences–all you do is insert a CD
that describes what you want migrated. You can do the migration from
Windows to Linux on a single system or move the Windows settings to a
Linux operating system running on a different computer.

Novell
had a booth the size of a city (probably because their headquarters
are local). In addition to promoting the Novell Linux Desktop and
other SUSE-based operating systems, they were focusing on their
traditional identity products, which support single sign-on and
various administrative conveniences. One of these earned another of
LinuxWorld’s much coveted product excellence awards. Companies
struggling to comply with Sarbanes-Oaxley take note: you can use Nsure
Audit to log a wide range of user access data to a central facility.

Sarbanes-Oaxley was on the minds of many vendors at the conference,
which I assume means that customers are worried about it too. For
instance,
ConfigureSoft
claims that their system for distributing and keeping track of updates
not only speeds up this onerous task, but provides evidence that one
has conformed to the software security requirements in
Sarbanes-Oaxley.

I have written in other blogs that storage and backup are key concerns
for large organizations, and are becoming more and more a concern as
the volume of data goes up. One interesting response to this has been
the release of a new version of an SSH implementation by
SSH Communications Security.
This is the company that developed the original SSH implementation,
known as OpenSSH and their proprietary implementation SSH Tectia
remains compatible with the ssh2 run by Linux users everywhere. But
the company has managed to speed it to the point that their customers’
data transfers take only one-third as long as they did on their
previous version.

Last (as they are alphabetically) but not least,
Zend Technologies
is evangelizing PHP 5.0 in the expectation that its object-oriented
capabilities will interest more and more companies to build
mission-critical applications on PHP; they definitely want to move up
the enterprise ladder. While PHP will always be a scripting language
and will therefore lack certain assurances that Java or C# offer the
programmer, it is very easy to learn and offers some pretty powerful
features, such as a simple API for Web Services and even SAP access.

LinuxWorld Boston summary

This LinuxWorld was definitely smaller than the New York City ones (as
well as the San Francisco one, of course) but was solid and
successful, at least for O’Reilly. I saw a few new things, but mostly
the show revealed some jockeying on familiar tracks: companies
striking partnerships and incrementally improving their offerings.

Other blogs I wrote from this conference:

• First day at LinuxWorld: moving up the free software stack and other progress
• Penguin Bowl results and other LinuxWorld happenings

Related link: http://linuxworldexpo.com/live/12/events/12BOS05A

While a few companies at
LinuxWorld Expo
base their business on offering Linux, and a few more on offfering software that runs on Linux, a hefty number operate within the larger computing ecosystem of which Linux a part. For instance, Linux is brought within the mission-critical task of storage and backup by such companies as
Arkeia
and
BakBone Software. This year I noticed a new micro-industry at LinuxWorld: Windows-to-Linux migration. Scads of books are coming out on the subject, and now some vendors are cleverly figuring out how to package up the tasks available for automation, mostly migrating all the little things such as customized dictionaries and calendar entries that one builds up in application-specific data stores over the years.

Of course, migration of any sort is a major undertaking that requires a lot of planning and marshaling with the organization, and software can’t help you with much of this. But wouldn’t it lower barriers to migration if you could reduce the time it took an administrator to convert a single user’s settings from an estimated six hours to twenty minutes? That’s what one of the migration companies,
Versora,
suggests can happen with their Progression software. They actually do much more than dictionaries and calendars; for instance, they can load a MySQL database with the data from SQL Server. They are finding a lot of interest in their product among companies with 500 or more systems to convert. In such an environment, the cost of the software might be justified not only by the time savings but by maintaining the system administrators’ sanity.

To me and to many potential customers, the inevitable question came up: could Versora automate a conversion from Linux back to Windows, in case the companies are not happy with their Linux migration? Although Linux ideologues might not like the concept of a reverse path, it might induce more companies to make the leap to Linux, and Versora is taking note of the requests.

MySQL AB
is also taking note of the environment in which their product runs. They now have a large project called MySQL Network, which contains their knowledge base, indemnification, and other sorts of non-technical components of enterprise computing. They think their upcoming 5.0 release will let them reach a new tier of enterprises with heavy database requirements, and actually pushed some features from 5.1 back to 5.0 to make it more attractive. (The major enhancements cited by VP Zack Urlocker were stored procedures, views, and triggers.) The code and staff they got from SAP were a great help in developing the latest wave of features. MySQL AB is also focusing more than ever before on their GUI tools such as the MySQL query browser. These tools are currently oriented toward database administrators, but will hopefully be helpful to users in the future as well.

I have reported several times on
Black Duck Software,
which maintains a database of open source software and lets companies check their own code against it to make sure no one has snuck in something from an open source project that taints the company’s product. Black Duck is facing (literally across the aisle at LinuxWorld) a new competitor,
Palamida. One of the services stressed by this new company is a database of binary fingerprints that lets one search for infringing binaries as well as source code. The search for binary infringement is like the snaring for viruses in email, and indeed Palamida bases its technology on research in the area of viruses.

Penguin Bowl quiz show

In previous years I haven’t bothered to report on this LinuxWorld Expo fixture, which has a precedent in the incredibly wacky and stunning Internet Quiz Shows that Jon Orwant put together starting in 1997 for the O’Reilly Perl Conference (now there’s a bit of free software trivia). But I find it harder to ignore the Penguin Bowl this year because I found myself on the stage. I was part of a “Media” team that competed against at “Analyst” team for the prize (membership on both teams was very loosely defined).

When asked to join the media team, I wanted to protest that the combination of search engines, handheld computers, and wireless connections has rendered obsolete the practice of memorizing facts and therefore downgraded the value of quiz shows. But I am not hard to draw in when an opportunity to make a fool of one’s self publicly comes up, as you can tell from the quantity of my blogs.

The analysts pulled off some impressive events, such as writing infinite loops (which MC Jeremy Allison of the Samba team called patentable perpetual motion machines) in eight different languages. But the media ended up slightly ahead, thanks mostly to the vast knowledge store of Don Marti, editor in chief of the Linux Journal. I picked up some points on the easier questions, such as, “Which desktop came first, KDE or GNOME?” My answer on this question made up for my incorrectly identifiying the inventor of the mouse as Alan Kay rather than Douglas Englebart. But my main contribution, I think, came at a point when the media had left the analysts in the dust, and I tried to sooth their feelings by explaining that the media was winning because we never get fired for saying wrong things, and therefore are bigger risk-takers.

The overall point is that we had fun and the judges balanced justice with charity, so the Penguin Bowl upheld a model of what the open source movement should be.

Related link: http://linuxworldexpo.com/live/12/events/12BOS05A

Jacking in from the first day of the first LinuxWorld in Boston,
Massachusetts, I’ll discuss the following in this blog:

• Silly obligatory St. Valentine’s Day reference

• Microsoft’s entry into free software, and other observations from OSTG

• Scalix: an example of moving up the free software stack

• Looking ahead to the rest of the conference

Silly obligatory St. Valentine’s Day reference

LinuxWorld happens to start on St. Valentine’s day this year. So (like
many other superficial-minded journalists attending the conference,
I’m sure) I searched around for silly metaphors involving
St. Valentine. Oddly, I found one that was appropriate.

St. Valentine is the patron saint of beekeepers. Bees are valued and
cultivated for their honey–which is certainly a miraculous
substance–but another, lesser-known product from bees may be even
more valuable. I am referring to propolis, a kind of glue that bees
make from the wax and resins they collect. Propolis has valuable
anti-biotic properties that make it useful even today for healing
cuts, burns, and dermatological problems. It provides a general guard
against disease and infection.

We all wait expectantly for Linux to yield us its honey–the rich
variety of desktop and multimedia programs that the free software
community has created for it–but we must remember that Linux is even
more valuable for its propolis–for the inherent security of its
design and the robust operation that earned it the term “unbreakable”
from Oracle Corporation.

Microsoft’s entry into free software, and other observations from OSTG

The ancient city of Jericho once experienced a crisis: its waters had
turned bad and polluted the land. The prophet Elisha, newly brought
into the role of prophet by the great Elijah, threw a jar of salt into
the water. A miracle! The water was purified, and the people could now
thrive. But strangely enough, as Elisha was leaving town, some youths
mocked him.

Why is that? Commentators suggested an answer two thousand years ago
by adding another dimension to the tale. They said the mockers were
merchants who had based their living on bringing water to sell to the
inhabitants. They were furious at Elijah for cleaning up Jericho’s
water and ruining their business!

Has anything changed over two thousand years? Even now, no good deed
goes unpunished. When people contribute free software that increases
the common pool of productivity, the narrow proprietary interests that
profited from the lack of functional software strike back.

While Microsoft publicly tries to poison the open source well with
stern animadversions, it quietly tests the waters by releasing open
source projects of its own. No, I am not talking about the tangled,
encumbered Shared Source initiative. Rather, check SourceForge for
Wix
(the Windows Installer XML toolset) and
FlexWiki
(a collaborative web-based authoring environment implemented on the
Microsoft .NET platform)–two of the bona fide open source projects
that Microsoft has put up. “To their credit,” says Colin Bodell of
VA Software,
the company that owns SourceForge. “They ought to be exploring open
source, and it’s good that they’re doing so.”

Can companies open up proprietary software successfully? Many
observers say that such efforts don’t work–whether because the
community doesn’t see the projects as their own, or the companies put
barriers up in front of user contributions, or for other reasons–but
Bodell thinks they can. He suggests that Computer Associates, by
making Ingress open source, created a base of expertise among its
users and thereby offloaded onto the users a lot of its customer
support costs. And he referred to other projects that had reduced
support costs the same way. (I cynically pointed out that an
investment in better documentation might have achieved the same
benefits.)

Bodell is one of those who believe in the conquest of free software up
the stack. Having achieved great things in providing infrastructure,
free software will take on applications next. It is already difficult
to find any proprietary software product for which there is no free
software project trying to compete–and bit by bit, the open source
alternative is becoming more viable. Bodell cited CRM solutions
(Compiere
and
SugarCRM)
in particular. We’ll see another example in the following section.

VA Software developed incrementally the list of
Open Source Technology Group
sites that are now household words (among technologically
sophisticated households):
SlashDot,
SourceForge,
Linux.com,
ThinkGeek.com,
and so forth.
An integrated vision for these offerings has evolved along with the
sites themselves.

Originally, as VA Linux, the company was searching for a way to
quickly bring into being the kinds of third-party applications that
existed for other vendors with proprietary systems. Rather than build
(slow) or buy (expensive), they decided to facilitate what the free
software community was already doing by starting SourceForge. As they
noticed other information gaps, they started sites to fill them. And
In subtle ways these sites are all integrated. For instance, a manager
might visit
ITManagersJournal.com
to find news about software that might be worth using, and pass on to
a staffer the URL that points to implementation details on
SourceForge. Postings on SlashDot (often consisting of nothing but a
URL, but modded up to the highest rating by users) take readers to
valuable information and software. Every level of potential free
software user is served, from novices (Linux.com) to developers
(SourceForge).

OSTG has just announced the milestone of registering its millionth
user. As it scaled up over the years, it’s had to make some
interesting technical innovations. It has enhanced the PostgreSQL
database, and passed its changes back to the project when they would
be useful to others. It also has a clever proxying server for CVS so
that multiple CVS servers can host different projects and be accessed
by users through the same interface.

Scalix: an example of moving up the free software stack

Looking ahead to the rest of the conference

At LinuxWorld this year, I will probably meet other companies hungry
for the Exchange server market, along with proprietary computer
vendors making the big transition and asserting their open source
credentials, racks and racks of blade clusters, and companies offering
GUI sugar for common administrative needs. I meet them every year. But
when one makes a mark in a way that’s worth noting, I’ll note it
here. And I’ll be on the lookout for new paradigms in free software.

St. Valentine (or one of the two other early Christian martyrs named
Valentine) was famous for healing a blind girl. He thus serves as a
good patron for LinuxWorld, which tries to progress year by year in
gradually curing the leaders in business and government of their
blindness toward the benefits of free software.

The blindness is slow to lift. Short-term thinking wins out over
strategic advantage. The importance of transparency in public
institutions’ software–like transparency in other areas of public
discourse–is little appreciated. IT departments fear dislocation and
the costs of retraining above everything–even in a world of constant
innovation where people always are having to learn something new. But
change comes nevertheless. The Boston Globe announced this morning a
repository for free software for government sites.

I do not by any means ignore the enduring problems of Linux,
especially as a desktop system (I run into some new problem every
week). Nor am I blind to the new ways of thinking required to get
free software tools working together smoothly. But this is what IT
departments in what large organizations are for. It’s time for
executives to open their eyes and get their IT departments to do what
they’re paid to do.

I haven’t explained to the public WHY I’m re-writing CD Baby from scratch, and I think the reasons would be VERY useful to you if you’re a programmer or webdesigner that may have started a project that will grow in ways you never expected.

When I started writing CD Baby back in 1998, it was only a hobby that I made to sell my CD, and some friend’s CDs.

I assumed it would ONLY sell CDs, nothing else, from my one location, in one language. That’s all I needed it to do. OOOPS!

• Need us to sell something much heavier than a CD? Can’t do it. In 100 different places in my code, it calculates shipping cost by just counting the number of items in the order. It has no concept of weight. (Also - no concept of NON-weight: like a download.)
• Want to sell something with different sizes/colors/variations? Can’t do it. The whole system is set up where an item is just an item and doesn’t understand the idea of variations or sub-items.
• My business could really use multiple warehouses, but the code can’t. The whole inventory/stock/shipping idea assumed there was only one warehouse.
• I wish the site could be in multiple languages, but like an idiot, I wrote all of my English words directly into my HTML code, so now to replace them with language-variables, I’d have to re-write every single line of HTML. MUCH harder than if I had just put all the language in one file to begin with.

The list goes on, in less-obvious ways. For years I had been saying, “If I could do it all over again…” - and that list got so long that it was time to do it all over again.

How to solve these problems and future ones I haven’t imagined yet?
EXPECTING CHANGE: question yourself anytime you assume anything will “always” be a certain way (examples below).
ENCAPSULATION: how to deal with anything that might change in the future: make sure it’s not assumed anywhere in your code. That there is only one definitive source for any bit of information or logic, and it hides its methods, so that if it needs to change, you don’t need to go change your entire program.

My example solutions to the problems named above:

• Shipping cost will be calculated in one single place. An order will get it’s shipping cost from this one definitive source, so that if my shipping rules change, (or if I start selling items that need no shipping), I only need to change that one file.
• Item will not assume it’s an album: more generic, they can be anything, or have have variations.
• Warehouse/Inventory/Shipping will not assume only one warehouse. I’m going to start calling our existing warehouse “warehouse #1″, and plan that there might be more.
• Words displayed on screen will not assume they are always English. I’ll put a variable where that phrase should go, and call it from a single language file, remembering the sentence structure might be different, so I can’t assume it will say, “Welcome, $username!”, because some languages might need to say, “You $username welcome are!”. (The language file itself uses printf-style, so it puts the external variable in a %s placeholder, where it’s appropriate for that language.)
• Can’t assume all languages read from left-to-right. Hebrew and Arabic are right-to-left.
• Can’t assume all currency is USD
• All visual presentation will come from a single CSS file.

I hope my bad experience helps you question some assumptions in your program. My best advice is making sure ALL words that display on the screen are taken from a single language-config-file. That one move will get you into the right mindset of expecting change and encapsulation. Then you can start noticing other things that your program/website is assuming will never change.

I just went looking for links that explain encapsulation to beginners, but I can’t find any good ones! Maybe I’ll have to write a future post with my take on explaining encapsulation to non-Java-geeks.