In a contextual sea change, Google has announced a beta program that allows publishers to embed AdSense contextual ads in RSS and Atom syndication feeds.

“This is gonna be huge… like HUGE HUGE HUGE!!!” gloats Jason Calacanis, the President of Weblogs, Inc., which - as you might expect - publishes a bunch of syndication feeds. As a publisher of syndication feeds in a small way, I suppose I ought also to be glad. But actually, I think monetizing RSS and Atom feeds in this way in part defeats the purpose of having a feed. Feeds are simply an information stream that point to further information. If they get cluttered, they cease to be useful, and subscribers will cancel. In some sense, the RSS or Atom feed is an advertisement for the full content in and of itself.

Case in point: it is against the policies of Hot Feeds and Syndication Viewer to display feeds that carry ads.

Here’s the way Weblog’s unofficial Apple feed, which Calacanis is using to test the syndication AdSense program, looks (with ads) in Syndication Viewer. Each Google AdSense ad is simply an HTML table embedded in feed items like this [identifying numbers and actual link omitted]:

<br /> …<br /> <item><br /> …<br /> <description><br /> <![CDATA[<br /> ...actual item...</p> <table border="0" cellspacing="0" cellpadding="0"> <tr> <td><span style="font-size: 11px;"> <br/></span></td> </tr> <tr> <td><a href="http://imageads.googleadservices.com/pagead/imgclick/XXXXXX pos=0"><br /> ….<br /> </a></td> </tr> <tr> <td> <div align="right"><a href="http://www.google.com/ads_by_google.html" style="font-size: 9px; color: #999999;">Ads by Google</a></div> </td> </tr> </table> <p>]]><br /> </description><br /> </item><br />

The good news: it ought to be trivial to parse these ads out of incoming feeds, simply by eliminating table tags and their contents from item entries if in no other way. I will certainly do so in Syndication Viewer.

Related link: http://www.perl.com/pub/a/2005/04/28/bdfoy.html

O’Reilly editor chromatic interviews me about my magazine, The Perl Review.

I had almost forgotten about that since we did the interview back in January. Both of us were distracted with book projects for O’Reilly.

Related link: http://www.bsdcan.org/2005/activity.php?id=54

Dan Langille tipped me off that BSDCan 2005 will host the premier of a new open source tool to perform opportunistic live mirroring of remote systems over the Internet. Anywhere you go, if you have an Internet connection, the tool will mirror changes to your filesystem to your backup server. It’s NetBSD only for now, but it sounds useful enough that there’ll soon be ports to other operating systems.

Related link: http://www.cdbaby.com/

CD Baby only has one warehouse today. But by the end of the year, we’ll have multiple warehouses.

I used to count current stock/inventory by using a simple database table called inventory that really only counted items_received, then subtracting the quantity of items sold from the order’s lineitems table.

But with multiple warehouses, this won’t work anymore! Each warehouse needs to know how much its own current stock is. Unless I were to actually tie an order’s lineitem with the warehouse_id it came from, I would need a new approach.

So - the new approach is this:

INVENTORY is a detailed historical trace of everything in and out of the warehouse.

STOCK is a current-status : how many of each item are in each warehouse right now. Used for quick lookups.

## THE DATABASE TABLES:
INVENTORY:
id | warehouse_id | item_id | quantity_in | quantity_out | created_at | person | shipment_id | notes

STOCK:
id | warehouse_id | item_id | stock_status_id | quantity

(SIDE NOTE: Stock_status_id is just something we can override by hand : if a musician tells us their CD won’t be arriving for 3 more months, we’ll set it to a DELAYED status. If it’s permanently out of stock, we’ll set it to PERMANENT status. Etc.)

Every time something adjusts the inventory, the stock table needs to be updated. So I decided to try my first PostgreSQL trigger. (See the PostgreSQL manual on PL/pgSQL).


CREATE OR REPLACE FUNCTION update_stock() RETURNS trigger AS '
DECLARE
w integer;
i integer;
instock integer;
BEGIN
IF TG_OP = ''DELETE'' THEN
w := OLD.warehouse_id;
i := OLD.item_id;
ELSE
w := NEW.warehouse_id;
i := NEW.item_id;
END IF;
SELECT INTO instock (COALESCE(SUM(quantity_in), 0) - COALESCE(SUM(quantity_out), 0)) FROM inventory WHERE item_id=i AND warehouse_id=w;
IF instock IS NULL THEN
DELETE FROM stock WHERE item_id=i AND warehouse_id=w;
ELSE
UPDATE stock SET quantity=instock WHERE item_id=i AND warehouse_id=w;
IF NOT FOUND THEN
INSERT INTO stock (warehouse_id, item_id, quantity) VALUES (w, i, instock);
END IF;
END IF;
RETURN NEW;
END;
' LANGUAGE plpgsql;

Everything before the first SELECT statement is just some basic setup stuff: you have to use the DECLARE section to first say what variables you’re going to be using below.
Then I had to start my BEGIN section by changing how I got the warehouse_id and item_id based on whether I had just done a DELETE on the inventory table (use OLD), or an INSERT/UPDATE (use NEW).
Then the basic stuff begins:
Subtract the total-out from the total-in for this item for this warehouse.
If it’s not found, then make sure stock table is not caching old data (DELETE).
Update the stock with the new current quantity.
Or if this is the first time, insert it.
That’s it!

Then, to have the database use this function automatically, I just added this to my database definition:

CREATE TRIGGER stock_update AFTER INSERT OR UPDATE OR DELETE ON inventory FOR EACH ROW EXECUTE PROCEDURE update_stock();


Now any time I make ANY adjustment to a line the INVENTORY table (insert, update, delete), the STOCK table is instantly updated for that item in that warehouse. It’s instant and wonderful.

Any improvements or suggestions are welcome

A friend and I were talking about blogs today, and started debating which has had more impact on journalism: cable news or blogs. We disagreed.

We weren’t talking about specific stories or events. We argued about the effect on new-gathering, ethics, and other journalistic concerns.

I won’t tell you who picked which side just yet. I want to hear what other people think. There’s a beer riding on this, so choose wisely. :)

I just got back from 10 days in Japan to check out the independent music scene there. Here, I ‘ll write up what I found in case it’s of use to anyone else.

I’ve been curious about Japan, because CD Baby’s biggest customers are there. Though we have only about 5000 customers in Japan today, they’ve spent over $1 million in CDs at CD Baby.

But we have very few musicians from Japan, probably because of language differences and the problem of mailing a box of CDs from Japan off to America. I had been wondering if it’d be wise to set up a local representative : a point-person there to be “CD Baby Japan” so that the musicians of Japan could talk to someone Japanese to ask questions, mail their box of CDs locally, have them shipped to fans directly, and also as a remote warehouse for our top-sellers in Japan.

I had been wondering so much about this that I decided to check out the price of visiting, and it was only $475 round trip airfare! Yes, the hotels and trains would be expensive, but all-in-all, worth a trip. So I booked it.

I emailed our top 20 customers (some of which had spent over $50,000). Most of them were resellers, running CD shops in Japan, so I thought they’d be perfect to ask for help getting to know the scene. (Soon you’ll see, though, that this might have been unwise, as it skewed the perspective.)

I’ll break down my interesting findings by subject. PLEASE NOTE that everything said here is NOT definitive fact, but just what I saw and heard from the dozen people I met with last week (April 2005). It wasn’t the most well-rounded perspective, so consider this to be just one point of view.

*** E-COMMERCE ***

There’s not much buying online. Though many people have credit cards (mostly ATM-style debit cards) they’re generally not trusted, and so the whole e-commerce scene in Japan now in 2005 feels like America in 1996. Early-adopters doing it, but the majority are suspicious or just not interested.

As recently as 15 years ago, the huge Tower Records in downtown Tokyo didn’t take credit cards. Now they do but it’s still only 10% of transactions. Cash is still 90%.

If people do buy something online or mail-order, it’s usually paid for C.O.D. (”collect on delivery”) - paying cash to the UPS-style delivery person. You can also pay this person by credit card, and some do : the ones that distrust entering their card number online, but don’t mind handing it to a trusted person to process.

Another popular payment form is the convenience store! Every local 7-Eleven (and all similar) convenience store has a full range of services including having your packages delivered there to be paid for and picked up in person.

Bank-transfers are also common, but it costs about $4 per transfer, so is mainly used for bigger transactions, not buying a CD.

PayPal has an unfortunate history here. Seems a competing bank bought the exclusive license, but then ran it to the ground so it wouldn’t compete with their own similar system. But since it only works with other customers of that same bank, it never took off. So the whole PayPal phenomenon seems dead in the water, here.

Cell phones are massively popular. Everyone has one. In fact, because of the sophistication of the phones (sending text-messages is very popular), and the disinterest in “web browser and a credit card” e-commerce of America, it seems the e-commerce style that will catch on in Japan will be direct-to-cellphone. I’m not talking about teens, by the way: I saw quite a few over-50 women and men spending their entire train ride typing on their cellphone : either sending messages or playing games. It is considered rude, though, to talk on your phone in indoor public places, and most people honor that. For as many people that have cellphones, I hardly ever saw someone talking on it in restaurants or trains.

It’s not so popular to have a home computer! Probably only half as many people have a computer as have a cellphone.

*** BUYING MUSIC ***

CDs cost $22 - $25 and are the main way to buy music in Japan.

There’s a regulated price-fixing for Japanese CDs: when a CD is created the price is actually named on the CD itself! The “2350 Yen” (or whatever) price is actually written right in that inner ring of data on every CD created in Japan. So if a CD is named as 2350 Yen, it is exactly 2350 Yen at Tower, 2350 Yen at HMV, and 2350 Yen at any little store on the corner. This price-fixing is seen as good for small shops, so they can fairly compete with the big giant shops.

Imported items, though, are free to be priced what at whatever they want. They usually sell for the same $22-$25 range, though.

People aren’t as price-sensitive, and there’s even a feeling that prices too low are just stupid & pointless, since they’re OK with paying more. Might even be a cultural bias so that if a CD is selling for $10-$15 it’s assumed there’s something wrong with it.

Apple iTunes hasn’t launched in Japan yet, though people say they’re planning to someday soon. The e-commerce differences make make it a tough sell, though.

There’s only one iTunes-style download music store, but it’s created by the majors and only puts its top hits for download. Some think they do it this way because though they know the digital-download future is inevitable, they’re trying to avoid it. Since people even spend $10 on a CD-single in Japan, so the idea of the $1 single is threatening.

Online streaming of music, even 30-second previews, are basically not allowed in Japan. Only label can license it, and labels don’t license it.

95% of the top-40 albums are Japanese : only a few like Mariah Carey, J-Lo, Avril Levigne make the top charts. Even U2 and Jack Johnson don’t.

The two biggest threats to the music industry are free downloading (like Kazaa), and CD-renting(!). You can *rent* a CD, take it home, copy it, and return it. CD-rental shops are very common (and legal).

By far, the most popular music to import from America are black artists. Old classic R&B Soul, gospel, modern rap, as long as it’s black. There’s a real fascination with black Americans. Because of this, the artist needs to be on the cover, or it won’t sell. (And as you’ll see below, it has to be a real professionally manufactured CD, not a CD-R.)

*** MUSICIAN SCENE ***

This was a harder one to figure out, since most of my meetings and introductions were with top CD Baby customers, they are importers more than exporters. They know a lot about bringing in American music to Japan, but not as much about Japanese musicians. (Often none at all!) I did meet with a few people on the music-making side, though. (Producer, promoter, musician.)

A strange thing came up many times, when I was asking about independent musicians in Japan: this feeling that anyone not on a record label is a bad amateur. Some said that the labels are so actively looking for anyone good, and signing them to small development deals, that it’s felt that as a musician in Japan, if you’re not on any record label you must not be any good.

There are quite a few distributors that will carry “almost any” album that anyone puts out. But most just list it in their catalog, and don’t actually put it in stores. Some stores (even one big chain: DiskUnion) will carry independent albums, even working directly with musicians, on consignment.

One person told me that this is how record deals work in Japan:
- label likes a band and signs them
- label assigns them a management company
- label pays a *salary* (!!) to the band, and to the management company
- management company gets a much larger percentage than the artist
- artists get almost nothing for music sold or shows
… therefore : a record deal is actually a very good deal for a band that does NOT sell, because you can end up with a few years salary, even without selling anything. But if you sell a lot, it’s a pretty bad deal, since your success is not rewarded as directly.

I didn’t hear any of the venomous hate of record labels, though, like we do in America.

A couple bands, recently, sold 100,000 CDs in Japan as an indie. This has excited and encouraged people (like Ani DiFranco in America).

When I asked about where the musicians gather, (popular musician magazines, websites, email lists, directories), they were stumped. Though there’s one annual directory (a la Indie Bible or Yellow Pages of Rock) - nothing else came up. I really kept pressing for this, and still got nothing. Maybe there aren’t any, or I might have just asked the wrong people.

The only thing that did come up when I was asking about the “musician scene” was one small neighborhood in Tokyo where all the little “indie rock” record stores are. They say all the musicians hang out there, too. Might just be a rocker-thing, though. (Like Sunset Strip was in the 80’s.)

*** CDs ***

The topic of CD-Rs kept coming up in all my conversations. Apparently it’s a real issue, in Japan.

CD-Rs are seen as *dangerous* - that they might BREAK your CD player!! I thought this must be total misunderstanding or myth, but found that there’s some truth to it: that the oldest, earliest CD players, when trying to play CD-Rs, would malfunction, and sometimes never work again. Very strange.

Because of this, though, CD players are now marketed with “CD-R compatible” and people are actually aware of this.

This came up a lot, because when talking to the importers/resellers, they were SO upset whenever they buy something from CD Baby and discover it’s a CD-R. It means they can’t sell it.

All this being said, someone said he suspected the whole CD-R fear was a conspiracy manufacured by the labels who felt that a CD-R revolution would damage their $25-per-CD business.

CD-Rs are given away by musicians for free at their shows. Only printed and manufactured audio CDs are seen as “real”.

It’s just as cheap to manufacture 500 CDs as 1000 CDs. Most do even less, without penalty. In fact once you do press up 500-1000 CDs with a manufacturer, you’re free to order re-runs as small as even 100 copies, for the exact same price-per-disc as you paid for the initial 1000-CD run.

*** - ***

In America, the music scene is all a-buzz about Apple iTunes, Rhapsody/Napster subscriptions, and the digital future of music. Because it’s so covered in the press, I know many people think there’s NO scene for CDs anymore, that it’s ALL digital.

It really surprised me that Japan, who most people consider to be the most technological-advanced nation, has a music scene that is almost entirely based around paying $25 per CD in cash to physical stores. That the idea of buying music online has been decidedly shunned.

Does the music scene in Japan need breaking and replacing? Even to ask feels arrogant.
Perhaps this music revolution that everyone is talking about only applies to America?
Maybe Japan will skip this whole transition stage we’re in and leapfrog into something much more advanced?

I didn’t go there to make any decisions. Just to listen, look, and learn. So I won’t end this with any conclusion.

Related link: http://www.onpointradio.org/shows/2005/04/20050426_a_main.asp

The NPR radio show
On Point
had an in-depth discussion this evening of the lagging adoption of broadband in the U.S., which is certainly increasing but not at a rate matching advanced Asian economies. One caller raised a formalistic and rigid version of standard free-market economic arguments: if there is slow growth in broadband, it must be because there aren’t that many people who want it. Where, he asked, is the demand?

In a situation like this where oligopolies in the local loop use political and market muscle to hold back competition, one has to look for other signs of the need. For instance, the rural areas of this country are emptying out. Even many cities are doing poorly as population piles up in a few megalopoli, particularly along the coasts.

This has all kinds of negative social ramifications: a crisis in affordable housing, increasing ecological damage and traffic snarls, exposure to flooding, and so on.

Basically, people are leaving the rural areas and the middle of the country because they can’t get jobs. They also find themselves disadvantaged when it comes to educational opportunities and other amenities. High-speed Internet access, with opportunities for telecommuting, distance education, medical videoconferencing, and other modern applications, can help restore a healthy balance to the country.

In short, demand is masked by flight.

The show was quite valuable in its discussion of the suppression of competition in last-mile access. The baby Bells squashed the hundreds of small Internet providers that tried to get a foothold in local markets in the 1990s and then told the FCC (with the desired results) that competition would be aided by having less competition–that is, that the FCC should let the Bells and cable companies duke it out without harrassment from small innovators.

Now, as mentioned on the radio show, the telecom companies and cable companies are using the same argument to hold back municipal networks: supposedly, holding back competition is good for competition. The irony is that municipalities step in to take on the big job of building out a network only when the private companies have stayed away. And a government-run fiber network can lay the groundwork for competition at higher layers.

Let’s have some real competition, and then the hidden demand will reveal itself.

Related link: http://xmldb-org.sourceforge.net/xupdate/

XUpdate is a product of the XML:DB group, but it is designed for use far beyond XML databases. It’s a lot like XSLT, being an XML-based host language for XPath expressions, but XUpdate instructions are tailored for update tasks. Doing the equivalent in XSLT is rather clumsy, which is to be expected since XUpdate is specialized for the purpose. I’m a big fan of XSLT, as my body of work amply proves, but I prefer XUpdate for what it does best.

XUpdate has always been a shoestring community standard, and has always been skeletal, and somewhat incomplete. Despite this fact, it has over a dozen implementations (half of which can be used independently of any XML databases). This relative success is because it is simple to understand and simple to implement. XUpdate is a poster child for worse-is-better.

There is a low-volume mailing list, which used to be riddled with spam, but has since been brought under control by Per Nyfelt. I think a lot of XUpdate discussion has been directed more towards implementation mailing lists rather than the general spec list. Certainly, we get a lot of questions about 4Suite’s XUpdate implementation. There is also a very useful use cases document by Kimbro Staken. The main problem with this document is that it includes some controversial use cases that are not included in the actual spec. This is one of the things that will, I hope, be cleared up soon. I also hope the many links to outdated XUpdate spec locations will be fixed.

As a meter of XUpdate’s continuing relevance, and as a service to interested users, I’ve compiled a list of projects that include separate XUpdate implementations.

• XML:DB XUpdate–reference XUpdate implementation in Java
• 4Suite–XUpdate available via Python API, command line or XML repository
• xmldiff–Python tool to generate XUpdate “diffs” between XML files
• RxUpdate–An “enhanced” XUpdate implementation in Python, including RDF support
• Apache Xindice–XML DBMS in Java. “At the present time Xindice uses XPath for its query language and XML:DB XUpdate for its update language.”
• eXist–XML DBMS in Java.
• Ozone–XML DBMS in Java.
• X-Hive/DB–XML DBMS in Java. See the XUpdate page
• dbXML–XML DBMS in Java. “XUpdate is also a transformation with some of the same goals as XSLT, but its syntax is simpler, and its purpose is to modify the content of documents in place.”
• Orbeon PresentationServer–full-blown XML platform thingy, in Java. See docs on the XUpdate engine and lower-level processor. The latter link includes a useful intro to XUpdate.
• Jaxup–”A Java XML Update engine”
• Mobius Mako Command Line Utilities–Mobius is a Grid technology project. Mako is “a service that exposes and abstracts data resources as XML”, supporting XUpdate.
• Montag–”a Java Web Services based system for the interaction with every Native XML Database that supplies a Java implementation of the XML:DB API.” Includes XUpdate support.
• XML-XUpdate-LibXML–Perl implementation

If you know of any I haven’t mentioned, please point them out in comments. Overall, I hope you at least have a look at XUpdate for relevant tasks, and better yet, become involved. It has about as low a barrier to entry one could possibly imagine.

Do you use XUpdate? Or do you really not accept its relevance?

Related link: http://services.google.com/ads_inquiry/sitetarget?hl=en

In a further departure from its roots in searching, Google has announced a new program that will allow advertisers to choose sites for target ads.

I’ve written in the past about Google’s transformation (at least looking at revenue) from a search company to an advertising broker. But contextual advertising - Google’s other-than-search bread-and-butter - still involves technology that automatically caluclates relevancy, just like a searching algorithm, and produces a marketplace for words. Whether the context is evaluated correctly or not by the automated mechanism is another story (analagous to questions of how well searching works).

In the new Google order of things, advertisers interested in branding can pick their sites without regard for contextual relevancy. The New York Times bills the changes as a move away from search for Google, and Google-commentator Brad Hill in his blog calls the move “industry shaking.”

Advertisers will pay for the new-style ads on a CPM basis, or per ad impression (not per ad click as with contextual ads), although the process of purchasing these ads will be blended with the traditional Google CPC (pay per click) word auction process.

These ads are intended to appeal to big advertisers who are looking for general branding (for example, all kinds of advertisers of luxury goods would probably like to appear on BMW’s site, even if the ads were not contextually relevant to cars).

Context-free ads may also work for advertisers who are better able to determine relevance than the automated algorithms - it makes sense to put ads for cheese on a oenophile site, but AdSense probably doesn’t think so. Google’s revenue stream will be a winner, as will big advertisers and owners of desirable Web content. Possible losers: anybody but Google in the business of brokering ads.

Related link: http://www.cs.tufts.edu/~mchow/excollege

Hard to believe, I am almost finished with teaching a full college course (one semester) –my course at Tufts University entitled “Security, Privacy, and Politics in the Computer Age,” offered by the Experimental College. It has certainly been an exhilerating few months, but it has been a very rewarding, memorable, and flattering experience.

So what did I learn from teaching computer security, politics, and privacy to a group of twenty, mainly non-technical, college students? Here are some of my thoughts in a nutshell:

• It is difficult to balance technical and non-technical information. Many students know what spyware and computer viruses are, but the technical workings of them are complicated. If you delve into complexities such as the operating system or the kernel, the students will be lost. I also recall making my cryptography lecture too simplistic, and I saw many students fall asleep.

• Students are dependent on reactive tools including firewalls and anti-virus software. Such tools have been well-marketed, but they can only do so much. That is, the “bigger point” is missed –numerous security holes in software are unpublicized, which leads to one massive hole. The message that I sent to the class was clear: the first line of defense is to protect yourself and your systems (be proactive as possible). Funny, I still receive assignments that mention relying on firewalls and anti-virus software to protect their systems.

• Few have knowledge about open source software, and alternatives to popular software packages. It is important to discuss the software life-cycle development process early in the semester because it will provide students insights on where a lot of the problems come from. One of the first comments from students that stuck me was that many have never heard of open source software, nor have they heard of alternatives to popular software packages such as GIMP, GAIM, and yes, even Firefox. As much as the technical community read and speak about OSS, the general public still don’t understand it.

• Few have used Unix or Linux. Unix and Linux are sometimes dubbed as the “the most important operating systems you may never use,” and I found this quite true. That is why I distributed free copies of Knoppix to students, and used it for my lectures on occasion.

• News and information evolve and change frequently. Several weeks after I gave a demonstration on password cracking, the news of Paris Hilton’s sidekick cracked via simple password broke out. We had to reflect back on our previous lecture. Same issue with the recent slew of consumer database breaches. The instructor (myself) have to keep up with current events especially when teaching such a course.

• Students enjoy examples. Students love screenshots and hands-on examples from the terminal.

• Instructor has to encourage feedback and dialog. Maybe it is because of the college environment, most of us have been there, done that. I found that students walk into class with very little expectation or motivation each day. They just want to go to class and leave, and probably forget the information. It is the instructor’s job to incorporate debate and dialog in the course. You just can’t hope that all students will be active. I had two debates and two expert panel sessions in the class, and they have been most engaging (as said by the students). Same goes for the discussions on copyrights, electronic voting, and P2P technologies –no surprise considering the topics are controversial and debateable.

• Need a hands-on assignment to show how hard security is. Security is hard, we know that. But talk can only do so much. Recently, I gave a two-part group project on designing a fictitious state lottery game and its secure system. Not only did the students find that designing a system is difficult and time-consuming, but also how hard it is the accomodate for everything there is. I had to use so much red ink on grading the design projects, both phases (the game design and the system design)

These are just some highlights of what I learned in my very first teaching experience. After I submit the course grades, I will sit down and collect all my thoughts about the course. Would I want to do this again? Absolutely, in a heartbeat.

Related link: http://www.spacedaily.com/news/spacetravel-05v.html

This amazing, detailed article on Space Daily shows how NASA’s policy, before and after the manned lunar landings, has gone from brilliant to bumbling. The message is that NASA critically needs to re-engage the mission-oriented planning and direction of the Apollo era, if humanity is ever going to get back to the Moon, and then to Mars, and then back, in one piece. (Thanks, Anselm!)

Statistically Improbable Phrases (a/k/a “SIP”) is the improbable term Amazon.com uses as a search ranking technique. Here’s Amazon’s explanation.

In more-or-less plain English, here’s how this works. Amazon indexes the “Search Inside” content of the books in its catalog (that is, the books in which publishers provide this content). In many cases, Amazon provides a list of SIPs on the main listing page for the title. For example, Starting an Online Business for Dummies by Greg Holden has a number of linked SIPs listed, including “your online business.” These SIPs are phrases that appear with anomalous frequency in the inside content of the cataloged book compared with the entire the rate of occurence of the SIP in the universe of books in general. This statistic over-occurence implies that the SIP is a significant representation of the content of the book.

By clicking one of the SIP links, you get other books in which the SIP occurs, sorted from most to least by the number of SIP references. For example, “Web Analytics” and “E-Commerce for Dummies” have the next highest occurences of the SIP “your online business” after “Starting an Online Business for Dummies.”

This is a different and somewhat appealing way to use Amazon’s search facilities to find books in which the author uses distinctive phrases. Longer run, the concept has an elegant simplicity (as did the original PageRank algorithm), and may be useful for automated tagging and ranking of content.

Click here for a lively discussion of SIPs in the context of author as phrase maker, and here’s a fun discussion and list of adult SIPs on Amazon (over 18 only please click this link).

Every week the developers of the O’Reilly Network (that’s me, three
developers, and two admins) have a status meeting to check in with our key
managers, decide or rearrange priorities, and work through problems. This is a
dream meeting for managers, questions are answered and plans are laid.

It’s fair to say that status meetings aren’t a developer’s dream. After
several years of weekly meetings, ours were feeling stale. So we agreed to end
the meetings with a round of tips and tricks. First up, bash tricks.

At the end of our meeting, the managers bailed and we stuck around to geek out
over the tricks that make our work easier. Everyone had something to
contribute. I can’t recommend this enough! I’ve learned a lot from books like
Unix Power Tools. But by
sharing directly with your coworkers you get advice that’s targeted directly
to the work you do.

Here were our gems, the most useful tips that weren’t already common
knowledge among the developers.

pushd/popd

Bash will keep a history of the directories you visit, you just have to ask.
Bash stores the history in a stack and uses the commands pushd
and popd to manage the stack.

pushd dir - move the current directory onto the stack and change to
the dir directory.

popd - pops the top directory off of the stack and moves you into it.

We’re opening files all over the file system, internal code, vendor code, templates, configuration files, logs. Because of this we like the ability to take a detour on the file system and still navigate back to our working directory of the day. I think these commands are so useful that I alias’d them in my .bashrc

alias cd="pushd"
alias bd="popd"

Now the cd command manages the stack for me as well as changing directories. Aliasing popd to bd is an easy to remember and easy to type way to move back up the stack, think “change dir” and “back dir”.

History

Bash keeps a history of the commands you’ve run. My group was already comfortable with the up and down arrows to navigate the history, !! to repeat the last command, and !foo to repeat the last command starting with foo.

Our newest admin had a better way, CTRL-R. That does command auto completion. Repeatedly pressing CTRL-R lets you tab through matching commands.

Home/End

CTRL-A takes you to the beginning of the line and CTRL-E takes you to the end of the line. This is probably basic shell knowledge, but I’m probably (hopefully) not the only person who didn’t know it.

For Loops

We’ve got a cluster of machines that we’ll sometimes need to loop through. Here’s an example from our admins that checks uptime across our cluster.

$ for s in `cat server.list`; do ssh $s uptime; done;

Working with the Previous Command

Sometimes you want to run several commands on the same file, like run ls before deciding if that’s the file you want to edit.

ls -l /long/path/to/file.txt
vi /long/path/to/file.txt

Bash provides a shortcut (!$) that holds the last word from the previous command. So in the above you could just write vi !$.

If your last command had a typo you can fix the command and rerun it with this construct, ^foo^bar. That replaces the first occurrence of foo in your command with bar.

Bonus Tip: use Awk

Our admins seem to think awk is pretty useful. And my boss thought it was
so useful, he wrote a book
on it. I can’t keep any of the awk syntax in my head beyond printing out a
column from a file.

The normal column delimiter is whitespace. So if you wanted to print out the seventh column in an Apache access log (that’s the request url in my logs) you could write:

cat access_log | awk '{print $7}'

You can change the delimiter with -F. So if you wanted to list all the users on your system, you could pull them out of /etc/passwd with:

$ cat /etc/passwd | awk -F: '{print $1}'

The /etc/passwd delimiter is :, which I’ve indicated
to awk with -F:.

We’re doing vim tricks next.

What’s your favorite bash trick? How else do you share tricks among developers?

Related link: http://www.google.com/searchhistory/

Google has a new feature that tracks your search history. (Click the link to open the sign-up page for the application, which otherwise can be accessed through Google Labs.) This is another one of Google’s wonderful tools that is a “beta” that is not really a beta.

So far, the functionality is pretty straightforward and (at least for me) very useful. When you are logged in, and you can log in of course from any computer, Google keeps track of your searches. You can click on any of the links that represent a saved search to see the full text of a search. You can also retrieve searches by date using the calendar that the Search History Tool provides.

Once you sign up for the Search History Tool, your Google home page changes. Up on the right-hand top, you’ll see your sign-in email, a link that takes you to your account history (which is where to find the calendar and search links, and also the ability to remove any or all search items), a link that takes you to your Google account settings, and a link to sign out. If you do sign out, Google’s home page will show you a sign-in link.

Keeping track of my search history is a very useful feature for me. I can’t tell you how many Google searches I do a day (probably in the three or four digits), although the Seach History Tool will in fact tell me this. Many times, I’ve “lost” information from a search that I thought I didn’t need (but actually did!) The Search History Tool will pretty much solve this problem for me, I think.

Down the road, the Search History Tool will probably let Google refine searches for me based on my search history (it remains to be seen how helpful this is).

The Search History Tool may allow customization that is an important weapon in the battle against search spam, because I may be able to “train” my future searches by deploying a “Junk” setting against my Search History results. Other forms of search customization, once I’m logged in to search, are also possible of course.

I also see the Search History Tool as a Trojan horse for the introduction of more Yahoo-like services. Google needs to know its users better to create these services: and what better way to know someone than to keep track of their searches?

Related link: http://www.mysqluc.com

On the last day of the

2005 MySQL conference,
I finally heard a speaker who stretched the audience’s assumptions and
pointed toward a liberating path forward. This is the sign of a good
conference, incidentally–most of the sessions deal intensively with
the problems of today, but one or two keynotes prepare the listeners
for tomorrow.

I wrote in

my earlier weblog about this conference
that MySQL was becoming conventional. Many people are doing innovative
things with it–I sat in today, for instance, on a session about MySQL
as an embedded server or library–but the largest attendance has been
reserved for traditional topics such as replication and performance
tuning. MySQL AB itself is concerned with catching up to its
competitors in terms of SQL features that centralize more and more
control in the database engine.

Adam Bosworth, in his keynote today, threw all that out and set his
ship headed in a different direction. The problem he found with
centralizing processing–with stored procedures and triggers and so
forth–is that it doesn’t scale. His talk also implied that it
restricts users from making innovative connections. Google, his most
recent landing place during Bosworth’s long and impressive career,
illustrates an entirely different way to handle data.

Adam Bosworth’s view of an open data query protocol

The promise of the Web was to aggregate the contributions of
individuals everywhere and make retrieval easy along any lines one
chose to use. As the volume of content became unmanageable, XQuery was
supposed to provide a Web-aware search mechanism, and Web Services the
infrastructure and protocols to connect sites. XQuery and Web Services
were too big and came too late, however. Nobody actually wants to use
them, even if they know how.

So the gap has been filled with RSS, the model highlighted by Bosworth
for the next stage in search. RSS and Atom are lightweight and easy to
understand. The put control in the hands of the content providers and
the potential viewers.

Bosworth’s extended vision is for a protocol that provides raw access
to data, somewhat as XQuery is supposed to do. It would be a very
simple and database-independent protocol that would make all data in
the world open. Then, he says, everybody could do what Google
does. And more–we could provide distributed updates too.

Where to impose structure

The Google approach to data, carried through in Bosworth’s vision,
runs head-on up against the ideals of the relational database model.
The entire relational approach, from the canon of Third Normal Form
(three is a holy number) to the enormously complex collection of
analytic functions, subqueries, and other ways to impose structure in
SQL, is an attempt to be as precise as possible about the data chosen
and returned.

Bosworth isn’t interested in that. If the user gets a few hundred
results and has to scroll through them a little bit, that’s fine. We
don’t need no stinkin’ metadata or knowledge management.

The philosophical debate underlying relational database design

Bosworth evoked earlier debates that I’ve found valuable and aired
several concerns of mine; his views of the XML specs and RSS/Atom are
familiar. But his brief critique of the trend toward putting more and
more features into the database engine–a critique that he whisked
through on the way to grander visions–left open a question about the
basic philosophy of SQL.

When MySQL was bare-bones and lightweight (which it still is compared
to commercial database management systems or PostgreSQL), it put
responsibility in the hands of the application programmer. If a value
was supposed to be limited to a particular range or two columns were
supposed to be entered in tandem, it was the application programmer
that made sure of it.

In contrast, traditional database design takes as much control away
from the application as possible and puts it in the database. A
constraint or trigger or stored procedure or foreign key can make sure
that no one gives someone an absurdly high salary or fires an employee
while leaving his phone number in the database.

This centralized control is a relic of the 1970s, when corporate staff
would sit at command-line processors and type in SQL to do what they
wanted. Nowadays, when an application and even a Web interface stand
between the user and the database engine, the never-trust-the-user
philosophy is less valid. At the very least, an application has to
know the rules the database is enforcing and translate error messages
into something the user can understand. The wall between application
and database engine is porous, so the application can take on more of
the validation and logic.

But both philosophies are valid, and now MySQL offers a choice. I
suggested to Arjen Lentz, the organizer of this year’s conference,
that he offer a debate next year between the application-aware
philosophy and the database-aware philosophy–when is each
appropriate?

Most of us still need to find that phone number for an employee and do
other everyday tasks; we’ll be using a relational database for that,
and MySQL will be providing that service for more and more sites. The
people with day jobs who came this year to find out whether MySQL
could bring home the bacon got their answers. But MySQL can also
support fun applications, and I hope to see more coolness next year.

I went to a taping of The Daily Show yesterday. In the small studio, I saw a blueberry iMac at the sound station, an orange iMac behind the scenes (literally!), and what I think was the top of a G4 Tower.

I saw a bunch of other monitors hooked up to various hardware things, but I couldn’t tell what they were running.

No big whoop.

In recent years, the desktop, be it commercial or not, has evolved into a network aware base for running applications. Although the desktop has been through an extensive round of spit-sheening, largely driven by orange-sunglasses wearing usability engineers, the desktop is still largely un-integrated. Sure, you can take content from one program and embed it another program; a kind of glorified cut and paste, but the applications still don’t integrate together in ways that really benefit the user.

To understand how to design for proper integration, you need to first explore what people actually use their computers for. Aside from recreational use, the majority of businesses users, and those who actually work on their computers all utilise them within the concept of a project. Within this context, you find users who mentally hook together different applications with the intention of satisfying criteria to achieve a project or goal. This can be demonstrated with a simple use case.

Imagine that John Smith works as a consultant. John needs to interact with a variety of software tools:

• Customer Relationship Management (CRM) system
• Email client
• Private and public shared calendar
• PDA that is synced to the calendar
• Website
• Office suite

This is the way that John will typically bring these tools together when working on a project:

1. First John speaks to the client on the phone and then manually logs the client and call in the CRM and logs a meeting on his PDA. He then syncs his PDA to his calendar in Evolution. He also adds the client contact details to the PDA and syncs those with the main address database in the office.
2. Next, John needs to fill in some forms for the paperwork. He gets the CRM details for the client and creates a document in OpenOffice.org. He meticulously copies the data over that he stored in the CRM and prints it out.
3. To prepare for the meeting, John wants to meet with his colleagues to discuss some aspects of the job. He discusses some things with one colleague over the water dispenser, but another is working from home. He cannot see him in Gaim, so he sends him an email and they have a lengthy (in time) email conversation.
4. John now cuts and pastes the discussion from the emails into the CRM to better prepare for the meeting.
5. John tries to contact the client to confirm the meeting but cannot get through. The client calls up on John’s office VoIP phone and the phone emails him the answermachine message. John listens to the message and copies the details into the CRM manually.
6. The meeting happens and John makes notes on his PDA which are then copied manually into the CRM as well as some other emails with the client.

This use case is a typical example of a number of different systems working together on the same project but not actually integrating. The result of this lack of integration is that there is a lot of manually copying and pasting between different systems, particularly into the CRM; a system that most people find difficult to keep updated.

In reality, the above use case is not actually realistic. In our busy working lives, it is difficult and practically impossible to make use of all of these systems and keep them up to data manually. Each of these systems relies on John remembering to update them with the right information, and as anyone who works in a busy office will know, it is far too easy to get sidetracked by fellow staff members, other projects, websites, other emails, other IM conversations and lets not forget the surprise visitors who want to bend your ear for an hour or so.

There is clearly a problem here. Most of us will use a variety of software tools in one way or another to achieve a combined goal, but these tools cannot talk to each other effectively - they cannot integrate, and this wastes both time and effort. The source of the problem is that the desktop integrates together at a software level, but not a task level. Integrating these applications does not just mean sharing data between them, but it also means pro-actively adjusting the user experience of these tools in favor of a project.

The solution

The solution to many of these problems is to adjust the desktop experience so that it supports Projects at the base level. This would involve creating a simple to use Project Manager tool and including support in a range of other applications to automatically update and integrate with this tool. You can think of the Project Manager tool as a means to simply help different tools to talk to each other and to provide a central place in which the Project is managed. This Project Manager tool would also provide a clear summary of the project, who is working on what and provide access to all files relevant to the project; all of this being useful for reporting an auditing procedures.

Imagine this case study for John’s situation:

1. John speaks to the client on the phone about the work that he client wants doing.
2. John now logs into the Project tool on his computer and creates a new project. He adds some details about the project in the Meeting box he selects the date for the first meeting. When he saves the project, the software will update Evolution with the meeting which in turns syncs with his PDA. The information about the project is also added to the CRM and the call and meeting are added as CRM activities. The software also generates an OpenOffice.org document with the relevant information added and adds the document to the project file store.
3. John now decides that he is going to work on the project. In the top right hand side of his GNOME desktop he clicks on the project icon and a drop down list of his current projects are displayed. He selects the right project and now his desktop has been adjusted to reflect those projects - the right bookmarks are loaded in Firefox, the right contacts (and only the right contacts) appear in Gaim, emails from the right people appear in Evolution and Nautilus is familiar with the files involved in the project, accessible from the My Project Files icon on the desktop.
4. John clicks on My Project Files and clicks on the OpenOffice.org document to update some of the details. When he saves the file, this action is stored in the CRM and the updates are possibly mailed to his colleagues automatically with a list of the changes.
5. John checks his mail in Evolution and sees that he has received an email from the client. When the mail was received by Evolution, it automatically noted it in the CRM and a message pops up hovering over the project icon informing that a new project related email has been received.
6. John decides to send a new email and selects on the people involved in the project from his now more limited address book (the address book is more limited when he selects to the project to find contacts quicker).
7. As John is working on some parts of the project, he sees interesting web pages in Firefox that he bookmarks in the project. As he works, the client pops up in Gaim. Gaim has automatically adjusted itself to only include buddies within the current project so John does not get distracted and ignore Gaim totally. He has the conversation and then the log of the conversation is added to the CRM automatically.
8. John goes to the meeting, makes notes on his PDA and get back and syncs the PDA. The notes are automatically added to the CRM and if a second meeting was arranged, Evolution would be updated with new time and the CRM would be notified of the new meeting as well as any colleagues that are required (Evolution would mail them).
9. At the end of the week John is working on his timesheets and most of them have been automatically filled in by the Project Manager tool with detailed times of then he was working on all these parts of the project. He only needs to fill in the gaps.
10. As John finishes the project, he is asked by his boss for report. He clicks a single button in the Project Manager and a PDF report is generated automatically for him with a detailed breakdown of what he has worked on, how much staff labor was involved, the labor costs (calculated by referencing the time spent on the project and John’s average hourly wage), the goals achieved, the steps involved and more. A few other colleagues have asked to see the report so John can add them to the report mail-out or generate an online report that is automatically uploaded and then mails interested parties of the report.

With this use case, you can see how much of the leg work is automated by the systems. This case specifically improves on the old one in that the logging of each event in the CRM is automated by the software and John does not need to remember to log in and make the updates himself. In addition to this, each application within his desktop is adjusted to reflect the resources that are part of the current project. This is particularly useful when it comes to communication.

Better communication with the integrated desktop

One of the problems with communications tools is that they are notorious for sidetracking you. Possibly the largest offender is an IM client such as Gaim. When you log on to IM, it is likely that you are looking to speak to someone in particular, or you may be specifically interested in speaking to a particular group of people. As an example, I have a number of friends who work in IT and a number of friends who are in bands. When I am mentally in work mode and I am working on a project, I often log onto IM to ask a particular question to an IT buddy. Typically when I log on, one of the music buddies will pop up to chat to me and I feel guilty if I just ignore them. I have a short conversation and typically get sidetracked by the discussion. This not only wastes time, but it also affects my concentration. The result of this is that I tend to leave IM switched off unless I am specifically looking to be distracted or speak to non-work friends. It seems such a shame to waste an entire medium of communication just because of distractions from the wrong people I seek to use the tool to communicate with. Oh, and switching your status to busy does not alleviate this problem…

In an ideal world, Gaim (or any other IM client) will check the contacts in the current project and only advertise my online status to them and their online status to me. This will restrict IM to a tool that is useful for the project in hand, and my contacts are likely to talk to me on matters that are relevant. Some of you may think, well just organise your buddies into groups and select the right group. The problem with this is the same old issue affecting the current desktop - I have to make the effort to adjust it. No. The tool should make the effort to adjust it. I see no reason why I should indicate to five or six tools that I am working on the same project. I should indicate my desire to work on the project once and then the Project Manager updates everything else.

Can it happen?

Now, all of this is being discussed in a perfect world where this can all be coded and works effectively. Can it actually happen? Yes, I do believe it can.

I am specifically interested in making all of this work in GNOME. There are a few reasons for this. Firstly, the GNOME project have a fairly strong integration and control over a number of different applications. I don’t mean this so much from a technical perspective, but more from a social perspective. If you read Planet GNOME or keep up with the GNOME websites, it seems that the GNOME developers seem far more in touch with each other and better integrated. This is largely due to the existence of a lot of GNOME hackers in companies such as Red Hat and Novell, but also the fact that the GNOME hackers seem to discuss and actually produce software efficiently in tight groups. Some may see this as an old-boys-club, but I think it is just good hackers working with other good hackers.

Another reason why this can happen in GNOME is because you can only make this work at an architectural level and not at an application level. If the Gaim developers buy into the idea but other tools don’t, nothing will happen. To really make this work, the foundation GNOME architecture will need to incorporate a means to talk to other project-aware applications in different ways. This will require data-aware widgets becoming project-aware, the GNOME file picker including the project in the GNOME VFS, nautilus remembering file location histories for the project, recent-files menus remembering recent project files, evolution-data-server supporting sharing of contacts all over the place and more.

A final reason why I think it can happen with GNOME is that I believe the GNOME developers are real innovators. Some of the work going on in GNOME has been quite ground breaking with work such as Beagle, Project Utopia, Sabayon, Tomboy, custom widgets in F-Spot, file picker improvements and the GNOME VFS, luminosity eye-candy, support for SVG and more. In addition to this, the GNOME project have really hooked into some of the freedesktop.org technologies such as HAL, DBUS, XGL and more. Another point is that I was quite pleased to see the GNOME developers had the balls to bravely include the spatial nautilus in GNOME. A decision that caused much controversy but also spoke legions in terms of dedication to usability.

I think that that reasonably, the technology is available to make much of this happen. A lot of the challenges can be solved with DBUS, but I also think that a deep integration in GNOME of many of these project driven principles can work. Even then, the concept of the project does not need to be the only limiting factor. In addition to a Project, you may want to incorporate other modes for integrating applications together. I am not sure what they are right now, but I am sure they can be discussed extensively on mailing lists in further detail.

I should clarify that I am neither a usability engineer or a GNOME developer, but I have a strong interest in both usability and GNOME development. I am a consultant who works every day to help businesses, charities, schools and consumers get the most out of Open Source. The problems that I mention in this article about the lack of integration are real, tangible and measurable problems that I myself, my colleagues and my clients face every day. Duplication of effort typically results in no effort with some systems, and then this effort must all be reproduced in bulk when a project is tied up. With all the approximate applications in place (email, IM, productivity, CRM etc.), there is a real potential to make them work together more effectively and provide a truly innovative reason to move to the Linux desktop. There are many other factors to discuss in the design of this integrated desktop, and I am certainly not the only person to discuss this idea, but hopefully this article can contribute to the design and the discussion.

Do you think it can work? How can it be better improved? Write your views below…

The new O’Reilly Radar site is pretty cool.
Tim O’Reilly,
Nat Torkington,
Rael Dornfest
and
Marc Hedlund
all post links and articles about cool stuff that’s on O’Reilly Media’s radar. There are charts and stats of what’s going on out in the forefront of tech.

It wasn’t until today that I got the play on words.

You have to type in radar.oreilly.com.

I’m surprised that they didn’t start off with a link to a Google Maps shot of Ottumwa, Iowa.

Related link: http://www.onlamp.com/pub/a/onlamp/2005/03/31/extreme_admin.html

Andrew Cowie (who generously wrote the article linked above) expanded his theory of dealing with complexity in operations in a talk yesterday. Here are a few of his ideas that stuck in my head.

• Programmers communicate through code. Operations people communicate through their procedures and documentation.

  If you take the idea that programming makes blueprints and compiling and deploying builds houses, the equivalent in the administrative world to programming is the design of systems and processes. (This paragraph is my idea, not necessarily Andrew’s.)

• You can’t hire a colonel, you can only grow one. Andrew pointed out that members of the military spend most of their time learning. Is systems administration any different?
• High turnover is detrimental to high trust. If your organization burns through people in months, how do you grow to trust your co-workers?

• The people really doing the work are the best ones to determine the risks.

  This is another similarity to XP, which allows developers to estimate the amount of time that each task will take. It’s up to the customer to arrange the tasks in the most desirable order, but the customer cannot change the estimates and the developers cannot change the order of the tasks. (This requires trust which requires time.)

• Observe, Reflect, Decide, Act, Learn. This is a pattern of behavior to use when encountering situations and making decisions.

That’s pretty theoretical, but there’s a lot to think about. Andrew also had lots of useful practical advice. One idea to consider is asking a friendly coworker in another department to sit in on an installation or change session. This has two advantages. First, if you grab someone from sales, marketing, or management (for example), you’ll show off the work you actually do in a more concrete way. Second, you can have him or her time and check off your procedures as you accomplish them, helping to verify your time estimates and keep you on track.

You can also send him or her out to buy doughnuts.

Want to hear more from Andrew on ONLamp.com?

This is a tale of two software companies (or maybe three).

They were the best of companies, and they were the worst of companies. And they’ve yet to go to a far, far better place to peddle their software.

To get back to my story, corporations, like people, have a lifecycle with a beginning, middle, and end.

Once upon a time, a long time ago, in a land far far away, Big Blue - IBM - was the be-all and end-all of everything to do with the computer industry. Beaurocratic, ponderous, rich, and powerful, IBM watched the nimble Mister Softee - Microsoft - steal the software side of the computer business out from under it.

Mister Softee was everything Big Blue wasn’t: a teenage rebel, improvising like crazy, able to turn on a dime, handing out stock options like candy in a bowl near the cash register of a restaurant where the food isn’t so good.

Now Mister Softy has grown as soft as its nickname and frumpy, with middle-aged love handles to match. Stock options are long gone. Microsoft pays a dividend!

Microsoft wants to climb the enterprise.

This company is no longer nimble, and takes literally years to pass software through its beaurocratic process before release.

We didn’t love Mister Softee when he was young and agile, but he always impressed us with his vigor and chutzpah (though we always wished he built better, less buggy, software).

Now Mister Softee is as rich as Croesus and out-IBMs IBM. He’s dangerous! He’s fat! He’s rich! His speed of innovation is falling way behind compared to younger rivals like Google. More than ever, he’s fun to despise.

Maybe, just, maybe, Mister Softee is also starting to become irrelevant (thanks to Open Source, Linux, the Web, and Google).

Related link: http://www.braintique.com/research/mt-archives/000147.shtml

If you haven’t tried it, the relatively new mapping capabilities at Google are very cool. I like the maps. You enter an address (or portion of one). The user interface is very sparse, with a widget in the upper left to control zooming in and out and panning across a map. Like Mapquest, you can get driving directions to or from an address. Unlike Mapquest, there are no annoying ads, pop-ups, and other distractions. You can use the Google maps to find businesses or services of a specific type in a given locale.

For reasons I can’t quite put my finger on, I think the Mapquest maps may actually be a little better for navigating by car than the Google maps. But one feature of the Google mapping application is, in fact, cool beyond belief. If you click the Satellite button on the upper right hand corner of the screen, you can see the aerial, satellite photographic view of any map. The zooming and panning tools work with these satellite pictures.

Start with where you live from above and pinpoint your block and rooftop. You can zoom in and out, see your whole city or state. Kids love this.

Some fine print: Google maps and sat photos are limited to the United States and Canada (more world coverage is promised soon). Coverage in rural areas can be spotty. This, however, corresponds rather well to the areas that are not much sought after (click here for a Google engineer’s visualization of frequency of search by locale).

More fine print: the photos seem somewhat dated (for example, big elm trees can be seen in the aerial view of my house, they came down in winter storms over two years ago). There are the usual sporadic reported glitches in the maps (this is not unique to Google’s maps).

Here’s a neat application that combines Google Maps and Craig’s List so you can view the location of Craig’s List real estate listings.

Related link: http://www.braintique.com/research/mt-archives/000126.shtml

An extremely important part of Google’s business is famously built upon contextual advertising: Advertisers bid on keywords using Google’s AdWords software, and the winners have ads placed “contextually” on web sites whose publishers have elected to affiliate with Google using Google’s AdSense software.

But “contextually” is a significant misnomer. Computers are very good at literally matching keywords, but very bad at catching the subtle nuances of context.

As a Web publisher, you find offensive ads placed by Google. For example, on Phyllis’s HighRisk.org, a site devoted to helping parents with preemies and high-risk pregnancy conditions, we get ads for thinly disguised anti-abortionists. You can deal with this one by blocking the domains in question (Google allows AdSense publishers to block up to 200 domains as “competitors.”)

It’s a little harder to deal with what turned up when I wrote a blog entry blasting intelligent design as a euphemism for creationism. Both the blog entry and my Main blog page for the month kept gettings ads from anti-evolutionists too numerous to block by domain.

Similarly, but a little funnier, when I wrote a blog entry commenting on a business press item comparing Google to Wal-Mart, and coming down hard on Wal-Mart, and another item just blasting Wal-Mart, both my blog items and