Related link: http://www.gulshatfund.org.uk

The problem - you need to produce a website … and fast

The goalposts haven’t just moved - they’ve fallen in the deep end of the swimming pool!

When I last took the time to learn how to produce the best web page possible, I stood proud in the knowledge that I’d used tables rather than frames for layout. CSS1 was unsupported by the browsers most people were running, and implemented differently by the rest. CSS2 was merely a twinkle in W3C’s eye.

So I had a lot of catching up to do when I was asked if I’d volunteer to produce a website by a friend who’s the chief fund raiser for Lily. She’s a little girl who’s come over to the UK for an operation she can’t have in her own country. And they had a TV appearance scheduled for later in the week when they wanted to give out the URL, so I had to produce something reasonably professional-looking … and fast.

Enter “Head First HTML with CSS and XHTML!”

Coincidentally, “Head First HTML with CSS & XHTML” had just arrived on my desk and what better way to test its merits? It was a little odd at first, looking at it from the point of view of learning about its content rather than learning how to write a Head First book! I guess this is a field-test kind-of review, but only from the personal angle of what was useful to someone in my position (i.e. OK with hand-coded HTML, clueless about CSS, got to produce a good website in 24h).

Personal Comments about the book

I skipped the first few chapters, which cover thing like text editors, directory structure, images, basic markup, hosting, etc - basically everything a complete beginner would need to get started on producing something. I skipped the chapter on XHTML - I didn’t require it (and neither does the rest of the book).

Once I got to the bit that was useful for me, I didn’t read the chapters in order. It was a case of flicking backwards and forwards finding the bits I wanted. For a start, divs and spans - I’d heard of them but didn’t know how to use them. The bit about the box model and the distinction between margin, padding etc was a good start and the chapter on using divs for layout was fantastic.

I flicked through the book until I found reasonable layouts then read around them to see how they were done. I typed in some of the code from the book then played about with it myself to see what would happen. Even if this hadn’t been my natural inclination, I’d have been encouraged to do this anyway by the little ‘Sharpen your pencil’ and ‘Brain power’ exercises spread throughout the book. I skipped these and made up my own, so that I only played with code that looked like it might do what I wanted.

I just took what I needed. I don’t know how to make stylesheets cascade, for example, but I’m confident that I know where to look should the need arise. It did break some bad habits - I’d never seen much of a difference between doing [br][br] and doing [p], and hadn’t seen the point of the [/p] tag. And I’m never going to use stuff like [b] again!

General Comments about the book

Something very interesting is that we learn about using CSS for layout - and quite complicated layouts - before we even hear of a table. This is brilliant for the beginner, who would never even dream of using a table for layout by the time they get to the end of the book - they’d see it as being a data only thing (and now, so do I!). It is also typical of the style - you learn about things as you need them, rather being told about too much at once and getting confused.

There are also chapters and sections that are over and above what it says on the tin (American readers should do a search for Ronseal at this juncture). Not only do we learn about text editors and directory structures, but there is a whole chapter on styling with fonts and colours which will hopefully stop a rash of beautifully-coded-but-rubbish sites appearing in the wake of this book.

One section I wish had been in there (to help with the problems I came across when doing the Lily Gulshat page) is work-arounds for the fact that some popular browsers (not pointing the finger at IE in particular …) don’t get the cell padding attributes right. There’s a list of the “top ten topics we didn’t cover” at the end of the book which gives a brief overview and recommendations of books or websites. Perhaps this should have been number eleven. Yes, times change and people install new browsers but I can imagine many Windows users (or people running IE 5 on the Mac - hands up both of you) getting confused about why their layouts looked different from the ones in the book.

The other section would have been something pointing out that transparent PNGs don’t render properly in the current versions of IE. I use Paintshop Pro for graphics, have IE on my PC and spent ages trying to work out what I was doing wrong when saving the image - only to switch back to the Mac and find the page as I’d imagined it. To be fair, a “No Dumb Questions” spot on page 175 mentions that PNGs aren’t supported by all browsers - I’m probably asking for the moon on a stick here as I was going against the books advice by using them.

Conclusions

From a HF point of view, this is the best title I’ve seen yet. The colour helps enormously and the layout is like Head First Design Patterns but even better - and much less cluttered than HF Java. The graphic elements flow more rather than competing - I think this is a mark of the series maturing over time.

From a web authoring point of view, I’d like to wholeheartedly thank Eric and Beth for thinking up as many little things that might go wrong as possible (e.g. the problems with the footer on the Starbuzz Coffee website overlapping other divs), pointing them out and showing you how to fix them. Most textbooks only show you how to do things right, then you spend hours working out what went wrong. (Things always go wrong.) This and other HF books show you what wrong looks like as well, so you recognise it when you see it, and are better equipped to work it out for yourself after solving similar problems.

Would I, personally, have spent $34.95 USD on it? No - I already knew too much about the subject, just as I wouldn’t buy a beginners’ chess book unless I wanted it for good teaching examples. Would I tell an HTML novice to spend $34.95 on it? Unreservedly.

Yes, there are free guides on the web, but your time is important, right? You’ll save so much time from the book pointing out common problems and giving you the tools you’ll need to work things out for yourself when stuff goes wrong. I had to learn this the hard way when I picked up HTML and am enormously thankful that I didn’t have to with CSS. It bought me enough time to get the site up and running before everyone appeared on the BBC’s “South Today” and ITV’s “Central News” programmes!

Kevin Dangoor, creator of TurboGears, just announced that he has posted a screencast of his PyCon talk at this address. I look forward to watching it. Since I was unable to attend PyCon this year, I hope other speakers will do likewise.

I’ve used wikis for years. I’ve even written a few. I’d use one myself to manage my notes and ideas, but I hate typing in textareas and don’t want to launch a web server and new browser tab whenever an idea strikes. Ideally, I could just type a quick note in Vim, as I always have a command-line somewhere. Here are the three lines of code it took to make a single-user Wiki in Vim.

I run an open source project which uses Sourceforge for its infrastructure. This includes running an end-user support mailing list using Sourceforge’s mailing list manager. Now our users are often not particularly technical. They are largely drawn from the large numbers of people who know a bit of HTML and want to add things like guestbooks and formmails to their web sites.

The people who run Sourceforge are (rightly in my opinion) very keen on internet standards. They enforce them in any way they can. One of the ways that they enforce them is that their mail server will only accept email from a domain that has a working “postmaster” email address. This doesn’t effect most people as they are using mail servers that are administered by people who know the relevant RFCs and who have configured the correct standard mailboxes on their server.

However we have a whole class of user who has moved beyond using Hotmail for their email. They have bought their own domain and use it for their web site and their email. In many cases their hosting company will give them a web-based configuration application which allows them to set up all of the email addresses that they want. But at no point do they get told about the RFCs or advised that there are a certain number of email addresses that every domain should have. They set up a number of addresses and that all works fine until they try to send email to a server like the one at Sourceforge which rejects their email because they haven’t set up a postmaster address. They will then get a message explaining the reason for the rejection and, in the case of my Sourceforge project, they often track down my email address and I get a mail telling me that my email system is broken.

Another good example is the number of domains without a working “webmaster” email address. People download our web programs and install them on their web sites themselves. Often this installation doesn’t go well and they’ll end up with a broken installation. Then a visitor will find the broken formmail and get in touch with me (the error message includes a link to our web site - not the best design decision we took!) and tell me that my web site is broken. Of course it’s not my web site, it’s just someone using our software. Only the owners of the site can fix the problem. But getting in touch with the owner of a web site with a broken formmail installation isn’t easy. I try to pass the error report on to the owners of the site by contacting the webmaster, but in most cases, the webmaster address bounces back as undeliverable.

I just wish that people who are selling these hosting services took a bit more responsibility for the domains that their customers set up and told them about the standard mailboxes that all domains should have. I know that no-one has the power to mandate the existance of these mailboxes - but it would make my life easier if more people used them.

(See RFC 2142 and RFC 2821 for more details)

My wife’s store has been running in production for a few weeks now without much event. As with any website/webapp, enhancements and bug fixes are inevitable. I’ve already installed one new release which fixed a Javascript bug. I’m currently working on another release which will add some additional content and a couple of areas of feature enhancements. I’ve also been refining my release management procedures which includes a more formal test environment.

Which brings me to the topic at hand. I recently created a test environment at my hosting service. The test environment consists of a different domain name (subdomain, actually), a different directory within my home directory for placing application files, and a new database instance. When I installed the latest version of my code from Subversion, I encountered an error that I could not get past and which took me several hours to track down. (As a side note, it’s amazing how “obvious” a solution is once you’ve solved it.)

It turns out that in my test environment, all “decimal” types from the database were being pulled into my application as strings. When I performed a math calculation on them such as adding what should be a decimal to an int, I would get a type error. If only it were that obvious at first. The first error that I saw was a pickling error. And it wasn’t the pretty traceback that Django typically gives. This looked more like a documentation page generated from Python source, only nastier. I couldn’t figure out why I was getting a pickling error. I thought at first that it didn’t like me putting an instance of a custom class in session memory. (By the way, the pickling error appeared to be happening when Django was trying to stuff something into session memory.) That turned out to not be it. Somehow, through changing code, I was able to get a better error message and diagnose the problem.

This odd behavior is only happening in my test environment, and in neither of my primary development environment nor my production environment. Here is a query in my production environment showing the proper behavior:

>>> from django.models import store
>>> p = store.products.get_list()[0]
>>> p
The Gosling Design
>>> p.price
Decimal("25.00")

And here is the test environment showing bad behavior:

>>> from django.models import store
>>> p = store.products.get_list()[0]
>>> p
The Gosling Design
>>> p.price
'25.00'

Just for the sake of getting everything working properly, I had to go through the code and explicitly wrap decimal database lookups with a call to Python’s decimal.Decimal(). The only thing I can conclude is that my hosting service created the new database instance on a different version of MySQL. So, it really could be either a Django or a MySQL thing.

This is slightly stale news now, but well worth reporting. In case you haven’t heard of TurboGears, it is a Python web mega framework that is comprised of a number of other project in order to create an all-in-one web development toolkit. You can find the changelog for the current release here.

The fine folks at QA Podcast just interviewed Ian Langworth and me about Perl and QA. Have a listen at Testing with Perl.

Its funny how software development rotates in cycles. About five years ago everyone wanted to make an IRC client. If you were a geek worth your salt, you were either hacking on an IRC client, a window manager or a media player. It seemed that the world knew no limitations on IRC clients. Instead of just improving an existing client and refining it, the hacker community engaged in feverish itch-scratching that was somewhat anti-social - everyone hacked on their own client. It was evident that some l33t scr1pt f00 was the order of the day for the pointy-haired IRC client hacking fraternity.

Today the same cultural phenomenon seems to have been applied to the much revered Content Management System (CMS). For those of you who have been living under a rock for the last five years, the humble CMS provides a website engine that can be used to create a site without the need to wrap yourself up with HTML, CSS, scripting languages and small paper napkins with strange CSS workarounds scribed upon them. From early successes such as PHPNuke, CMSs have grown dramatically. A quick look at the CMS Matrix provides a head count of 527 CMSs at the time of writing. Yes, 527. Insane.

As consumers, we like products and services that are tailored to our experience, expectations and demands. We don’t want to be bogged down in features and functionality that is not directly related to what we want to achieve. This is one of the core principles behind good design and usability - know your audience, know their expectations and experience, and cater for them. If you step outside of your projected audience, the world starts to get confusing for your users and they tend to daydream and wander off somewhere.

So what has this got to do with CMSs? Well, I believe that the vast majority CMSs are making this very mistake. For the last four or five years I have played with a bunch of different CMSs and they have been quite clearly marketed towards people who want to set up a website, but don’t want to resort to building their own site from scratch. In the majority of cases, the typical user seems to be someone who simply wants to point and click to create their website; a site that they can configure by using software as opposed to writing software. A fairly reasonable intention it may be, but one that many CMSs fail to deliver on.

The problem is that “setting up a website” is an heroically diverse task. Although it sounds simple, there are a huge array of variables involved in building even a simple website - style, organisation, menus, differing types of content, user permissions, media types, report generation, audits, syndication, security; the list goes on. Interestingly, I suspect that out of this list of related issues, you probably read some and thought “sure, but I don’t need that…or that…or that!”. This is exactly my point.

Most CMSs seem to identify their direction in terms of simplicity as opposed to context. The majority of CMSs are geared towards providing a simple, yet generic platform on which a site can be crafted. The problem with this approach is that the number of variable factors associated with the task of ‘building a website’ has a consistent effect on the level of simplicity offered. The concept of simplicity is primarily affected by choice. Put someone in front of three buttons that are clear, intuitive, learnable and culturally grounded, and they will get on a lot better than putting them in front of 100 clear, intuitive, learnable and culturally grounded buttons. Its a simple theory - the more options and configurability you put in front of a user, the more difficult it is for that user to achieve what they want to do unless they know the full extent of the system. This is why cars don’t have aircraft cockpit style interfaces.

Define the context

I am a firm believer in contextual usability, and there is one particular project has capitalised on context and managed to create an incredibly usable CMS - Wordpress. For some time now, the Wordpress project have continually created exceptionally high quality, very usable, well designed software. As a consultant I have recommended many, many people to use Wordpress because it is devilishly easy to use.

When you crack open the Wordpress success story, you can leverage much of it to context. Wordpress is intended as a personal publishing platform. It is primarily intended for use as a blogging engine, and with the recent ability to add static pages to the site, it is entirely usable for someone who wants to create a simple website with the ability to add regular textual content such as news, blog entries, event details and more. With this focused direction you know that everything in the administration interface is to do with something relevant to the extent of your site. As such, Wordpress fits the needs of its users perfectly and they don’t try to solve every web related problem in the world; they just concentrate on their context. Other CMSs really need to take a leaf out of Wordpress’s book, and avoid trying to take a huge overwhelming hammer to crack a very small nut.

I would love to see a number of CMSs that target very specific uses such as band websites, Open Source projects, online art galleries and more. By defining a clear context and dedicating development to simplicity, there is so much more potential to give people exactly what they want with little fuss and little clutter. There will always be a place for these catch-all CMSs, and they are good for large comprehensive websites, but they don’t so much provide a simple way of building a site but really provide a RAD way to build a site. There is room in the pond for everyone and I really genuinely hope we can see more context sensitive projects such as Wordpress stand up and claim their users.

What do you think? Wisdom or rubbish? Share your thoughts here…

Related link: http://www.wasabisystems.com/gpl/

Honestly, I don’t know how thousands of businesses can thrive on free
software. They certainly wouldn’t if they listened to the software
industry experts. First there were all the claims (borne out by some
real major failures in the industry during the dot-com bust) that
there was no business case for making money on software that everyone
could download, run, and freely alter. Now that free software is a
well established industry, there comes a
white paper
by Jay Michaelson of Wasabi Systems, which got reported on by

ITManagersJournal.com,
NewsForge,
and others.

You’re probably expecting me to sneer at Michaelson’s paper, but I’m
not going to. It’s an excellent essay, in my opinion. It describes a
real–though very limited–problem. In fact, I think a recognition of
this problem may lead to an increase in dual licensing, which will
allay many of the fears expressed in the paper.

Basically, Michaelson’s white paper lays out the controversy over the
GPL’s share-and-share-alike provisions, often called “viral” by its
critics. This controversy (especially the paper’s corresponding
praise for the BSD license) goes back a couple decades and is familiar
to anyone who’s followed open source.

Where Michaelson does not report fairly, in my opinion, is that he
doesn’t make it clear what a tiny sliver of businesses are affected by
this provision: businesses such as his, which sells embedded systems.

Even these businesses should not be wringing their hands over the GPL,
because they have hardware to build their revenue on. They needn’t
fret over releasing the source code to their drivers–they should
instead be expressing gratitude that Linux provides such a great
platform for them to release their drivers to.

Still, companies cite various legal reasons (cross-licensing,
government restrictions on radio emissions, and so forth) for needing
to keep source code secret. You can pick each one apart, but
Michaelson is within his rights to point out that companies do worry
about this issue, and that the GNU/Linux communities have left the
area deliberately ambiguous. The new GPLv3 doesn’t seem to offer any
resolution to this issue either.

Michaelson makes another valid point (though not as directly as I make
it here). Most companies that used closed source software have
explicit licensing agreements that protect them from liability from
lawsuits and the provisions of Sarbanes-Oxley. Companies that use free
software don’t have those agreements in place. That’s why I think that
people reading this paper may have good reason to offer dual licensing
for the software they produce, and to sign such licenses for free
software they bring in house.

But even here, GPL critics go too far in singling it out for
blame. Many of the licenses that are usually seen as more
industry-friendly, such as the Apache License and the Sun Community
Source License, contain restrictions of their own, and these could
just as easily turn into traps. Sarbanes-Oxley in general requires
companies to be careful–very careful. (By the way, legal folks have
been talking to me about the interaction between Sarbanes-Oxley and
free software for a year or two; this article does not reveal anything
new.)

And why are the “intellectual property” provisions of Sarbanes-Oxley
so draconian? Not because of free software advocates, I can tell you
that. The provisions must be there because major copyright and patent
holders wanted the largest possible stick with which to beat companies
that dare to use copyrighted and patented products without jumping
through the licensing hoops set up by the intellectual property
holders. If these enemies of free software have set up such a
frightening legal phalanx to further their own business needs, it’s
only poetic justice that the same phalanx can be called into play to
uphold free software.

The ubiquity and lack of barriers to using free software allow people
to abuse it by hiding it in proprietary products.
Companies may find it worth hiring
Black Duck Software
or
Palamida
to make sure they comply with free software licenses. Yes,
intellectual property regimes help make it dangerous to go into
business. Free software can add its own complications, but code
reviews and dual-licensing provide recourses.

Related link: http://listmixer.com

“ListMixer is an easy way to track web pages that momentarily hold your interest. It’s handy for tracking blog comments or for pooling timely web pages among friends. No account is required. Really.”

Friday night I hatched this idea. Late Monday night it went online. After some debugging early Tuesday, I emailed my friends. Not only is this my fastest TAT for a web app, but it is the most useful web app I’ve made. I love this thing, and I think you will too.

It’s ListMixer, a bookmarking service for the meme generation ;-). This is what I am telling my friends:

It uses a bookmarklet. Each bookmarklet has a (long) unique ID in it, so there’s no need for formal accounts.

Click the bookmarklet to add the page you’re viewing to your Mix. If a link in your list goes unclicked for 30 days, it gets automatically deleted. The result is a nice, informal parking place for interesting links. If you want to promote a link into your permanent bookmarks, just hover over it and ‘add this’ links for del.icio.us, etc., appear.

Share your unique bookmarklet with friends to pool links into a little <buzz>meme tracker</buzz>.

You can embed a link roll on your site to share.

You can subscribe to your link roll via RSS.

It’s. So. Cool. (-:

Take a look at my own Mix to get a feel for it.

The initial uptake has been good. Paul Kedrosky says:

… Sid Steward has a funky new service called ListMixer where you add your bookmarks, and then if you don’t use it again in 30 days it disappears. Love it! It’s like the vestibule for bookmark heaven, a place to keep things until you’re really and truly ready to commit to permanence.

Chris DiBona says: “you totally have to have to have to kill the logo. Sorry dude :-)”

Please give it a try and let me know what you think. Thanks!

Here’s an example of one link roll style. Give it a whirl!

Here is another, more lightweight style. It is also good for mobile devices:

More at the ListMixer link roll page.

Captchas have become a popular, though not impenetrable, method of keeping bots and automated scripts from abusing a website’s resources. The distorted strings of random letters and numbers over an obscuring background of gradients, speckles, and lines help ensure that the user of a web form or other server resource is a human, rather than another computer. — unless the person is blind or visually impaired. Captchas can’t be read or used by the screen reading software they use to surf the web.

In the Web Site Cookbook I wrote about a hosted Captcha service that makes it easy for website builders to implement captchas on their sites. Now the guys at captchas.net offer an audio alternative to their service, which is free for non-commercial use. You heard it here first.

In Django, you can specify an image as a database “column” type. Since it’s a data type where you define your database model, you would think that the image is in fact stored in your database, but you would be wrong. The only value that is stored in the database is a reference to the file. This is another example of the integration between Django’s database and automatic form creation. Here is the “column” I defined for the photo I wanted to upload:

photo = meta.ImageField(upload_to='photos/%Y-%m/', blank=True, null=True)

I was using a standard AddManipulator to process the form that this photo is part of. Every time I tried to upload a file, it only stored the data from the other form fields, but didn’t store a reference to the file, nor did it save the file itself. I thought it was odd that I was only saving the request.POST data in my view, but I thought that maybe it should be doing some magic under the covers to get the request.FILES data as well. I found a post on Google Groups thath mentioned that to get file uploads to work you have to update your request.POST dictionary (which it’s standard to copy that dictionary to another dictionary) with your request.FILES dictionary. Once I included this missing step, I was able to upload files.

This is one of two non-wife, non-job projects that I’m currently working on. Since it’s non-job and non-family, I’m only able, and willing respectively, to devote the wee morning and late night hours to working on it. In a couple of days of my scrap time, I’ve been able to get the functionality pretty much done. I guess I should be saying “Django rocks” at this point, but I think that’s obvious enough that I don’t have to say it.

After rolling out my wife’s site, I decided to build two of my own - in Django, of course. I won’t get into what they are, but I created a data class (think database table) with a __repr__ attribute (I’ll get to what it’s for in a second) that looked like this:

    def __repr__(self):
        return "%s" % (name)

Django’s database approach uses a data object’s __repr__ attribute to determine how it displays in the admin interface and in forms when you are taking advantage of all the Django-form-goodness-stuff ™. The particular class I defined the above __repr__ on has a “name” attribute, and I wanted the display value to be the object’s “name”. Clearly, I should have prefixed “name” with “self.”. Even though I was able to create a new record for this data type in the Django admin, this apparently kept the data from rendering in the admin interface and all I was getting was an error after creating the record. Now that I have my code fixed, I need to dig in a little bit and see if there is anything that Django could have done to make the error messages a little more helpful. Except for this little bump, due to my own stupidity, everything has been smooth sailing.

I had the pleasure of meeting Richard Bejtlich of TaoSecurity at the RSA 2006 conference. Richard and I worked at Foundstone, but I never got a chance to meet him in person until last week. Richard has authored and contributed to a few books, the latest of which is “Extrusion Detection: Security Monitoring for Internal Intrusions“. Check out the full list here. Richard just wrote about my RSA talk on his blog, and I’m glad he thought well of it. I had a lot of fun speaking there.

I’ve been meaning to mention Bejtlich’s TaoSecurity blog. I make it a point to read it every day, and it is quite informative. If you are interested in the topic of network security, I highly recommend you add his blog to your daily list of blog reading.

Related link: http://www.pool.ntp.org

Someone stops you on the street and asks “Hey, do you know what time
it is?” You tell him and continue on your way. But what if it was
hundreds of thousands of people every day, because they didn’t know
who else to ask? You might decide to not answer any more. That’s the
situation that some important Internet time servers are in, and some
simple changes in your computer’s configuration can help ease the strain.

NTP, the Network Time Protocol,
is a standard for net-connected computers to find out the correct time
from other computers. Your computer’s clock can easily lose accuracy
over time for many different reasons, and setting it to regularly check
a server via NTP will keep its clock always in sync with the correct time.

What’s the correct time? There are a number of servers attached to highly
accurate atomic clocks, or to clocks synced to GPS satellites, called
stratum 1.
These are basically the master clocks for the Internet. Servers that
sync to the stratum 1 servers are in
stratum 2.
Because of the time it takes to communicate between servers, there is
always a little bit of a drift from accuracy when connecting between
servers, so stratum 2 is slightly less accurate than stratum 1, and
stratum 3 slightly less accurate than stratum 2, and so on.

For servers that must be highly accurate, such as for extremely
precise scientific applications, only connections to stratum 1 servers
will do. For the rest of us who can tolerate being off by a few tenths
of a second, stratum 2 and below will do fine.

Unfortunately, as the number of computers on the Internet has exploded,
so has the number of computers syncing their clocks to stratum 1 servers.
These servers usually don’t need the accuracy of a stratum 1 server, such
as nist.time.gov,
but because they are so well known as a time server, people often use
them by default. The big popular servers were beginning to buckle, and
some stratum 1 servers have had to move to a permission-based system,
and some have withdrawn completely. The load of tens or hundreds of
thousands of users with hourly cron jobs for rdate -s time.nist.gov
to set the time has just proved to be too much.

To fight this problem of unnecessary load on stratum 1, and to
make it easier for people to make their systems well-behaved, in
2003 Adrian von Bidder created the all-volunteer NTP Pool project
(http://www.pool.ntp.org).
The pool is a set of freely usable time servers. When a client machine
tries to sync to pool.ntp.org, that machine is referred to one of the
pool servers, round-robin style. This helps distribute the loads to
different servers in the pool.

How you can help

You can help relieve the load on the strained servers by switching your
computer’s settings to use pool.ntp.org. Your system will start
using one of the pool servers. You can even choose a geographically
close pool by using us.pool.ntp.org for United States, europe.pool.ntp.org
for Europe, and so on. See
http://www.pool.ntp.org/use.html
for more details, including how to modify your system’s settings.

You may not need to change your settings. For example, Mac OS X comes
set to check Apple’s own time server at time.apple.com, and the
Ubuntu Linux distribution is set to check ntp.ubuntulinux.org.
Your ISP may also have a time server available. My ISP, Speakeasy,
runs a time server which is only four hops away. On the other hand,
if you’re checking clock.isc.org or time.nist.gov,
please change to pool.ntp.org immediately.

The pool.ntp.org project can use more volunteers, too. At this
writing, there are 600 servers in the pool, with the number growing
every week.
Ask Bjørn Hansen
of
Develooper.com has taken over the
administration of the project and is always seeking more servers to add
to the pool. All it takes to run an NTP server is a little tech savvy,
a static IP, and a continuous connection to the net. For more information
on how to join, visit
http://www.pool.ntp.org/join.html.

Have you checked what time server you’re using? Did you need to change it?

I just installed the “Tamper Data” Firefox extension to test some code that I just wrote for my new Django project. Basically, I’m creating a storefront where users can each add multiple stores and multiple products to their account. They can associate each product with multiple stores and each store can contain multiple products. But I don’t want users to be able to stuff their products into stores that don’t belong to them by doing nasty stuff in HTTP POST data…not that many people would think of doing that. So I wrote code to prevent that kind of misbehavior. I then started testing my code with “Tamper Data” and found that Django had already beat me to the punch. If data that is POSTed back to a form didn’t initially belong to the form (in my case, an ID from a select list), Django tells the user they need to fix their input. I don’t know which all form input types this applies to, but it apparently applies to select lists. This error wouldn’t have happened if someone (me) weren’t monkeying with the data. I was really amazed when I saw the Djangoish validation error and the form show back up. This is a very nice touch in form validation. I guess the bad part of all of this is that now I’ll need to rip out my “nastiness-checking” code. Oh, well.

Related link: http://www.navicasoft.com/Newsletters/February%202006%20Newsletter.htm

The ever-insightful Bernard Golden’s February 2006 Open Source Newsletter compares and contrasts Internet advertising giant Google (whose product sells for pennies per unit) with the growing but unassuming MySQL UC (with a free product, of course, but also licensing fees more than a few pennies per unit). Yet for all Google’s giant presence, it needs MySQL or something like MySQL to succeed. What does that mean for your organization? Bernard has a few ideas….

Earlier this week, I noticed a couple emails from the Mayor of Seattle in my inbox. That’s not so unusual. I get lots of mail from high sounding offices, although most of them seem to be former heads-of-state from Africa. What if the mail was real though? Is unsolicited mail from our governments as unwanted as unsolicited mail from the companies we have “an existing business relationship”? Should we have to opt-in to this sort of mail, and how does that work if the Mayor really needs to get the word out?

I was curious about the email, titled “Alaskan Way Tunnel Open House” and what sort of scam that might be, but the email looked genuine. I checked the Seattle Mayor’s website, and indeed there is an open house about the Alaskan Way Tunnel. Apparently some shipworm is eating through the tunnel walls, and the 2001 Nisqually Earthquake damaged the viaduct and the seawall. The city needs to do some work and they want community input. The email text is an edited version of the Mayor’s web page for the tunnel.

Why did I get this email though? I live in Chicago. It doesn’t seem like the sort of thing to send to potential visitors: “Don’t drive through our crumbling tunnel!”.

I figured I’d call the Mayor and ask him about it. Heck, it’s raining in Chicago and that puts me in a Seattle mood anyway. I wanted to find out since it’s not often that I can actually put a name on the person (Greg Nickels) who sent me unsolicited mail, but by nature of his public office, I can even put a face on it. Indeed, i can even see him throwing the first pitch at a Mariner’s game and posing with Mariner’s catcher Ben Davis. Why doesn’t Chicago’s Mayor Daley have some action shots of him with the World Series Champion White Sox? Before I even start I’m think Seattle’s mayor is cooler than mine. He’s got better pictures and he knows about email (even if the etiquette is still catching up). Although he doesn’t have RSS or Atom feeds, he does have streaming video and audio of his weekly call-in shows.

I called the mayor’s office and spoke to Sharon Thomas. She confirmed that the mail was genuine and that anyone who had ever emailed the Mayor’s office received the mail.

A long time ago, I wrote to the City of Seattle’s Department of Transportation to verify some facts about their bike paths I had read in Bicycling magazine. I got a nice reply that answered my questions and thought that was the end of it. Apparently not. Now I’m in the list of people who have written to the Mayor’s office. Anyone who has ever emailed the Mayor’s Office is on this list and got that email.

They got it twice, in fact. My first message was on February 14 at 2:16 pm PDT, with my old email address in the To: field, and the second one later the same day at 5:21 PDT, with “undisclosed-recipients” in the To: field. The body of the message looked the same save for the first line that said “Sorry for the duplication but please disregard the previous message.” A diff on the message body confirmed that, aside from whitespace, the messages are the same.

Acccording to Ms. Thomas, the Department of Transportation asked for the list of email addresses, and the Mayor’s office gave them the list. She couldn’t tell me who in the Department of Transportation asked for it, though, or who decided to try this. If this was the first time, a limited trial might have been in order. I can’t be too judgemental though, because I’ve certainly done stupid things with some of the lists that I’ve managed. Computers and networks make it really easy for orders or magnitude more people to notice a mistake, and even easier to complain about it.

Ms. Thomas is also the one who apparently has the job of apologizing to anyone who responded negatively to the email, and I found a message from her replying to my old email addresses autoresponder which tells people I don’t get mail there anymore. I’m curious how many times she had to send that same message. Out of a city with around half a million people, how many have emailed the mayor? I wonder how Seattle stacks up to other cities for the level of involvement of their citizenry in digital democracy.

To her credit, she was sincerely apologetic and realized the mistake of sending mail to everyone, and said that I should talk to their communications people. They weren’t available, though, but she offered to remove my email from the list. When I asked her about all of the other people who didn’t ask to be on the Mayor’s mailing list, she said that she removed the addresses for the people who responded to complain. That is, if you don’t want to be on the list and didn’t complain, the norm in dealing with spam so that the bad guys don’t know they have a good address, then you are still on this list. She again assured me that this was a one time thing, and from the response they’ve received, I’m sure that’s true.

So, what to think of all this? Spam is usually some predatory or nefarious thing with a purposedly anonymous actor at the other end. This mail is about a legitimate subject from a real person whose office I can actually call, and they’ll talk to me about the mail. I’m a bit annoyed that the City of Seattle wasn’t careful about managing their lists and not using an opt-in method, but they also want more involvement in community projects and the things that affect the people of Seattle.

Is unsolicited mass email from our representatives spam?

Related link: http://www.microsoft.com/downloads/details.aspx?FamilyId=F22E51E5-B82E-4A54-9CCC…

If you haven’t been keeping up with things, IronPython is an implementation of Python for the .Net framework. Basically, you can write Python code, compile it to .Net’s bytecode (I think they call it “Intermediary Language”), and run it from the .Net runtime. Probably more importantly, you can call .Net libraries from Python.

This release has a number of bug fixes, many of which appear to be specifically intended to insure compatibility with “standard” Python. I keep saying it every release, but it warrants saying again. I’m continually pleased that Microsoft is going through as much effort as they are to keep IronPython compatible (in syntax and experience) with standard Python. They could have, at any time, decided to abandon compatibility with Python and to create a press release announcing the creation of a new language, P#. (I know. That doesn’t work with the implied musical connotation of C#.) But they haven’t done so. Oh, wait. That’s right. They’re waiting to hijack the language until after 1.0 final and after they’ve accumulated a sufficiently large user base. I know that thought is running through people’s minds. I confess it’s run through mine at least once. But I don’t think it’s going to happen. At least, I’ll be very surprised if it does.

Related link: http://hop.perl.plover.com/~alias/list.cgi?2:mss:576:200602:blkdlhdcogeklhcpgikm

Recent threads on the Higher Order Perl mailing list discussed a particular tasks for which Perl’s functional programming techniques make an easier solution than the obvious object-oriented approach.

Syntactically, Perl 5 isn’t always great at either. There’s definitely a bit of line noise in functional programming in Perl 5 and the boilerplate code in setting up objects and classes gets tiresome after a while. (Fortunately, Perl 6 corrects both.)

When the idea of functional versus OO approach came up, someone offered a Python solution to the problem. Because Python’s functional programming support is, despite the protestations of overzealous snake-handlers everywhere, somewhat less than complete (and no, don’t send me mail on this — read-only, single-line closures don’t count), the example took the OO approach.

Then the question of maintainability came up.

To understand the closure-based approach, a programmer needs to understand lexical variables, closures, and first-class functions. He also has to be able to read the code. To understand the OO-based approach, a programmer needs to understand classes, instance variables, and objects. He also has to be able to read the code.

There seems to be a rough agreement on the list that the functional approach takes significantly less code. In that case, it seems to me to be a win for maintainability to use the functional approach (and it’s nice to have a language that supports it).

One often-stated objection to using so-called advanced language features (or languages that provide more than one way to solve a problem) is that novices may have difficulty reading code written by experienced programmers. My theory is different:

I believe there’s a cost for each significant unit in code, whether syntactic or semantic. If the right approach for a problem means I pay for semantic complexity in order to avoid a higher cost elsewhere, fine.

That means that my successor and co-workers either have to be good programmers themselves (as I hope I’m a good programmer) or at least trainable. Fortunately, that’s pretty much my minimum requirement for writing good software anyway.

I wonder if framing the issue in terms of the cost of complexity helps make certain decisions clearer.

Here’s where you try to convince me that Python’s lexicals really aren’t broken and that no one really needs to write to closed over variables while ignoring the actual point of my post.

Related link: http://desktop.google.com/plugins/i/foldershare.html

This is a follow-up to an earlier post of mine about Google Desktop’s new remote computer search feature. I am enamored with the idea of PCs as peers on the Internet, serving any of: VoIP, webmail, blogs, photos and media to my friends, family or the world. You can see this emerging today in Skype, Hello.com and FolderShare.

I think Google Desktop is moving in on FolderShare with some of its new features. FolderShare appears to have more respect for privacy, however. And FolderShare has mature p2p file sharing today.

The EFF criticizes Google Desktop’s remote search for storing your PC’s files on Google’s server:

… If a consumer chooses to use it, the new “Search Across Computers” feature will store copies of the user’s Word documents, PDFs, spreadsheets and other text-based documents on Google’s own servers, to enable searching from any one of the user’s computers. …

Yikes. Meanwhile, the FolderShare user agreement makes it pretty clear they want nothing to do with the contents of your private documents:

… You are solely responsible for the contents, modification, management and/or deletion of any and all files and data used by you in conjunction with FolderShare. … ByteTaxi does not maintain such files or data on its server, but it may log certain information regarding the files you use in conjunction with the FolderShare service, such as file name and file size; provided, however, that we will not access the file itself or any data or information contained therein.

This sounds great. But why say “provided, however”? I’m not the best at interpreting legalese — I hope that means what I think it means.

I sent an email to FolderShare customer support about where they store the search index for remote computer search. On the peer, of course:

Hello-

I see I can search remote computers using FS + Google Desktop or MSN Desktop Search (using plug-ins). I wonder: is the search index stored on my machines or on your server (or a third-party server)?

…

---

Hi Sid-

When users remotely search their computers with FolderShare and Windows Desktop Search or GDS the indexes are stored locally, not on any central servers.

A p2p service that keeps everything on the peer. I like that.

What is the most frequently used type of navigation on a website? Links? Images? Drop-down menus? Wrong. It’s the browser’s back button. And buried in the developers’ explanation of Firefox’s so-called “memory leak” is this little nugget of statistical goodness that (er) backs me up: “To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited less than 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab.” The full explanation is on the Inside Firefox site.

Related link: http://www.perl.com/

It’s time for more Perl Lightning Articles — short, 250 - 500 word pieces on some aspect of Perl programming and culture. Take your inspiration from Lightning Articles and More Lightning Articles and send me an e-mail with your idea by next Tuesday, the 21st.

Or post your idea here, I suppose….

Related link: http://daddison.com/wscb/excerpts/writing_link_text.html

The little candy hearts of Valentine’s Day teach an important lesson about writing for your website: Using simple language and knowing your audience are among the most effective ways to get your message across.

But since V-Day only comes once year, a better source of inspiration for ways to improve your web writing are the dozens of emails you send every day. But according to one study, the current state of affairs is grim: Your recipients have only a 50 percent chance of misunderstanding you. According to one researcher, “the reason for this is egocentrism, or the difficulty some people have detaching themselves from their own perspective.” We can and must do better people!

Email writing makes a good (though not necessarily perfect) platform for practicing your website writing. Check out these articles from 43 Folders and Guy Kawasaki for some helpful pointers.

I’ll be speaking at the RSA conference on February 15, 2006 (Wednesday.) I plan to discuss the following topics:

- Case for open source assessment tools
- Exploiting the WMF vulnerability + Dissecting the WMF exploit (Metasploit)
- Nessus 3.x + Writing plug-ins for Nessus
- Writing plug-ins for Ettercap
- Finding vulnerabilities via Google
- Problems with Google’s anti-phishing Firefox plug-in
- Launching attacks via Tor

My talk will be at the SJCC Ballroom A4 (3:25pm.)

I’ll be at RSA for the entire week. If you’d like to chat, or go grab a cup of coffee, do send me an e-mail!

Related link: http://shiflett.org/archive/196

Here’s my Top X List of Mac OS X Annoyances:

1. Separating Menu Bar from Window Is
   Stupid.
2. Apps Don’t Really Close When You
   Close Them.
3. Maximizing Is Broken.
4. Alt-Tab to a Minimized App, and It
   Stays Minimized.
5. Too Many Option Keys.
6. No Dedicated Page Up, Page Down,
   Home, or End Keys.
7. Only One Desktop.
8. The Clock Sucks.
9. iPhoto Sucks.
10. Safari Sucks.

I elaborate a bit more on my personal blog, but these are the main things keeping me on Linux for my primary desktop.

Feel free to point out ways to get around these annoyances, and of course, let me know if any are just a result of my own ignorance. :-)

What annoys you about the Mac?

I think the biggest enemy of an IT company is the human resources department.

This may sound initially like sour grapes, but you’ll have to trust me that it isn’t. I recently accepted a good paying job with lots of benefits in a major software company. However during my search for a steady job, I had trouble getting to the interview stage for even intermediate C++ programmer positions at well over a dozen companies.

Why this is a shock has to do with my credentials as a C++ expert. I cowrote the C++ Cookbook for O’Reilly, I was a columnist for the C++ Users Journal, and I have a decade of professional software development experience. People who have read my column know that I am a pretty good C++ coder.

So why did I have such trouble finding work? Well three reasons.

1. Recruiters don’t know anything about programming and are ignorant of virtually everything related to software development. Many hadn’t even heard of Boost, O’Reilly, or the C++ Users Journal. They didn’t understand the significance of my credentials.
2. Recruiters view my freelance experience as a negative point, even though they say they want “self-motivated independent problem solvers”. Apparently I am too independent!
3. I only computed two years of a university degree in computer science.

So instead interviews are going to people who has have been suckling the corporate teat since they graduated from university only a couple of years ago. There is a lot these people don’t know about the business and practice of writing software.

If you are running an IT company, I would suggest that you take a long hard look at your applicant screening process. Perhaps you should consider getting your lead developers involved in the initial screening of applicants. You might have missed an opportunity to hire someone like myself, and instead find yourself having to choose from a bunch of inexperienced college grads, demanding far too much money for what they can actually do for your company.

Django is the web application framework that I’m currently using. It has this really nifty feature of allowing you to create URL mappings which do regex matching on requests coming in and then pass them off to “views” which process the request. You can read more about it here .

Here is a little snippet of a URL conf from one of my current projects:

    (r'^stores/$', "dj_store.stores.views.stores"),

Using the view “dj_store.stores.views.stores” caused a naming conflict. I was a little bummed by that because I wanted to keep a clean URL structure and provide a simple URL interface such that a GET of a simple resource name (such as “/stores/”) would display a list of all available resources of that type. (So, a GET of “/stores/” should display a list of all stores.) I started thinking of acceptable alternatives for what to change the URL to: “/get_stores/”, “/show_stores/”, etc.. None of these choices appealed to me because it felt like I was uglying up my URL just to avoid a naming conflict. But then it occurred to me that I didn’t have to do that. In Django, the URL is independent of the Python function that handles the request.

So, I changed the above URL mapping to:

    (r'^stores/$', "dj_store.stores.views.all_stores"),

(and obviously changed the view to “all_stores”) and I was able to keep the URL naming I liked and avoid a Python naming conflict. I know this is really simple and really obvious, but sometimes I can’t help being like a kid who is fascinated by playing with a cardboard box. And I think my “duh” moment and fascination with simplicity and obviousness doesn’t detract from the excellent design decisions of the Django developers regarding their URL configuration.

Dun & Bradstreet is a third-party information source about companies and businesses. I can verify information about other businesses (i.e., that they actually exist), and other businesses can do the same for me.

I checked on the Dun & Bradstreet report for Stonehenge Consulting Service today, and some of the information is a bit dated, so I wanted to update it. I haven’t done that for Stonehenge before, so I had to add myself as an authorized person. That’s no big deal.

But I can’t. I try to fill out their “User Identification” form, but I get some sort of Microsoft error. I’m only submitting very basic information, so why does it need to do fancy processing? Do that stuff somewhere else, but let me give you my information. You have to manually verify it anyway (or you’d better, but that’s a different problem).

Active Server Pages error '8002802b'
Create object failed
?
An error occurred while creating object 'sockICW'.
Microsoft JScript runtime error '800a01ad'
Automation server can't create object
/product/eupdate/eupdate.asp, line 289

I reply to customer service that I get some odd error. I don’t say much:

On Fri, 10 Feb 2006 customerservice@dnb.com wrote:

> https://www.dnb.com/product/eupdate/update1.html

When I submit this form, I get a web server error. Is there another way I
can submit this information?

The answer I get back tells me that I’m not supposed to use their website unless I’m on Windows and using Internet Explorer. I’m using Firefox on Mac OS X. I would show you the whole email in all of its weird “English as a second language” goodness, but I can’t (well, shouldn’t). I didn’t realize there were people who were recommending browsers. That’s so 1990s.

All platform-advocacy issues aside, remember what I’m doing. I’m just submitting a form with my name and company information. It doesn’t update anything and they have to look at it to decide if I’m who I say I am. It shouldn’t change anything. Why would I need to use any particular operating system or browser to do that? Just what is Dun & Bradstreet doing that requires ASP, JScript, or anything else exotic?

What’s the real story here? Did they get trapped by some vendor who can only handle Windows? Or does it relate to something else?

Poking around, I did find some interesting Javascript, but not anything too worrisome. Besides the basic form validation stuff, they use something from Coremetrics. If you look at the Javascript file, you’ll see a lot of stuff tracking how much time I take to do things. That doesn’t really worry me that much, but it is pretty stupid. Don’t look at the code because it’s patent pending (unfortunately, the USPTO only shows applications back to 2001, and no results for patents issued to Coremetrics).

<script language="Javascript1.1" src="coremetrics/v25/cm.js"></script>

<script language="Javascript1.1">
<!--
/* Data Collection JavaScript v.2.5, 06/26/2001
   COPYRIGHT 1999-2001 COREMETRICS, INC. ALL RIGHTS RESERVED. U.S.PATENT PENDING. */
if (cmSiteUp == "Y") {
CM_Param("pn","eUpdate:update1");  //unique name for page
imgReq = CM_BuildTag("C",1,1,1);
}
//-->
</script>

And hey, look at that date. It’s 2006 fellas, so how about some Firefox support?

Here are a few thoughts that I have now that I’ve completed porting my wife’s website from TurboGears to Django. This isn’t intended to be a comprehensive list of differences between TurboGears and Django. Nor is it intended to assert superiority of one framework over the other. These are simply my thoughts and feelings regarding the two frameworks. That being said, my conclusion is that Django is a better fit for me.

1. TurboGears has a little bit faster hit-the-ground-running startup time because it creates all the files you need with a single command without having to run subsequent commands to create the application. This is a definite tradeoff, though. Even though it’s another command to create an app (within a project) in Django, the flexibility of being able to have multiple apps self-contained in their respective directories is, in my opinion, worth the extra step.

2. TurboGears’ directory structure and file layout is simpler than Django’s. Again, the number and location of files and directories in Django is a direct result of the flexibility of being able to have multiple applications in the same project. I don’t mind the slightly deeper directory nesting or the few more files in a new Django project (and app).

3. The template system in Django took a little bit to get used to. One thing that tripped me up for a while was Django’s variable substitution does dictionary and attribute lookups, method calls, and list indexes all using a dot. I expected to access a dictionary’s key using some_dict[”some_key”], but instead, it’s some_dict.some_key. And I expected to access a method by some_object.some_method(), but instead, it’s some_object.some_method. This was clearly my fault as it is obviously documented here After getting past that hitch, I really like Django’s templating system. It’s really simple but powerful enough to do anything you want to do. Since you can’t just execute arbitrary code in a Django template, it forces you to pass in the data you need from your view. This separation is a good thing. Unquestionably, in my mind anyway, Kid is a very powerful templating system. However, the ability to execute pretty much any Python code you want to from within a Kid template is a little concerning. And the requirement of Kid documents being well-formed XML can be a bit of a pain. And when you create one that isn’t well formed, you don’t always get back an accurate error corresponding to your Kid template. For example, here are the first seven lines of a Kid template which I intentionally mangled (line numbers supplied from vim):

      1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://w        ww.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
      2 <html xmlns="http://www.w3.org/1999/xhtml"
xmlns:py="http://purl.org/kid        /ns#">
      3
      4 <head>
      5     <title>${title}<title>
      6     <link type="text/css" rel="stylesheet"
href="/static/css/ppp.css"/>
      7 </head>

Notice the mismatched title tag on line 5. The error I received was ExpatError: mismatched tag: line 7, column 2. A tag name would be more helpful here. And an accurate line number corresponding to the Kid template would be helpful as well.

4. Which leads me to the errors you get back from Django templates. They are excellent. If I try to access a URL that isn’t specified in the urls.py file, I get a nicely formatted error page with the contents of my urls.py file that helps me see the errors of my ways. And if I have a syntax error in my code, I get a traceback and source code with highlights of the offending code.

5. I’ve really come to appreciate Django’s URL mapping approach. URL->function calls are explicitly declared in the urls.py file. This allows me to easily create URLS that look like I want them to look and not have to hang one class off of another to create the nested structure that I may want. It also lets me create RESTful looking URLS if I so wish. Another benefit is that you get (for free) a “table of contents” of your site all in one file. TurboGears is a little more awkward to fashion your URLs the way you want them. I did read the other day that there is a package that sounds like it allows CherryPy to use an alternative URL mapping scheme which sounds more like Django, but I haven’t investigated that yet.

6. Django’s admin interface is really slick. I haven’t been able to use it as much as I would like because it doesn’t appear to be able to display nested data structures on a single page. For example, if I have a Customer data object which can have multiple CustomerOrder (as in, the customer has ordered some merchandise) data objects and each CustomerOrder can have multiple OrderItem (as in, individual pieces of merchandise in the order) data objects, I haven’t found a way to display the Customer related to the CustomerOrder related to the OrderItems all in the same page in the admin interface. The last time I looked at CatWalk in TurboGears, it seemed like it was a pretty slick database browser, but not really geared for turning end-users loose on.

Related link: http://google.blognewschannel.com/index.php/archives/2006/02/09/privacy-experts-…

The new Google Desktop has privacy watchdogs barking. Enough complaining — what’s your solution? I offer a couple information condoms.

These are pretty simple-minded ideas, yet they each have their merits. They are based on how a search index is an abstract of a document’s contents. Somebody smarter than me should be able to hatch something better and put this issue to bed.

Loose Word Order

A page of words can say a lot, until you randomize the words. For purposes of search, it is enough to know what page or document contains my search query. So create an index that treats word order very loosely. I won’t get readable snippets of text in search results, but I wouldn’t mind. How about a thumbshot, instead?

This script of mine randomizes the text on web pages, to give you an idea of how effective this obfuscation is. It chunks words using block-level tags:

Index Word Hashes, Not Words

If that’s not enough, then consider hashing each word before entering them into the index using a one-way hash. Be sure to stem them, first. When you go to search this index, stem and hash your query. Salt your hash or get as fancy as you want. This way the server hosting your index really has no idea what you’re storing.

“Don’t Use It”? Not Good Enough

FWIW, my $0.02 on how to solve the remote privacy problem. Shoot them down, invent your own, but please let’s talk about a solution to this issue. “Don’t use it” isn’t good enough. I want darknet/p2p search!

Related link: http://www.techcrunch.com/2006/02/08/google-desktop-new-version-tonight/

This is exciting, at least from how I see it.

I wonder if Google will provide p2p downloading a’la FolderShare, next?

I think Microsoft (or Apple) has much to gain as the Internet begins to include ‘peers’ more closely in its web. As it does, then it will all come back to the desktop. Assuming Google doesn’t own the desktop by then.

Also see: hello.com.

* Update *

FolderShare is already all over this. See:

• FolderShare + MSN Desktop Search
• Google Desktop Plug-In for FolderShare

Also see my new entry:

Data Condoms: Solutions for Private, Remote Search Indexes

Sorry about that imagery.

Related link: http://pyatl.consultracker.com/

The Python Atlanta “Meetup” group has been using Meetup.com for quite a while now but we’ve recently decided to discontinue using them. I had suggested using something like Upcoming for scheduling our meetings. With a week until our next meetup (he created this last week), one of the guys in the group came up with this, which I think perfectly fits the bill for what we were needing: scheduling and reservation management. I’m sure if we wanted to, we could expand it a bit to handle group email announcements, but we have a Google group for that. I was pleased all around by the initiative to create this site, the usefulness of it, and the willingness to host it for the group. Thanks, Rick!

Related link: http://gnso.icann.org/issues/whois-privacy/prelim-tf-rpt-18jan06.htm

The conflict between the open Internet–the one we all love for its
small users and free speech–and the commercial Internet–with high
barriers to entry–continues. The conflict’s back this week in
ICANN,
which came into being in the midst of the conflict and has been
subject to its vicissitudes for over seven years.

A long-standing debate over the use of WHOIS data is coming to a
head. It seems like a small, fussy issue, but it’s significant. If
you want your own domain name, you have to provide contact
information. This information, called the WHOIS database, is currently
open to the whole world. ICANN realizes it has to be more private, but
has been struggling with how many safeguards to put around it.

As an open medium, the Internet offers free expression to
everyone–and everyone includes those who must remain anonymous for
any reason. We’re getting used to thinking of the Internet as a medium
that gives users unprecedented opportunities to spill the beans about
themselves: the results can be seen all over the popular social
networking and photo sites.

And the maturing Internet puts more and more pressure to attach
identifying information to participants. Commercially, of course,
tracking users offers a lot to vendors and makes it possible to trust
whom you’re dealing with. Even the free-wheeling world of commentary
is creating its own reputation systems. If people know who you are,
you can start to rate the things you like; you can start to be taken
seriously and to have others take seriously the things you care
about. This can be empowering.

But just as important, the Internet also offers the crucial
opportunity to remain hidden, while revealing information for which
governments or other powerful actors could desire retribution. So far
as WHOIS goes, publicizing contact information has risks ranging from
unsolicited commercial email to harrassment to government persecution.

Do you need a domain name to express yourself? Why not just get a
blog? Some initiatives call for more than a URL hanging off of
somebody’s else’s server; a site may deserve its own domain name. You
can get a telephone number without revealing your personal information
to the public; domain names should be the same. This is part of the
right to communicate on the Internet.

The debate at ICANN seems almost absurdly narrow. On the one side are
defenders of noncommercial interests, along with the people
responsible for actually running the system (the domain name
registrars and registries). They want contact information released
only for narrow reasons required for maintaining domain name service.
On the other side are large trademark holders and their
representatives, who would like information released for “technical,
legal or other issues related to the registration or use of a domain
name.”

The trademark holders are placing people at risk just to save
themselves a couple steps when they have a legal problem. There are
plenty of laws and mechanisms to facilitate the take-down of
information that is displayed illegally (because of trademark
violations or other infractions) and for discovery and legal action
against perpetrators. The registry can simply move a domain name entry
and stop pointing queries to the associated systems.

Therefore, it is a dangerous and gross overextension of WHOIS to allow
it to be used for nontechnical disputes The argument used by the
trademark holders is cynical: they base it on the excuse that these
risks were not recognized several decades ago when WHOIS began, in a
age when the numbers and types of Internet users were radically
different from today.

It’s also ironic that ICANN (at the behest of the U.S. Commerce
Department, in the memorandum bringing it to life) created the system
of domain name registrars, an unnecessary extra layer between the
holder of the domain name and the registry maintaining the
information. The registrars present a natural bulwark against the
release of information; a built-in buffer between vulnerable users and
those who want to harrass them. And now ICANN may breech this
security.

Let’s enjoy our growing public information resources on the Internet,
but keep it safe for people who want to stay in the shadows as well.

Related link: http://technology.guardian.co.uk/news/story/0,,1699156,00.html

From How I stalked my girlfriend:

For the past week I’ve been tracking my girlfriend through her mobile phone. I can see exactly where she is, at any time of day or night, within 150 yards, as long as her phone is on. It has been very interesting to find out about her day. Now I’m going to tell you how I did it.

It’s shy on details, but it says enough.

Link courtesy of tech.memeorandum.com, a place to find (some) Slashdot stories before they appear on Slashdot.

<Update>
I would have contacted Cingular about this, and given them time to fix this before talking about this publicly, but Cingular has already gained negative press about this for MONTHS (see http://www.google.com/search?q=cingular+voice+mail+spoofing.) People have been exploiting this vulnerability since a while now. The aim of this post is to help those who are Cingular customers protect themselves from this issue (see last paragraph), and possibly help contribute noise into this problem so someone at Cingular escalates this issue. This should have been fixed months ago.
</Update>

I purchased Spoofcard credit last night. Spoofcard (and many other services like it) allow you to spoof your Caller-ID information. In addition, Spoofcard also allows you to change your voice, and record conversations. I tried calling a few friends to make sure it worked, and it did. They were quite surprised and confused at first, but got a kick out of it when I revealed my identity (after joking around for a few minutes.)

This morning, I called a friend who has a cell-phone from Cingular. I used Spoofcard to spoof his own Caller ID. He wasn’t around to pick up the call, so I was forwarded to his voice mail. The Cingular voice mail system trusted the Caller ID information - it assumed it was my friend (using his handset) checking his own voicemail, and allowed me to access all his voice mail messages. I was quite alarmed, and immediately notified my friend. I also tried this with a co-workers cell phone (with his permission), and it worked.

Gaining access to cell phone voice mails via Caller ID spoofing is nothing new. Many voice mail systems have been known to be vulnerable to this. For example, a few months ago, when I was setting up my T-Mobile voice mail, I had to dig around for the right option in the voice mail system to force it to ask my for my password when I call the voicemail system from my phone. T-Mobile recently upgraded their voice mail system to encourage this behavior. However, I am alarmed Cingular has not patched this.

This doesn’t work with T-Mobile and Sprint. Their voice mail systems seem to have intelligence in place to recognize that the call is originating from an external gateway.

That said, if you are a Cingular customer, you might want to call your voicemail, and configure it to ask for a password even when you call the voicemail system from your handset. This should fix the vulnerability for you.

The chess bit

This weekend, I went to the zonal round of the National Primary Schools Chess Championships with the school under 9 team. It’s a very young group - half of them are six-year-olds who have a good eye for the game but haven’t been playing for very long.

The point? They had 35 minutes each to make all of their moves, meaning that a game could last for 70 minutes. In the first two rounds, most of the games were over in 5-10 minutes (with a lot of banging the clock!). They were spending at most a couple of seconds on each move - and were missing lots of things that they were capable of seeing if they took the time to look.

Chatting with the team after losing 4-1 in round 2, I realised that saying “take your time” was worse than useless. If you tell a kid that but don’t try to explain how to think during that time, they’ll just stare into space for a bit before mentally coming back to the board and taking approximately two seconds over their next move!

I found that the best way of explaining it was to give them guidelines about how to think. Use the time to try to imagine more than one move ahead. If you think of a move that looks good, then fine - keep your hands under the table and see what you think your opponent might do in reply. This means your blunders happen in your head rather than on the board! I demonstrated what I meant by “thinking” out loud about a position from a game where half of them wanted to make one move and half of them wanted to make another. I showed them how I would go about deciding which of the moves I wanted to make.

The Head First bit

Many people are stuck in the “I don’t get it” rut when it comes to learning. It’s a horrible place to be. Someone gives you a problem and you spend the same couple of seconds thinking about it as the young chessplayers take over each move. Except it’s much worse because you can’t transfer the tension to someone else by playing the first move that comes into your head. You have to sit there feeling that you don’t get it, and that you haven’t a clue what to do next.

From the very start, I’ve looked at the Head First books to be first and foremost about how to think so you can solve problems by yourself. The actual content is secondary to this. So instead of just showing you the right answer and the right way of doing something, we look at working out how to go about it - how to think - the bit that most books miss out. As with the chess advice, we do this with a mixture of guidelines and practical examples.

It worked at the weekend - the team won the rest of their matches, finished second and qualified for the national semi-finals.