When I set out to create open source software on a PocketPC, I figured that was pretty strange, given the thoroughly proprietary nature of the tools, from the OS through the compilers, I’d be using. As it turns out, it’s not that strange - lots of people are doing it!

One of the largest projects out there is OpenNETCF.org, which offers all kinds of pieces that aren’t built into Microsoft’s .NET Compact Framework. Their library offers support for things like serial communications, bluetooth, GPS devices, wireless networking, and cryptography. They call themselves shared source, not open source, though the license looks reasonable.

While I was looking for alternatives to SQL Server CE, Edd Dumbill pointed me to SQLite, which looks like it does everything I need. (I need less than SQL Server CE had to offer, actually.) Conveniently, there’s a version for Windows CE, and classes for using it with .NET.

Finally, somewhere in there I stumbled over PocketConsole, which gives me the ever-popular command prompt. The tiny size of my screen and keyboard make the command-line approach a bit tougher, but it’s still great to have.

It’s nice to close out the year on a high note. Now I just need to get my own project properly underway.

Know of more useful PocketPC open source?

Related link: http://xformsinstitute.com/essentials/xfi.html

Google Suggest is great–the gap between ‘in a blink’ and ‘however long it takes for a browser reload’ can mean the diference when you’re drilling down to your target. It’s also good to remind people how capable browsers really are. For the right datasets, though, you can go one step further, and avoid the XMLHTTP overhead alltoghether. A good sample dataset is the full text of an O’Reilly book, say, XForms Essentials. See what I’m talking about at the demonstration page (JavaScript required).

I posted some technical details on my weblog.

What other features would you like to see in an application like this?

As part of my home network rejiggering, I wanted to start serving my music files from my desktop machine. In part, this is to save space on my iBook, and in part, it’s just because I can.

I started by googling for a precise howto for my system. While this didn’t turn up anything for Ubuntu, it did point me to mt-daapd, the multithreaded DAAP server. DAAP is Apple’s Digital Audio Access Protocol, which is what iTunes uses to share music. mt-daapd also requires Howl, a cross-platform implementation of the Zeroconf standard (which Apple calls Rendezvous).

To set up my server, I first installed howl:

sudo apt-get install howl-utils libhowl-dev libhowl0

Unfortunately, there’s no deb for mt-daapd in the Ubuntu universe. So I had to download the deb from SourceForge and install it like so:

sudo dpkg -i mt-daapd_0.2.0_sid_i386.deb

I created a directory /var/media/music for my music files, and copied some MP3s there for testing. Then I edited /etc/mt-daapd.conf to tell mt-daapd where to find music files, as well as some other details. Then I took a deep breath and started mt-daapd with its script:

sudo /etc/init.d/daap start

The admin web page (on http://localhost:3689 by default) came right up and showed me that mt-daapd was running, and had scanned four MP3 files.

For extra bonus points, I added mt-daapd to the runlevel startup scripts, so it will start every time the machine starts up:

sudo update-rc.d daap defaults

I fired up the iTunes player on my iBook and connected to the server. There it was! However, double-clicking on a song didn’t do anything.

The mt-daapd log showed a variety of errors. After scanning the mt-daapd forums without finding an exact remedy, I decided to ditch the deb and installed the latest source tarball (version 0.2.1-pre2) instead. This required installing a few more dependencies (like libgdbm-dev and libid3tag0-dev), but all of them were available in the Ubuntu universe. I configured it to install in /usr/local, so as not to conflict with the version from the deb.

To make sure that the right version of mt-daapd starts up, I edited /etc/init.d/daap, changing the DAEMON variable to point to /usr/local/sbin/mt-daapd.

That one worked. I packaged up all the music files ripped on my iBook and copied them over to my Ubuntu box. Et voila! Now I had music streaming across my home network.

But I still can’t play all my music files locally on the server, where the nice Harmon Kardon speakers are plugged in, because many of them are encoded as AAC. XMMS, my music player of choice on Linux, doesn’t seem to handle AAC encoding.

Next step: finding and installing an MP4 codec for XMMS. I’ll leave that for a future weblog.

Are you streaming iTunes on Linux?

In the past couple of weeks, the blog tools community has been abuzz about a rash of mass comment spamming and how to cope with it. The Movable Type community I am quite active in was hard hit because of its popularity and a couple of existing of bugs that launched resource intensive rebuilds even though they had been blocked or put into moderation. (A fix was just released.)

I've watched the discussion by many well meaning people as to how to stop this attack on open communication this past week or so with a certain sadness.

Initially that sadness was from the realization that many talented, well-meaning and intelligent people were putting a lot of thought and effort into handling this issue instead of doing what they do best. As I've read many of the view points and ideas to combating spam floods of their personal online property, it has shifted to the realization that freedom in open spontaneous communication means being enslaved to the constant clean-up and defense of spam and the side effects of those measures.

Open communication through comments/forums and effective sustained protection from this type of abuse are in direct conflict with each other making an ideal solution unobtainable.

A lot of ideas have been floated to confuse and trick spambot from finding their targets or making automated posts through various HTML trickery. They will all fail eventually.

The truth of the matter is whatever your browser can read, and thereby you and your readers, so can a spambot. It's all a matter of them programming in the logic to interpret what is there. Some of these measures may work for a short period of time, but eventually they will be thwarted once the spambot developer catches on, another flare up and then a new trick(s) will be needed – back to the beginning.

It's also worth noting that a spammer does not mind making mistakes. If they guess wrong what you may have done to trick their bots, they get to try again until they do. Once they figure out the latest tricks in vogue they can set their bots off to make thousands upon thousands of bogus posts while the community figures out what to do next. These unscrupulous folks are a desperate lot after all.

The most difficult road block for them to work around are ones that require human intervention and intelligence to make a post. The side effect of this being that it also requires legitimate commenters to be inconvienced stepping through this same process.

In a sense repeating the content of a skewed graphic (CAPCHAs) or answering a question like what is my last name? is along the lines of human intervention and intelligence, but not to the extent that these can't be worked around eventually since they don't match to a specific identity. Spammers have already identified a way around these test through free porn as Boing Boing reported nearly a year ago.

I thought Six Apart's attempt on marginalizing the issue of comment spam earlier this year with their TypeKey authentication service and API was laudable because it put a few of those hard to program steps in place without requiring separate accounts on each weblog or installation of additional software. It also struck a balance between distributed and localized control. Sadly it was panned by many before launch and the community at-large hasn't entirely warmed to the idea. I have to wonder if that is about to change somewhat.

I've seen the comments against implementing any authentication and login systems that say it inhibits the conversation and turns some away. I can understand that point. However I have to wonder what effect all of these automated measures in addition to new and more aggressive strains of attack are having on inhibiting those same conversations. Are these varied alternatives really any better? I think not, but admittedly I choose long ago not to enable comments on my weblog for personal reasons more then spamming concerns.

This whole matter reminds me of something Clay Shirky said during his 2003 ETech keynote A Group Is Its Own Worst Enemy that is highly relevant to the circumstances the weblog community is currently facing. (Transcribed and lightly edited here. A long, but worthwhile read if you have yet to do so.) Under the heading of things to design for he stated:

…you need barriers to participation. This is one of the things that killed Usenet. You have to have some cost to either join or participate, if not at the lowest level, then at higher levels. There needs to be some kind of segmentation of capabilities.

He continued:

It has to be hard to do at least some things on the system for some users, or the core group will not have the tools that they need to defend themselves.

Now, this pulls against the cardinal virtue of ease of use. But ease of use is wrong. Ease of use is the wrong way to look at the situation, because you've got the Necker cube flipped in the wrong direction. The user of social software is the group, not the individual.

I think we've all been to meetings where everyone had a really good time, we're all talking to one another and telling jokes and laughing, and it was a great meeting, except we got nothing done. Everyone was amusing themselves so much that the group's goal was defeated by the individual interventions.

The user of social software is the group, and ease of use should be for the group. If the ease of use is only calculated from the user's point of view, it will be difficult to defend the group from the group is its own worst enemy style attacks from within.

So, the weblog community is reaching a cross-roads. Gone are the days of open spontaneous communication through comments. It can now take measures to better defend itself from itself or watch history repeat as its cherished comments wither away under the weight of a past ideal.

Over the past year, most of the programming I’ve done has been for sytems most people would call underpowered - Apple ][ emulators over the summer and now PocketPC. I may perhaps be overly fascinated by programming in highly constrained environments, but I think there’s a lot to be said for that.

I’ve written about this before, in The Strange Pleasures of J2ME, but it’s striking me more and more that the constrained environments of PDAs and similar devices resemble the Apple ][ environment I learned on, and that maybe there’s an opportunity here.

Right now, both J2ME and .NET Compact Framework programming seem to be treated as special hard cases for which experts are needed, harder cases than ‘ordinary’ J2SE or .NET programming. Some of that is reasonable - after all, experienced programmers already understand the languages, and there’s less hand-holding available in the smaller editions. A lot of the books I’ve seen on these subjects address them from the “what’s missing?” perspective, assuming a huge amount of prior knowledge and showing developers how to make up for familiar pieces which have been dropped.

That perspective seems to me to be missing something huge. Rather than bemoaning the loss of features, why not look at this as an opportunity for people to learn programming in a very cool environment that spares them the overhead of 1000-page books on GUI design. The GUIs in these environments are small enough that it’s possible to construct them by hand without massive headaches. Data structures are generally simpler too.

In an age where gadgets seem to be more and more the rage, why not make it easy for people to create programs for their gadgets? Heck, call programming “advanced customization”, and maybe we’ll find a whole new generation of budding programmers at the gates.

(J2ME currently provides some support for this enthusiast vision because of the zero pricetag of their basic tools. Just getting started in the .NET Compact Framework requires a $550+ investment in Visual Studio .NET, something I hope will change.)

Can cool gadgets and simple environments bring more people into the craft of programming?

Related link: http://www.svgopen.org/2005/cfp.html

The Call for Papers of SVG Open 2005, the annual Scalable Vector Graphics gathering, has just been issued. 2005 marks the 4th consecutive year SVG Open is running, coming back to Europe this year as it is set in Enschede in the Netherlands, following three succesfull stints in Zürich, Vancouver and Tokyo.

It’s highly likely that topics such as mobile SVG, mapping and GIS applications, new standards in the SVG realm (JSR-226, SVG 1.2, SVG Tiny 1.2 and sXBL) and new exciting SVG-centric products will be starring at this conference. I’m probably going to at least present about my recent public project, the premier community website, SVG.org. I hope a lot of readers from the O’Reilly Networks seize the opportunity to come educate themselves further about SVG as most of the SVG luminaries will be around. I also hope that people that have not shown up at previous SVG Open events will come and enlighten us with their own SVG activities.

SVG has certainly been gaining momentum through 2004, especially in the mobile space with SVG Tiny, and summer of 2005 should be a very interesting time to reflect on the state of SVG. For more information about the conference, the SVG Open 2005 website should satisfy your every needs. See you in Enschede!

During a conversation about Web Services today, it struck me that a lot of what people find really exciting is stuff they’ve already had, sometimes for a decade. The fun part is that they often don’t know it.

I’ve been in the XML community long enough that the cycling of old material seeming new is starting to become obvious. Even a few years ago, Web Services felt a lot like a recycling of the early claims about what XML would do to liberate data, which themselves reiterated the old claims of SGML. I came in as the SGML/XML transition was in full swing, so I no doubt made a lot of those same claims.

Now, as I’m developing a small system for collecting and distributing geographic data, I’m having a lot of fun playing with new technologies:

• The .NET Compact Framework - yes, I’ve been writing some C#, the first real, non-demo C# I’ve ever done.
• MapServer and PostGIS - I’m really only getting started with these, but I need a good place to store my data and a way to share it, and since I already spent the budget on the data collection side…
• Bluetooth and GPS - My Earthmate BlueLogger gives me more data than I presently know how to manage, something I’m working on.

There are a few key pieces here that let me connect all of this up, and most of those pieces are old. I’m using Apache as the web server, both to act as the interface between my incoming data and PostGIS and as a host so Web Server can present that data attractively. NMEA 0183 lets me interpret the stream of data from my GPS without getting locked in to a particular vendor.

At the heart of the system, however, are two rapidly aging but critical components: HTTP and XML. Both of these reached the form that I’m using (1.1, XML with namespaces) in 1999. Over the wire, it could be 1999. Even beyond that, I’m only using a little RELAX NG validation - no need for any of the various official “Web Services”. These two combined make it easy to get data from Microsoft PocketPC to a open source Linux-based database and web server system and from there to whatever other system you want.

The next time you’re contemplating major technological improvements to your existing architecture, look around for spare parts you already know. They don’t need to be new. Build the core of the system on those. That might spare you the cash for some extra spiffy parts in the places people see.

Can mixing old and new keep everything new from being old again?

Related link: http://ansuz.sooke.bc.ca/software/molester/

As a followup to Ed Felten’s 15 lines of Python, Matthew Skala has written a fully operational p2p application in only nine lines of Perl.

I feel a new programming competition coming on.

Related link: http://www.oreilly.com/catalog/revolution/

I just got a copy of Revolution in the Valley: The Insanely Great Story of How the Mac was Made. Even before opening it, though, I was struck by a quote from Steve Wozniak on the back:

“It’s chilling to recall how the cast of young and inexperienced people who cared more than anything about doing great things created what is perhaps the key technology of our lives. Their own words and images take me back to those rare days when the rules of innovation were guided by internal rewards, and not by money.”

I haven’t had a chance to read the book yet, and I’m sure I’ll hear how money plays a part in the story, but Wozniak’s words really echo for me. I think there are still sectors where technical creativity is flowering, but it often seems to follow a pattern often seen in neighborhood gentrification: first the artists come in and make the neighborhood “interesting,” and then the business folks arrive and make it “respectable.” Or something like that.

I frequently share Wozniak’s concern, but I hope that there are still enough innovators out there to keep tech interesting, and those “rare days” alive somewhere. There are even some companies that manage to stay interesting and stay afloat at the same time. It’s a tough balance to maintain.

Is there a difference in the kind of innovation driven by internal rewards as opposed to that driven by external rewards (like $$$)?

Related link: http://www.google.com/webhp?complete=1

The latest beta offering from Google gives you word completion in the search input box. As you type, a popup menu appears with suggested words to search for.

It works for phrases, too. It seems to build the list of suggestions from actual text found on the web; it will present you with common typos to complete your words.

For example, typing “google su” will eventually get you to “google suggest”, but the first completion is not flattering to Google.

The FAQ for Google Suggest says it uses similar technology to the “Did you mean?” feature that offers substitutes for your possibly misspelled search terms.

Yet another innovation from Google. Keep ‘em coming!

Did you mean “google sucks”?

I’ve been thinking about building a small server for years, and getting closer to it now, but I’m trying to figure out how to avoid creating another vampire in my household.

At one point, I was running four computers here twenty-four hours a day in my office, and it greatly reduced the need to heat this room in the winter. My electric bills were impressive, even compared to the gas bills in my uninsulated house with an ancient furnace.

I’ve pushed those bills down over the years, replacing most of the computers with laptops and cutting out the servers in favor of appliances - first a SnapServer, then a Linksys network storage link, along with a few HP JetDirect boxes. The appliances still suck power, but it’s a total of about 60 watts instead of the 500 their predecessors used. I leave an iMac up most of the time I’m here, though it does at least get turned off when I travel. Oh, and I replaced the furnace and insulated the house.

I’m at the point now where I’d like to have a server again, running programs that go beyond the file and print services that were easy to replace. Mapping and database services don’t seem to come in prefab boxes, at least not cheap prefab boxes.

This leaves me with a problem. This server will have a lot of work to do for short periods of time and long stretches of complete idleness. Power management can help with some of this, turning off parts that aren’t needed, but there’s still either a significant power drain or a substantial delay when the system first gets pinged.

Options I’ve looked at include:

• Trying to do this with a mini-ITX system that uses less power overall. My initial plans should fit within its capabilities, but I’m not nearly as certain over the long term.
• Use parts designed for notebooks, like the Pentium M chips and the Intel 855 chipset. There aren’t a lot of motherboards available for this, and it is more expensive, but I’ve been very fond of it in my notebooks. The chip’s ability to select a speed (and power) based on use makes it very appealing, though more expensive per MHz.
• Build a standard system, maybe with a less power-hungry chip, and hope power management tools are more effective and less intrusive than I remember them being.

Any suggestions for striking a balance on this are welcome. It seems like power consumption is an area that the server manufacturers have explored for higher-end blade systems, but not something that’s come down to more ordinary box-building.

Oh, and I’d like it to be really quiet too.

Any suggestions for how to balance processing power with power consumption?

Related link: http://www.cs.tut.fi/~jkorpela/www/links.html

Are there any good counterarguments out there?