Related link: http://intertwingly.net/wiki/pie/RoadMap

There’s been a lot of excitement recently around the Echo weblog format (formerly known as Pie), and hopes that a fresh start will help the weblog community move past the infighting that’s snarled RSS. I share some of that hope, but also recognize that RSS isn’t just complicated because of personalities - it’s complicated because there are some genuinely hard issues underneath it.

I’ve done my best to stay out of RSS, though I did edit (thankfully not write) a book on it. I’m not known for my reluctance to get out the flamethrower, but the RSS flames have been extra-hot for a long time, and I don’t have that much asbestos to spend on it.

There’s a lot more to fights over RSS than personality. The questions that RSS answers are pretty tough questions, not to mention that the list of questions has changed and expanded regularly over time. Different people come to RSS with different expectations and different hopes, and every now and then someone throws a platypus into the mix. RSS has always included more than channels and items, but the variety of things it covers has grown over time. Some people want RSS to provide full XHTML content or access to multimedia features, others want much less or much more.

Creating interchange formats that make a lot of people happy is a tough project. Extensibility and modularity are important for balancing these kinds of needs, but they often add overhead (XML namespaces, RDF) that some people can’t stand. Choosing which pieces go into a core and which pieces don’t is a difficult problem. Some of these problems are issues that affect most standards projects, but RSS has an enormously diverse audience, with perhaps too many knowledgable vendors and consumers (over 100 on this page alone). There are a lot of cats to herd here, and “forward motion” still offers one hundred and eighty degrees of possibility.

I wish the particpants luck, but I plan to stay well away from it. I suspect some people will be glad for one fewer likely dissenter during the process, but it’s really only for my own sanity. At least they did choose a different name. I’m glad about that.

Can Echo escape recent history?

The wiki discussion that Sam Ruby began just over a week ago to develop a common syntax for syndication, archiving and a publishing API continues full throttle. We have a name – it's Echo. Numerous people are in support of the roadmap. SixApart, Blogger and LiveJournal have all said they support this effort. Even logos are appearing. Excellent.

Keeping up is dizzying. Shelley Powers makes an excellent post to summarize the action so far. Stay tuned.

There are moments when if people compromise something great can happen and if they don't the opportunity passes. [Dave Winer via Joe Greggorio]

Could it be possible to reach general consensus on a syndication format and common publishing API for all? Could we put an end to the morass of pointless political posturings, egotistical rants and FUD slinging that have plagued RSS and Weblog APIs?

Sam Ruby is hosting a wiki to experiment with exact this notion. Today he writes:

A week ago I quietly introduced a wiki to discuss the anatomy of a well formed log entry. It got a lot of interest And a week later, it is still being actively developed.

Wow. It seems that I am not the only one desiring a bit of forward motion in this area.

Amen to that Sam. I'm quite thrilled to see this discussion has legs in the form of tangible progress and general consensus. Hitting the reset button and taking a step back seems like the only sane option left.

Mark Pilgrim (Sam's partner-in-crime in developing the RSS Validator) writes we've all collectively learned so much in the past few years from the real-world successes and failures of syndication formats, archive formats, wire formats, and all the rest of the tech-behind-the-experience of weblogging. Most of that wisdom has been collected on Sam's wiki in the past week. I think the time is right to apply this collective wisdom, to get a fresh start, with a new format.

Having written on this topic in the past year at great length both here and on my personal weblog, I have to agree and fully support the roadmap for the next phase that has been posted.

Related link: http://www.cranesoftwrights.com/schedule.htm#pfux

XSL-FO combines an enormous (and verbose!) vocabulary and lots of possible combinations. In our second day of XSL-FO training, the potential number of interactions is seems ever larger, as does the number of cases where repurposing features (especially empty blocks, and footnotes) to produce results which aren’t immediately obvious.

There’s a strange kind of genius at work in the page-layout systems of XSL-FO, demanding that stylesheet writers specify the logic by which their documents operate. The XSLT/XSL-FO combination lets developers create enormous logic puzzles which lay out pages rather than the more familiar (to me) process of human interaction in applications like PageMaker, Quark, InDesign, and FrameMaker.

While it’s sometimes possible to leap into the XSL-FO instance markup, there’s still (deliberately) a large layer of indirection between that markup and the final rendered product. Working with XSLT stylesheets which generate XSL-FO, which is for the most part the preferred approach, adds another layer of indirection. While it’s certainly possible to tweak stylesheets to render a particular document, I have to admit that I really miss the cascade of CSS and its much easier element-by-element tweaking capabilities. There are best practices for creating tweakable (well, modularized) XSLT stylesheets, but you’re much more at the mercy of the creator of the stylesheet with which you’re interacting.

I have a hard sitting through five days of anything, especially huge volumes of technical information, but Ken Holman has done a remarkable job of keeping this interesting. He combines enthusiasm for the training with a long list of projects outside of training, and his work beyond the training made the examples a lot more compelling. I can’t imagine teaching this for five days solid, but he seemed just as ready to explore the details of keeps and breaks this afternoon as he was to explain the overall architecture on Monday.

I came to the training as a skeptic of XSLT and especially XSL-FO, and I haven’t experienced a massive conversion. At the same time, though, I have a far better perspective on what these tools are good for (and not good for), not to mention a much deeper understanding of how they fit together. I’ll be taking what I’ve learned back to O’Reilly, where I’m hoping to use my improved XSLT skills for a variety of editorial tasks, and see if I can fit XSL-FO into my process so that authors can get a better snapshot of what their work will look like.

After five days of this, I’m pretty tired, but very happy. It’s a different feeling than I tend to get after a conference or reading a book, and it’s one I’d encourage more people to enjoy.

Does training make a big difference for you?

Related link: http://www.cranesoftwrights.com/schedule.htm#pfux

After three days of intensive XSLT, the training has changed over to Formatting Objects (XSL-FO). Instead of thinking about how best to push or pull information through my logic, I’m thinking about how to make boxes line up, trying to remember my CSS properties (since they mostly work in FO), and thinking about how to integrate the logic of FO with the logic of XSLT. Fortunately each was designed with the other in mind.

XSLT, which can seem pretty alien when you’re trying to convert from one XML vocabulary to another completely different vocabulary, works a lot more easily in the context of document-to-document conversions. While there are still times in document processing where document order isn’t what you want, converting from one document in a particular order to a formatted representation of the same document in the same order is pretty simple.

(Come to think of it, that’s not so far off from what I’ve done with CSS in the past, but dealing with paginated regions is tricky enough that having XSLT to handle odd cases and insert extra information is sometimes convenient, even when we’re processing in document order.)

FO’s multi-stage processing model is definitely a challenge for my brain, since I need to keep track of how my XSLT stylesheet generates an instance tree which then becomes an FO tree, which is then refined into a refined FO tree, which then becomes an area tree, which is then rendered. (Processors can combine those steps internally, but I still need to keep this pipeline in my head to get results that resemble what I want.)

Fortunately, Ken Holman has enough diagrams, examples, and good humor to keep us moving forward through the maze. It’s a maze I’ve wandered before, but I suspect I’d have to do this repeatedly for the best practices to seem natural. Exercises during the day help a lot, at least in part because they’re an opportunity to get things (sometimes horribly) wrong. As the current slide says, “Risk of overlap is borne by the stylesheet writer” - there are lots of risks like that, not to mention the risks of typos in 35-character attribute names. And then there are those ghost-list labels for ghost-list items…

The niftiest things I’ve discovered today are the combination of starts-row and ends-row properties in tables, the conditional nature of their processing, and a nice proportional-column-width() function. It’s been a long time since I’ve wrangled tables on a regular basis, but I could feel old problems fading away.

Have you had a chance to play with XSL-FO?

Related link: http://www.cranesoftwrights.com/schedule.htm#ptux

In day 3 of XSLT training, we’ve reached the point where a lot of divisions and the reasons for them are becoming clearer. The division between processing based on labels (XSLT 1.0’s approach) and processing based on types (new to XSLT 2.0) is one of the most profound differences, but other such divides abound.

We’ve spent a lot of time here exploring how traditional programming assumptions can lead developers down the wrong path in XSLT. As Ken Holman put it cheerfully, “I wanted you to trip over some of the non-obvious problems, so you can get out of a rut.” The basic problems seem to boil down to different expectations about what the programmer should be doing and what the program should know.

In the particular example we were working on, programmers could create their own mechanisms for tracking chapter numbers, effectively using variables and treating the processing of the document like the processing of an array. While it worked, it was a lot more complicated than the XSLT which used the XSLT processor’s understanding of where the stylesheet was in the document. XSLT was already doing the work, tracking labels and hierarchies so the programming we added was duplication, serving primarily to make our inner programmers comfortable and putting an extra layer between our code and the actual state of processing.

(This also helps explain why XSLT isn’t suitable for all data-processing tasks, as its strong foundations in tree structures make it less agreeable for working with structures, like graphs, which may be represented as XML but don’t necessarily enjoy the hierarchy imposed by XML.)

The other divide, appearing regularly in discussions of XSLT/XPath 2.0, is the divide between processing labeled hierarchical structures and processing data of a given type. Labeled structures are explicit in every XML document, but types are generally kept in separate documents or processes - W3C XML Schema documents in the case of XSLT/XPath 2.0. While schema awareness adds some extra capabilities to XSLT processing, it also adds a layer of indirection. Programmers who like this kind of indirection will no doubt be pleased, but those who came to markup for its relatively direct approach to labeled information are often infuriated.

It’s hard to tell where XSLT will go, given the strange mixture of markup-centric assumptions in its processing model and the layers of type-centric assumptions being added in version 2.0. Rather than a clean division, where markup tools are specific to markup assumptions, XSLT 2.0 challenges developers with a combination of a core language based on labeled hierarchies and features based on typing.

Today has been especially enjoyable for me, as I have the markup perspective and little patience for the typed programming perspective. (I’m a Java programmer, and I still find types only mildly useful - I know, I know, I should switch to Python.) This morning’s discussion has illuminated how that divide plays through XSLT, and how XSLT 1.0 will likely remain my tool of choice for the kinds of XML processing problems for which I find transformations appropriate. As Ken Holman put it, addressing the more traditional programmers in the audience:

“Hopefully by the end of the day, you’ll feel more comfortable with these native XML structure processing approaches.“

There’s one other clash worth noting, though primarily to note how it’s less troubling in this context. When I first encountered XSL 1.0, the obvious clash seemed to be with Cascading Style Sheets (CSS) - both discussed formatting and had some surface similarity, but very different architectures (CSS annotation, XSL transformation) and syntax. At this point, though some animosity between the two camps remains, it’s fairly clear that the foundations of the two are similar. They’re both declarative, they both rely on the labeled hierarchies of the documents on which they operate, and they both build on understandings shared by markup communities. Relative to other clashes, that one is small.

How diverse should programming styles be?

Related link: http://www.cranesoftwrights.com/schedule.htm#ptux

XSLT has long been a tool I’ve used, but as little as possible. After two days of training, I’m both remembering why that is and learning ways to work around the aspects of the language that have bothered me for years. Getting a more complete understanding of XSLT’s context and intent makes a huge difference.

XSLT is, in many ways, alien territory to “classical programmers”, whether those programmers came from procedural or object-oriented styles. Variables don’t vary, but they are namespace-qualified. Whitespace doesn’t matter most of the time, but it creeps up in unexpected ways that foil seemingly obvious ways to do things. Scope issues frequently confound approaches that briefly looked promising, though recursion often holds a solution. Parentheses aren’t nearly as harmless as they are in the languages I’m used to, like Java. Meanwhile, there’s frequently more than one way to do things, and the differences, if any, are often subtle.

While this might sound daunting, it’s actually made the training a lot more interesting because of interactions you can have much more easily in person. A lot of the fun in this class has been Ken Holman’s phrasing problems in terms which make great sense to programmers, but which aren’t the best way to solve the problem in XSLT. After we visit the possibilities, we find a better answer. While much of the course is lecture, exercises and general interplay give us all (myself included) a chance to make mistakes both in private and in public. These aren’t the kinds of interactions you can have in a half-day tutorial, never mind a 45- or 90-minute presentation. By the end of this, we’ll have experienced firsthand - and solved - most of the common (and mysterious) problems that arise in XSLT development.

My fellow trainees are an interesting mix of Canadians and Americans, docheads and dataheads. Only two people (of ten) are using DTDs for their XML, but it doesn’t seem that everyone’s rushed to embrace the latest in XML technology, as almost no one is using XML namespaces in their work either. (I use them about half the time.) Some attendees have prior XSLT experience, but everyone arrived knowing XML already.

This course is explicitly about XSLT 1.0 and XPath 1.0, but XSLT 2.0 and XPath 2.0 have come up a few times. Sometimes it’s been in the context of problems which those new specs will solve better, but at least as often the future is dark, cloudy, or unclear. It’s hard to imagine covering those mammoth specifications in this kind of detail - I have a hard time imagining any but the most dedicated sitting through the ten days of training those specs would seem to demand. Larger specs create problems that go way beyond the scope of the problems they were meant to solve.

Some of Ken Holman’s choice tidbits are well worth repeating, though they may make more sense to you if you know XSLT.

“Many new users don’t realize how little XML has done. And maybe that’s why it’s so successful.” (To be fair, he meant that XML has succeeded by doing little, not that users didn’t get it.)

“// is the biggest waste of [processing] time in stylesheets… This isn’t to say that // doesn’t have a role at times.“

“Not intuitive, and not in the REC.“

Perhaps the most important quote I’ve heard from Ken, the one which most clearly tells me the boundaries where XSLT is useful or not, is:

“All we’re doing is building a result tree.“

If I have a problem I can (reasonably) describe as result-tree building, XSLT is a good candidate for the work. If that’s a stretch, there’s probably another tool that’s more appropriate. We’ll see if my brain can keep up after a few more days of this, but right now I’m very pleased with a better understanding of problems that have troubled me for years.

Ever build a result tree?

Related link: http://www.cranesoftwrights.com/training/crane.htm

It’s been a long time since I’ve attended a technical training session. I look in on tutorials at conferences, and I’ve given plenty of training sessions, but right now I’m getting back into that feeling with a five-day XSLT/XSL-FO session from Crane Softwrights. The results aren’t what I expected - they’re better.

So why am I taking an XSLT training session? I’m not known for my fondness of either XSLT or XSL-FO; the kindest thing I’ve ever found to say about them is that “I’ve come to terms with them.” I use XSLT on a regular basis for my work at O’Reilly, both manipulating the DocBook files we use for books and for the writing work I’m doing myself. I’ve used XSLT (with some EXSLT trigonometry extensions) to draw SVG maps of forest sections. Despite that kind of practical work, there’s still nothing like an intensive course to get your mind refocused.

I spent much of the morning reminding myself not to boil over (much), as Ken Holman has been explaining a lot of the “why” of XSLT and XSL-FO - and I’ve disagreed for years about those design decisions, often vehemently. At the same time, though, Ken’s explanations have done a great job of forcing me to think about my positions and how they play through my work, with the result that my handout is now covered in yellow post-it notes. I’ve asked a few fairly cranky questions, but fortunately Ken is gracious enough to answer them without increasing the overall level of crankiness in the room. I expect that’ll be less of a problem as the training moves into more specific fields.

While I wouldn’t have imagined saying this a few years ago, it’s also pretty clear that XSLT 1.0 and XPath 1.0 are about the highest level of abstract description of XML documents which pretty much everyone finds useful, regardless of their project type. Although it’s certainly possible to process XML without XSLT - and most of the code I share with the world is in Java - it’s hard to have a conversation at an XML conference without at least the prospect of XSLT coming up. Specifications which have come later - including XSLT and XPath 2.0, as well as XQuery - try to solve many more problems, but in so doing, have created a group of people who aren’t willing to go near them, largely because of their strong connections to W3C XML Schema.

This is just day one of five, but I feel like I’ve already had a (necessary) kickstart to my thinking about the markup I use every day. It’s hard to express the value of being forced to closely examine tools you’ve used before, especially with a group of people learning to use them. Different questions come up in training sessions than on mailing lists primarily composed of experts, or even at regular conference sessions. Conference tutorials, while valuable, are often just a morning or afternoon, and rarely have the leisure to go beyond a day. Especially in these times where specifications are now larger than the books that explain them, sitting down with a subject and a group of people seems like a necessary supplement to my library of books and email archives.

How does face-to-face training complement (or compete with) books for you?

Updated: I’ve now published a
revised version of this article.

I love my Sony Ericsson P800 phone. Here are some applications for it that I just can’t do without.

They’re all freely usable, and most are open source. (I still can’t get over the way that in the PDA market you can charge $10 for a glorified Hello World program.)

SymIRC. A neat little IRC client. I’m ashamed to say that on more than one occasion I’ve appeared on IRC while out at the pub. GPL licensed.

Ogg Vorbis Player. I can fit 2 CDs encoded as 64kbps Oggs on my 64MB memory stick duo card, and still have plenty of room left for taking pics. The developer plans to release as open source.

SMan. A handy tweaking tool. Most usefully provides the feature SonyEricsson have left out, ability to send multiple files over Bluetooth in one go. You’ll have to navigate round the web page a bit to find it, unfortunately. Some source code is available, but there is no explicit license.

Opera.
Makes a much nicer job of browsing than the built-in browser.

MobiPocket eBook Reader. Always neat to carry around reading material for the next time you’re unexpectedly stuck in that airport.

And finally, be sure to upgrade your phone’s firmware if it’s not the latest. I took mine to a service centre (there’ll be contact details for these in your P800’s manuals) and got it upgraded from R1D to R2F. The things I noticed right away were that Bluetooth transfers were much quicker, and that stylus input seemed more responsive. One word of warning, though, the phone’s memory gets wiped in the upgrade so don’t forget to back up your data!

Got P800 or Symbian tips? Let us know.

Related link: http://simon.incutio.com/archive/2003/05/27/funWithLinks

Do you know of any linking-related XML+CSS tricks?

I’m taking rolling notes from the Clickz Weblog Business Strategies Conferencehere. I believe this is the first event dedicated to weblogging in the business. Should be interesting.

Related link: http://yaml.org/spec/

While I have a reputation for being impatient with people who just see XML as a serialization format, I’m happy to report that I’m quite pleased with a project called YAML that seeks explicitly to come up with a better text-based serialization format than XML.

Long ago, YAML was “Yet Another Markup Language”, part of the activity which emerged from the SML-DEV mailing list’s work on XML simplification. They’ve changed the name and sharpened the focus on serialization, wisely severing their ties to markup practice per se, which covers a much broader set of issues.

YAML keeps perhaps the largest benefit of XML over its many competitors, a sort-of-human-readable Unicode-based text format. Beyond that, however, YAML has a much tighter focus:

[YAML] is both a human-readable data serialization format and processing model…. YAML document streams encode in a textual form the native data constructs of modern scripting languages. Strings, arrays, hashes, and other user-defined data types are supported. A YAML document stream consists of a sequence of characters, some of which are considered part of the document’s content, and others that are used to indicate document structure.

YAML doesn’t look much like XML, though it comes up regularly in XML contexts. Kendall Clark noted some of the difficulties YAML has had - and its prospects for emerging from XML’s shadow - in an article last year:

In rummaging around for a plain, concise description of YAML, I kept stubbing my toe on a felt need to define it by referring to XML in some way. That was a mistake. YAML stands on its own very nicely, even if its most immediate point of contrast is XML.

If data serialization is your primary concern, take a close look at YAML. It doesn’t yet have the tool support that XML has, but it may be a much better fit for the job. YAML seems to me to combine XML’s structured text foundation
and ASN.1’s programming orientation, perhaps giving programmers the best of both of those worlds without their complications.

Can YAML solve data serialization problems, and let XML focus on markup?

Related link: http://norman.walsh.name/2003/06/01/xmlnotoo

Norm Walsh provides yet another excellent perspective on the differences between XML and objects.

Norm describes himself as “an object oriented kinda guy”, and he also knows markup very very well, having led the DocBook effort for years, not to mention his participation in the XML and XSL activities. He’s got a clear perspective on where things do and don’t fit.

The only thing I can think to add is that XML is pretty explicitly a rejection of an aspect of OO practice that Norm touches on only briefly: encapsulation. Everything accessible all the time is pretty clearly a hallmark of XML work. You can hide things if you want to, but it takes a lot more effort.

Why can’t objects, tables, and markup all be the same?

Related link: http://oscom.org

This past week, I attended the 3rd Open Source Content Management Conference (OSCOM). I was commuting between the conference and O’Reilly’s Cambridge office, so I didn’t see everything I would have liked, but there’s definitely a lot worth exploring at OSCOM, and it helps that MP3 versions of many sessions are available!

Slides for many sessions are posted, and MP3s are available for some sessions as well, but it’s hard to gauge the overall feel of the conference from individual sessions. OSCOM was a mix of very different groups of people with common interests in the very wide field of content management. There were RDF developers there and people who had no (or negative) interest in RDF. There were bloggers there and people who had no interest in blogging, and Web Services developers and developers with no interest in Web Services. Some people were managing content on a grand scale, others on a tiny scale.

That diversity held a wealth of riches. The first session I attended, Jo Walsh’s Collaborative Mapping on the Semantic Web, explored a use of RDF for mapping that is very different from the traditional “map company sold me this map”. In Walsh’s work, individuals provide descriptions of places and the connections between them, growing a map out of many people’s different perspectives and experiences. The bottom-up approach was a refreshing change from a lot of the “information must be backed by authority” approach I hear all too often. (Sam Ruby also has some commentary on this aspect.)

I missed Dave Winer’s keynote the next morning, but Aaron Swartz took notes on IRC and Dave has since finished his thoughts.

Switching gears again, I heard Greg Stein’s talk on WebDAV, taking a closer look at how protocol plumbing can make a huge difference in how we manage information. I have to confess that I’ve known about WebDAV for years, and haven’t done nearly enough with it, so I’ll be taking a closer look. As it was most everywhere at OSCOM, metadata was an important (and tricky) issue.

The next two sessions I saw, Justin Ross on Bebop and Brian Carroll on Managing Change in Web Services, both brought home how rapidly problems can become complex when multiple systems are connected. In Bebop’s case, keeping multiple browser window interfaces separate added complexity. Web Services face the same problems of change as other projects, combined with the expectation that those services be managed.

George Dafermos emphasized the open source side of the conference, taking a hard look at the position of open source in the larger context of politics, business, and their intersections. I’ve been concerned for years about the requirement that open source be strictly open to all comers, and worry that my code might well end up serving purposes I vehemently oppose. While XML processing tools (my end of the programming universe) may not hurt anyone directly, I don’t enjoy the prospect of them being used by governments tracking dissidents, for instance. While I’m not sure that the Greater Good Public License is the answer to this problem, it’s a conversation worthy of far more discussion than it’s had so far.

The next day opened with Jon Udell’s keynote, Do the Simple Things (summary). Jon focused on maximizing the metadata we already provide, in things like HTML titles and URL structures. It was kind of a change from the on-the-edge work that seemed to fill much of the conference, but it makes good sense to make the things we already have convey as much information as possible.

The last session I managed to see before the seven-hour drive home came from a presenter who lives about five miles from my home, Chris Wilper of Cornell’s Digital Library project. Although technical problems kept him from showing his slides, he explained how the Fedora project is dealing with the (seemingly infinite) challenge of archiving the changeable information that appears on the Web, as well as the role of Web Services in that process. Fedora’s work is content management of a different sort, a good reminder that publishing information is just one aspect of content management.

I’ve only captured a tiny amount of the ferment at OSCOM here, and there was undoubtedly much more in the tracks and sessions I didn’t get to, not to mention the conversations in the hallways and bars. Even the pieces that seemed least immediately applicable continue to echo in my head, a good sign that the conference did what I always hope for in a show: it made me think.