I wrote a recent article the other day on how Google has decided NOT to use XML for a recent project they open sourced. I received a LOT of very opinionated responses to that post. Unfortunately every one was from a complete MORON.

I just came across an article that announced Google open sourced their ‘Protocol buffers’ but decided NOT to use XML.

It is great that Google is contributing to the community and is showing the world how their system works, but I wonder if the inmates have taken over the asylum. I am a software engineer, so I enjoy the technical information, but I also ran my own company, and evolved to view problems from the business end. If you let the ‘engineers run the show’, then you get a very narrow viewpoint.

I think we have a severe ‘Not invented here’ syndrome inside Google.

My fellow blogger, M. David Peterson posted a very informative blog in relation to Jeff Atwood’s rant on XML. It is a great article, and quite a treat from Mark’s usual concise style. I have deep respect for Mark’s technical skills and he has some very valid points about the true nature of XML. If anyone has seen the ups and downs of semantic tags, it is M. There is no specific piece that I can pick apart. Unfortunately, it is also wrong. Dead wrong.

Effective participation in standards bodies involves quite specific commitment and development of expertise, it is not a generic capability that can be instantly redeployed, Rumsfield-style, to trouble spots.

For example, while knowledge of OASIS procedures may help you understand some standards admin issues, it does not give any specific knowledge that can help understanding or predicting ISO procedures. Similarly, if you are an expert in implementing a word processor, that does not give you any specific knowledge of schema languages. And being an expert in a problem domain does not give you specific knowledge on different implementation/architectural strategies for schema language standards.

So what you end up with is say 10,000 standards committees each of which can benefit from 3 or 4 different kinds of expertise. If MS has 45 staff permanently working on particular standards, they may well be well-suited for a few dozen committees each, and may be able to skill up over a year to serve on a few hundred others. But that does not mean that MS, IBM or even governments necessarily have the ability to jump into SC34 WG 1 (or whatever) and instantly be effective.

The strange attack of modesty

Several bloggers have picked up the strange comment of Microsoft’s UK National Technology Officer McKee that it had little or no experience or expertise around software standards until the company was mid-way through the process of getting Office Open XML approved by the International Organization for Standardization. Andy Updegrove has a post (which is notable for actually agreeing with a point I made) and MS’ Matusow made a response too.

Can they all be true? Can MS be inexperienced (McKee), yet also be so big and active for such a long time (Updegrove)? And in how does Matusow’s comments that, no, MS is actually big and active for a long time actually counter Updegrove’s comments?

I think the problem is of merely talking about “standards participation” as if it were a generic commodity is the problem: Please buy 100 units of standards experience now! In fact, while certainly experience in one standards body gives a better grounding in how other standards bodies might work (compared to no experience), the devil may be in the details.

Mission-critical and multitudinous

Even within an organization, participation in one technical group (e.g. an JTC1 SC) might help in participation in another group from a procedural point-of-view, because typically standards work involves diving deep into the technical issues to the extent of being able to judge and suggest alternatives and new possibilities, it is very likely that a participant in one group is not a domain expert for the new group.

Furthermore, this is all compounded when you have international standards bodies: if the largest multinationals only have enough resources to commit one or two people globally to any area in the long-term, they certainly will not have the skills on tap. This is not only a problem for MS, but for all large companies; standards participation has not been seen as a revenue generator, nor has standards skills been seen as a mission-critical asset for high-tech corporations (or governments, for that matter.) (For example, I remember asking a Google honcho a couple of years ago what their standards commitment was, and being surprised to hear they didn’t have any at that stage; individuals could participate though.)

But standards are now mission-critical.

And they are vital to competition policy: for helping form markets from inchoate messes, for helping allowing broader stakeholder participation in oligopolistic markets, and for reducing barriers in dominated markets (see my blog All interface technologies by market dominators should be QA-ed, RAND-z (voluntary) standards for more on this.)

Expertise: contribution not domination

So the real story is something like there are many of the big high-tech firms have multiple people working on multiple standards at multiple standards bodies, but none of them have the expertise on tap that they can instantly have someone up-to-speed and participating effectively on any arbitrary standards group. And this is worst for national body participation and international standards participation.

But I don’t see why this is a bad thing.

While I certainly think all stakeholders should get their act together and participate, and develop experience, the last thing that standards groups need is invasion by non-technical suits making decisions on non-technical grounds, and the second-last thing needed is participation by technical people who do not have proper commitment from their corporate masters to commit to implementing the standards, however they come out.

After a decade and a half in standards work, I know and accept that the big companies blow hot and cold on standards bodies. (This is my “trap the bear when it is in the cage” argument.) They jump ship to the bodies that will give them the best result, they try to convince people that they jumped ship because the ship was rotten and not because they were rats, and they implement a standard only as far as it fits in with their development/marketing cycle, which often means minor version changes are not implemented, or that there can be periods of stagnation. Even the largest companies find it hard to stay on standards committees when they know they have no intention to implement that standards or its updates.

Increase in idealism then pragmatism

But we are seeing a sea-change in the attitude of regulators towards standards. Aware
that so much of the technical change of the last decade has ridden on the standardized
technologies, governments and regulators are increasingly likely to want to mandate standards; however, they then immediately discover that the standards processes of
all standards bodies in the IT area are not geared towards this being justifiable against
public policy: this has been an unintended consequence of the concerns expressed by many other stakeholders that OOXML was dominated by a single vendor (it turns out that most standards were contributed by single vendors or come from committees dominated by a handful of commercial players or from organizations with direct representation by vendors.)

Sensible stakeholders are taking the pragmatic view: encourage standards adoption but not prematurely or naively, penalize non-adoption of standards, and above all, participate.

The problem I saw in the OOXML (and the ODF) standardization at JTC1 was not so much that the large corporate stakeholders (MS, IBM, Google, etc) did not have resources on tap who knew about office formats (the domain areas) *and* SC34 technologies *and* JTC1 procedure *and* ISO editing *and* national body experience, but that there were so few other stakeholders involved who did.

Certainly I think all the corporate stakeholders should be involved more. And they should have assertive representation at OASIS, W3C, Unicode consortium, SC34 and the national bodies. But all stakeholders need to identify strategic standards and make sure they have the skill sets in place to be able to participate in the boring ongoing process of maintenance, not just the headline grabbing period. One of the good results of the OOXML standardization process is that more stakeholders have put their money where their mouths were, and started to participate at OASIS on ODF, as well as on OOXML.

Participation: babies are not freeloaders, non-participants are!

One of the great disappointments of the open source movement has been the way that lazy users don’t feed changes and improvements back, but are passive recipients. And often we see open source programs reflecting the priorities of its sponsors not its users. However, the standards process (when running correctly) have procedures in place to make sure that stakeholder comments will get looked at; but just like with open source there is an enormous intertia and laziness among stakeholders to participate.

The value proposition of open source and open standards, for many organizations, is that they get something for free, but that attitude ultimately means they get something sub-optimal for free. Organizations, and governments need to consider this very strongly, who have mission critical deployments or procurement programs based on open standards or open source need to assertively, pro-actively participate in the development and maintenance efforts of those programs.

There is a great quote (I’d have to track down who from: Dan Savage?) about gay couples holding hands in public: that to some extent in order to live in the world you want to you have act as if it were there rather than waiting for it to happen outside your actions. The same is true for standards: participation is essential.

[NOTE: this was previously posted to the OReilly DEV NEWS site by mistake.]

Firstly, I, like many of you, am glad to see that Dare Obasanjo’s indefinite hiatus from the blogosphere was short lived.

Secondly, while I most certainly agree with the premise of his recent “In Defense of XML” post — which was written in response to Jeff Atwood’s most recent attempt to get everyone in the XML communities of the world to hate him ;-) — the fact of the matter is:

Jeff Atwood is right!

Now before any of the members of the XML communities (of which I am a proud member of several!) pull out their pitchforks, and before any of the XML haters of the world declare some sort of moral victory, let me be perfectly clear:

So is Dare!

“WTF?” you say? Well, let’s first take a look at a portion of Dare’s argument:

It is interesting to see the progression of free software along side the proliferation of the web. When I first started programming, I got involved with a web CMS I used in my contract work. I would write a new plugin or feature along with migrating a design to the software and afterwords, I would try to contribute it back. One time, the designer I was working with asked me to remove some of the project branding as well as a GPL notice on the login page. Some of the community found out and a rather long dialog started regarding whether or not I had violated the licensing. In the end, I came to compromise with the original author and we all moved on.

I still think I was right. I contacted the FSF about the issue and they confirmed my evaluation. My argument was that users of the site were not the same as users of the software. My clients had access to the source code and they were free to change it however they wished. I considered the client the actual users of the software. My removing of GPL licensing information from being visible in the login HTML did not limit or restrict the users freedom in any way and I continued to include the same copyright notices in the source of the page. Needless to say, while I disagreed with the author’s perspective, I had no problem coming to a compromise. The whole situation was far from heated and I personally think it was a healthy dialog for the community.

My situation made it clear there were still questions to be answered in terms of free software on the web. It seems the AGPL is one answer for this kind of problem and there are some folks actively promoting it as a means of distributing web based free software. While, I think the terms (as I understand them) of the AGPL would not work for something like a CMS, the concept of providing software over a network and considering the output as licensed has its value.

The best thing about the AGPL is that it provides a licensing option to help provide free software on the web in a similar way the GPL effected desktop software. I don’t think many web developers have thought about web applications in the same light as desktop applications. Installation challenges, database dependencies and server requirements have all made running web applications something left for developers and advanced users. Fortunately, server software is becoming common place and programming languages like Ruby and Python are helping smooth over the issues with distribution and deployment. It is good to know that as web applications continue to evolve, free software will evolve with it.

I’ve known, loved, and respected David Carlisle for quite some time now. As of today, I’ve known him for 24 hours longer than I did yesterday, yet love and respect him twice as much as I did the day before (and that’s saying a *TON*),

Teaching XSLT vs. Teaching XQuery - O’Reilly XML Blog

If you know XQuery and want to learn XSLT you need to learn about template matching, if you know XSLT and want to learn XQuery, you just need to learn a new syntax.

*YES*! The world needs more straight shooters like David (who, like David, have the credentials to back up every word that comes from their general direction), don’t ya think? (NOTE: If you think differently, that’s just because you have no clue what you’re talking about. But that’s okay, I’ll still luv ya. Mmmwwahh! :D)

Is it easier to teach XSLT or XQuery to an experienced SQL developer? My recent training experiences indicates that XQuery is easier to learn.

For the last six years I have been building metadata management systems using a diverse set of XML-centric technologies. These languages include XML Schemas, XSLT, Schematron, XHTML, XForms and most recently XQuery. And to be honest, I really do enjoy XQuery. My job as a consultant is to develop feature-rich and highly customizable metadata management systems for my customers and also transfer the skills needed to maintain and extend these systems to my customers though formal training classes as well as one-on-one mentorship.

I have found that it has been very difficult to teach XSLT to an average support person that is only doing occasional XSLT development. But teaching XQuery has been much easier for me to teach when you consider that most of my students have had some exposure to SQL. Looking back on my own learning process, I recall took me about five months of almost continuous study to really feel comfortable with XSLT. Most of this learning curve was because I had not done production XSLT development. But I picked up XQuery in just a few weeks. Perhaps this is because I was already familiar with SQL and XPath. But perhaps this is because XQuery is a little bit more approachable.

I want to note that this does not necessarily imply a poor design of XSLT or the merits of functional programming. After I did learn XSLT I became a real evangelist of its elegance and beauty. At first I was frustrated by not being able to change a “variable”. Later I realized that this restriction is what made XSLT beautiful, simple and elegant. These features keep the transforms free of side-effects. Once XSLT scripts are deployed I seldom found problems. I became enthralled by the fact that the simplicity of the language implied that XSLT custom-hardware could allow transforms to be orders of magnitudes faster than software-only solutions. XSLT may always have a place in CPU-intensive applications.

The difference in my learning time and those of my students reflects the state of our existing knowledge base: most of are already familiar with SQL. Anyone that knows SQL can take a crash course in XQuery designed specifically for SQL developers. Priscilla Walmsley’s excellent book on XQuery (O’Reilly 2007) includes a single chapter targeted at SQL developers making the transition to XQuery that I use in many of my classes. And 90% of the small support and maintenance tasks that many support and maintenance people need to perform do not require them to ever use the more complex functions and modules features of XQuery.

So with this in mind, I have moved most of my metadata registry tools away from XSLT and toward XQuery. This seems consistent with other metadata managers are doing today. There are several people now working on open source metadata management systems.

What about you? Do you have experience teaching both XSLT and XQuery to SQL developers? What is your experience on the learning times?

It’s amazing to me how seemingly complex programming techniques are really not that complex at all if you break them down into their basic Unix CLI equivalence.

Take, for example, Polyphonic C# where they introduce the concept of the asynchronous join pattern,

Continuations, futures, and whatnot - Thoughts on some asynchronous patterns - B# .NET Blog

Join patterns

If you’ve had a rendez-vous with Ada in a previous life, you’ll be familiar with the concept of join patterns (or shorthand joins). Our Microsoft Research department has done quite some work on this field already, as you can read here: http://research.microsoft.com/~crusso/joins/. It’s part of Polyphonic C#, which is by itself part of Cw and based on join calculus. We already got the LINQ inspiration from it, joins might be lurking around the corner too. There actually just two basic concepts to understand here:

* Asynchronous methods: imagine a keyword async as a modifier on methods (which would imply the method is a procedure, hence the void return type becoming redundant for those type of methods). No longer do you need to create separate versions of a method with or without the Async suffix to work around return type overloading limitations imposed by the runtime. You can think of the method as a wrapper around a task scheduling call, wrapping the entire body. Whether or not this spawns a new thread is another matter.
* Join patterns: also known as chords, is a declarative way of a WaitAll style of guard mechanism to enter a method.

Our monk would look like:

    class Monk
    {
         public async RequestCard(string to) { ... }
         public string GetCard() & public async RequestCard(string to) { ... }
    }

Basically requests for cards queue up on the monk’s GetCard operation, ready to get processed when the monk is awake. Actually this sample is convoluted and our monk suffers from some concurrency diseases (think of the cards it would hand out if there are multiple outstanding requests - any ordering guarantees without “ticketing”?) but I won’t get in details for now - it’s just (yet) another way of thinking about asynchronous processing with today’s materials in the room. If you’re inspired by join patterns, check out the Polyphonic C# paper.

Sounds interesting, and extremely cool. And I can’t wait to start playing with this feature if and when it becomes a part of the core C# language specification. But I’m not sure it’s something we haven’t been able to do for *years* already In the mean time, for those that want to begin sinking their teeth into async/concurrent programming techniques, it seems to me this same general idea can be simulated using nothing more than nohup and Unix pipes. For example, take the following function which just does some random piping operation enclosed in a loop, taking an argument passed to it as an identifier as to which operation is currently taking place, prepending a formatted date in front of it, and then piping the result to sed to perform a search and replace operation on the current day of the week,

The standard for OOXML, IS29500, was approved a few months back with 75% of votes in favour, 14% against (for a breakdown, see this chart.) Some of the National Bodies who voted no are very determined in their opposition: South Africa, Brazil and India have lodged appeals, as allowed under the JTC1 process. [JTC1 = Joint Technical Committee 1 (Information Technology) of ISO and IEC.]

The details of the Indian appeal is uncertain, but the South African and Brazilian appeals are available online.

It will be quite interesting to see how this goes. An IEC spokesman has been quoted

This is the first such appeal after a BRM process in ISO/IEC JTC 1, although appeals occur regularly in other technical committees

How is it likely to go? I don’t want to go into each objection systematically, but I’d like to suggest some things I expect may frame the appeal resolution.

Cassandra with a Cigar

Before I start, a quick comment on politics. You would expect that appeals will be taken quite seriously by JTC1.

This has been a very contentious standard, everyone is aware of the politics, slurs and noise, high emotions and the genuine problems. Certainly, it has long been the expectation that there would be dissatisfaction no matter what the result: indeed, BRM convenor Alex Brown predicted it:

Be forewarned: the ISO process will “fail”

Speaking to fellow NB representatives, it is clear that lobbying (in many directions) is intensifying as the BRM nears. It would be naïve to expect anything else, I suppose. It amuses me the degree of self-certainty both sides have (coupled with very high levels of mistrust). One corollary of this is that they both profess that the only thing that can undo them is a “failure of process”. Tweedledum believes their DIS is so good that only a “failure of process” can thwart it; Tweedledee, however, is convinced that the DIS is so deeply flawed that only a “failure of process” could allow it to become standard..

Note that Dr Brown’s point is not that the process would necessarily be perfect, nor that NBs would not in their rights to make appeals, more that intemperate outbursts would not be tolerated during the BRM—the conduct of the BRM was the topic of that blog entry— but were on the cards for after.

And I personally welcome the appeals, because they are a good chance to get more clarity on issues. But to paraphrase Dr Brown, Be forewarned: the appeal process will “fail”; these are issues where there are quite strong views about how the world and institutions should operate which underlie, and to some extent select, the particular issues.

That there will be grandstanding later does not mean that the appeals themselves are mere grandstanding.

Keep them talking

Next, I’d like to draw a couple of lines in the sand, for setting expectations.

First, if an appeal concerns something that is specifically allowed or disallowed by the JTC1 Directives, it will not succeed. For example, the JTC1 Directives clause 13.1 says

The criteria for proposing an existing standard for the fast-track procedure is a matter for each proposer to decide.

Given that, I cannot see any objection of the form “The fast-track procedure is designed for blah blah and this goes against it” getting anywhere.

Second, to see the JTC1 process as an adversarial process where things can be won on a technicality is to fundamentally mistake its nature. It is not a court of law, it is a forum for formalized discussions aimed at agreement on the text and contents of documents. (Furthermore, these documents are not laws or legislations, they are voluntary standards.) In clause 1.2 of the JTC1 Directives it says:

These Directives shall be complied with in all respects and no deviations can be made without consent of the Secretaries-General.

These Directives are inspired by the principle that the objective in the development of International Standards should be the achievement of consensus between those concerned rather than a decision based on counting votes.

[Note: Consensus is defined as a general agreement, characterized by the absence of sustained opposition to substantial issues by any important part of the concerned interests and by a process that involves seeking to take into account the views of all parties concerned and to reconcile any conflicting arguments. Consensus need not imply unanimity. …]

The interesting parts of that are first that variations are possible with the consent of the Secretaries-General; this is the kind of provision you would expect from a mature standards organization (created about 60 years ago) that has had to weather challenging standards before: a chain of authority but a process to prevent the goals of the organization being thwarted by technicalities. Second, that sustained opposition is a serious thing. And third that the response to conflicting arguments is supposed to be processes of reconciliation: you have to leave the win/lose mindset at the door and think of win/win (where no party in win/win necessarily gets everything they want in the form and timetable that they want, but they can get something.)

So from these, you would expect that even where a National Body had a legitimate grievance that it had been discriminated against during some committee work in this case the BRM, the result is likely to be not that any standard would be invalidated, but that the NB would be urged to participate in maintenance of the standard with the assurance that the situation would be monitored. (And this is especially true where the issue concerns non-normative text, or additional text, or wordsmithing, or anything that can be done by maintenance)

Messer im Kopf

Someone asked me what kinds of things would be legitimate causes for an appeal to succeed.

Frankly, I find it very hard to see how conduct of the BRM can invalidate a vote of NBs (especially as the Directives give a lot of discretion to the convenor, where there were ISO and IEC officials advising directly, where the goals of the BRM are so limited, and where the final ballot shows that the BRM’s goal of an improved text was successful.)

I think if there had been a proven miscount of the revised ballot, that would be a ground for an appeal (again, I don’t think it would result in failure of the standard, just a correct count of the already cast ballots.)

And if the endorsed text from the Editor differed from a NB’s view of what the editor’s instructions meant, that would be grounds for an appeal (but again, not resulting in a failure: probably resulting in the issue being dealt with by maintenance.) [I should say, I think ITTF would be remiss to not release the endorsed text of IS29500 to NBs as soon as it was ready, and certainly with time for it to inform NBs before the appeal deadline is closed: this has been an expectation of mainstream delegates from several different NBs.]

If there was a reasonable allegation of cartelization, I think that would be reasonable grounds for appeal. This is where some NB’s legitimate requirements had been deliberately ignored by other NBs, as a kind of gang. However I would expect this far more to apply to ballots that rejected a standard because where a standard was accepted without some material, the forum for reconciliation would be be the maintenance process.

New information about some showstopping technical flaw would be grounds for an appeal. It would have to be something fairly fundamental (for example, if the ZIP format utterly did not operate the way the spec said it did, undermining OPC and everything built on it) and something which could not be dealt with by maintenance.

If some SC was hijcked by multinational loons who wanted to standardize a perpetual motion machine, or by cannibals who wanted to standardize some headhunting apparatus, then that would certainly be a reputation problem. I am struggling a bit here: obviously JTC1 does not deal with machinery!

I don’t see how JTC1 can find reputation arguments very compelling: it will be hard for them to tease out any real reputation issues from the fake ones. I saw this week that someone just automatically added the ISO and IEC Secretaries-General to the list of people to be suspected of corruption, and this is even before they have actually been involved AFAIK! It is like reading “How to win friends and influence people” by Chicken Little.

So I apologize if I am a bit fuzzy: but again it goes to ISOs approach (as I conceive it, at least) that the purpose of the appeals is to provide forums for formalized discussions to continue. That the Directives use the language of issues getting “resolution” rather than of sides “winning” should be a central part of understanding the process.

Proportionality

A good indication of how appeals will be handled is how alleged “contradictions” were handled during the ISO Linux API standardization. That there was an almost total overlap of this with the ISO POSIX ABI, clearly overlap in application area or even details was not a contradiction. However there was a particular real contradiction, which related to different function signature. So the issue became, this is clearly a contradiction, but is it big enough to cause the ISO Linux standard to be cancelled. Clearly not. Instead it is labelled, and more discussions between the Linux and POSIX people (if they still exist!) would be encouraged.

So the lesson from that, I think, is that the idea that a small issue can be a “spanner in the works” is unrealistic. There needs to be a sense of proportionality between the technical damage and the response to it.

Fairness

I think the most challenging issue I have seen from in the appeals so far revolve around the issue of unfairness, in particular this seems to be a thing of the Brazilians. It seems to me that there need for fairness goes to the heart of ISO procedures, however, so it is quite important.

I think there is an assumption that there is some kind of “poison fruit” doctrine operating, where an earlier SNAFU would prevent later processing of the standard. But I don’t see that in the JTC1 Directives. I am certainly not saying that this would not be a consideration, just that I cannot see why it would be compelling. It would presumably be one of the technical or administrative “principles” which could be discussed under the s11.1.3 procedures.

This is because of the big bottom line here: the National Bodies have voted.

Don’t shoot the messager

I suppose this would go without saying, but I suspect it would be difficult for appeals which are really objections to the JTC1 Directives succeeding. The people who characterize convenor discretion as “making the rules up as you go along” would surely be appalled at retrospectively changing the rules! The resolution of such an appeal would be “take it up with JTC1″.

Who votes at the BRM?

As I have said, I think having legalistic rather than goal-responsive view of the administration of the JTC1 Directives is a mistake. But some people take that as a cop-out, so I would like to point out something about one of the procedural issues that has been raised by some. It goes down to the issue of who is allowed to vote at the BRM: P members of SC34 or both P and O members. (If you don’t know the difference, nations can registers as Observers or Participants for a particular technical committee. They have different rights and obligations.)

It has been pointed out that clause 13.8 says about the conduct of fast-track BRMs:

At the ballot resolution group meeting, decisions should be reached preferably by consensus. If a vote is unavoidable the vote of the NBs will be taken according to normal JTC1 procedures.

Some astute detectives trace this back to clauses elsewhere about P members voting. However, in this particular case clause 13.7 establishes who the NBs are:

NBs of the relevant SC shall appoint to the ballot resolution group one or more representative who are well aware of the NB’s position. NBs having voted negatively, whether or not an NB of the relevant SC, have a duty to delegate a representative to the ballot resolution meeting.

This is quite an important clause, for people wanting some insight into the process, because

• it is inclusive, not exclusive: it tries to get as many of the NBs at the table as possible
• it puts an obligation on the delegations to be properly prepared (which is not to say that there is any blame attached to delegates who are specialists or novices or last-minute additions: there always will be a certain number of these): I think this has some implication against claims about size and logistical problems with review and BRM proposals
• it also explains why poor NBs, if they are trying to be scrupulous with the Directives, could prefer abstain over simple negative, “yes” rather than “no with comments” (as a conditional yes). It is expensive to send a delegation.

Is Drupal on your IT map yet? Chances are pretty good that either you are shaking your head vigorously in the affirmative, or you have no idea what I’m talking about. Drupal is an open source web content management system … though this is actually a little like saying that a Jaguar is a car; it’s true as far as it goes, but the description doesn’t really do Drupal justice.

Drupal started out in 2000 as a community project in Belgium originally called Druppel. The creator, Dries Buytaert, planned originally on calling it Dorp (which means Village in Dutch), but he introduced a typo when filling out the domain registration, and liked the way that it sounded. The idea behind it was simple - build a CMS system that promoted the concept of community rather than simply being a way to store content. To do this, Drupal was build early on around the idea of content nodes (think of them as very simple documents with a title and body) and the heavy use of syndication.

Update: Okay, so I really needed to clarify my point regarding Miguel a bit better. Hopefully the updated title more clearly describes what I was attempting to suggest.

Also, as chromatic points out in a follow-up comment below,

I think you may have misspelled “Jim Hugunin”.

Point well taken, chromatic! Jim certainly deserves the credit for bringing dynamic languages to Microsoft, doing so on a foundation of openness. I believe Miguel and friends deserve the credit for breaking down the barriers from an external perspective. Jim, John, and the rest of the dynamic languages crew at Microsoft deserve the credit for picking things up and moving them forward internally.

Thanks for helping to set things straight, chromatic!

[Original Post]
I first met John Lam when I was working as the Technical Evangelist for the Windows CE (1.0) team back in ‘96/’97. I was impressed by him then in the same way I am impressed by him now…

He doesn’t give up. Nor does he give in to political pressure.

John Lam on Software: IronRuby and Rails

Perhaps even more important than all of this technical stuff is what the IronRuby project represents at Microsoft. IronRuby has pioneered a number of new processes that make it easier for other folks at the company to build and release Open Source products. What we learn from building IronRuby will be applied in other product groups to help us become more open and transparent than we have been in the past. We have a great leadership team that is willing to push the envelope on openness and transparency to create a world where both Microsoft and our customers can benefit.

IronRuby represents a beacon of change at MSFT. As does John Lam and the rest of the dynamic languages teams at MSFT.

It’s time we pay better attention to what they are accomplishing.

SIDE NOTE: Question: Could any of this have been possible if it wasn’t for Miguel de Icaza?

/methinks not. Prove me wrong.

At my day job I’ve been working on integration between different systems through creating widgets. There are a few different avenues to take. My metric was the preceived integration with the site, so using the DOM ended up as the best option.

The DOM actually makes for a pretty decent medium for communication. If you force widgets to truly be written in the DOM instead of an iframe, every widget has access to those values that are in the DOM. Sure, you could also used namespaced JavaScript to grab values from other widgets, but then you require those widget’s dependencies. When the DOM is used, you can mock the information the other widgets provide. This allows you the option of rendering the widget on the server or via Javascript. This also allows you to specify initialization details via simple HTML instead of including extra JavaScript.

Likewise, by using the DOM, you can integrate your widget throughout the page. Things like CSS can apply directly to the widget HTML. Also, a widget may not even insert visible HTML in the dom. For example, an annotation widget might look through a DOM element to attach small balloons for notes that others left. None of this is trivial, but assuming the cross domain security issues can be dealt with reasonably, utilizing the DOM as something of a data store has its advantages.

Where this concept becomes even more powerful is in light of the Semantic Web. The Simile project provides a wealth of examples on how to take Semantic Data and present it in a useful way. Eric Miller describes this as creating an experience, which conveys the true value of Semantic Data. The idea of using the DOM in my widgets is to support a certain user experience where different services are used in unison to integrate the value of the systems. As the Semantic Web continues to evolve, the user experiences are where the biggest gains will be seen.

I just read this article on where semweb technology really works well. The context of the article is searching and points out how semantic search excels at essentially performing complex relational queries over the web.

Hopefully, it is clear from some recent discussions of tools like eXist and XML focused data stores that there is a market for generic XML document stores. Conceptually, I think this kind of technology is similar to CouchDB with the difference being there is a wealth of XML libraries to help more complex processing, where CouchDB might be more limited to specific usage where JSON excels. I’m not suggesting either technique is better, but rather the idea of a document store that can be queried in more typical SQL-like contexts seems to be gaining popularity. Taken within the context of RESTful services, I think it is a killer combination.

Solution One: Put phriggin’ AdSense ads on everyones profile page already!

Do you honestly think anyone would care? And if they do, offer them a premium “Don’t put ads on my profile page, damn it!” opt-out plan. WeatherBug does it. (For weather info I can get for free by sticking my head out the window!) Why can’t you?

Oh, and don’t even think about charging me for the right to “tweet”. Imagine a world where the ideas of yester-year to impose an SMTP “stamp” came into fruition.

Actually, nevermind. Forget that. SMTP would have never existed (clarification: SMTP, in the form of sendmail, existed *LONG* before the “email stamp” tax ideas came into existence. So SMTP would have survived. The idiotic politicians who made attempt to implement this plan, on the other hand, wouldn’t have survived.), nor would it continue to exist if such a scheme were imposed.

Of course, we all seem willing to pay our mobile phone providers anywhere from a penny to twenty cents USD to send the equivalent of a tweet to our buddies mobile device. With this in mind, herein lies the other revenue streams you seem to be more than happy to neglect,

By popular demand I am reposting this entry. It disappeared mysteriously—Kurt Cagle cruelly suggested it could be something to do with my incompetence, which has the ring of truth unfortunately— so our apologies to readers.

Here is a great quote from William Hazlett’s essay On the Pleasure of Hating, for a change in pace:

The pleasure of hating, like a poisonous mineral, eats into the heart of religion, and turns it to rankling spleen and bigotry; it makes patriotism an excuse for carrying fire, pestilence, and famine into other lands: it leaves to virtue nothing but the spirit of censoriousness, and a narrow, jealous, inquisitorial watchfulness over the actions and motives of others. What have the different sects, creeds, doctrines in religion been but so many pretexts set up for men to wrangle, to quarrel, to tear one another in pieces about , like a target as a mark to shoot at? Does any one suppose that the love of country in an Englishman implies any friendly feeling or disposition to serve another bearing the same name? No, it means only hatred to the French or the inhabitants of any other country that we happen to be at war with for the time. Does the love of virtue denote any wish to discover or amend our own faults? No, but it atones for an obstinate adherence to our own vices by the most virulent intolerance to human frailties. This principle is of a most universal application. It extends to good as well as evil: if it makes us hate folly, it makes us no less dissatisfied with distinguished merit. If it inclines us to resent the wrongs of others, it impels us to be as impatient of their prosperity. We revenge injuries: we repay benefits with ingratitude. Even our strongest partialities and likings soon take this turn. “That which was luscious as locusts, anon becomes bitter as coloquintida;” and love and friendship melt in their own fires. We hate old friends: we hate old books: we hate old opinions; and at last we come to hate ourselves.

Has anyone noticed the same trend I have? We’re in this weird area of language evolution in which those of us who have been trained to think statically are beginning to envy some of the niceties provided by dynamic languages (such as implicit types) while at the same time those of us who have been trained to think dynamically are beginning to envy some of the niceties provided by static languages (such as explicit types.)

Weird.

Take, for example, C# 3.0 implicitly typed local variables,

Update: If the embedded YouTube video doesn’t render in your feed reader of choice, you can access it directly via http://www.youtube.com/v/bDsIFspVzfI&hl=en

A recent proclamation from the W3C’s Technical Architecture Group recommends against using XRIs as identifiers.

Hold up a second, XRIs?

Unless you’ve been paying extra-special attention you won’t have heard of these little critters much anyway. XRIs are “Extensible Resource Identifiers”, and are detailed in specifications that find their home at OASIS.

XRIs are essentially fancy-pants URIs that support extra features such as decentralized allocation, federation and delegation. And as such they are pretty complex machinery. Here’s a few example XRIs I pulled out of one of the specs:

xri://(tel:+1-201-555-0123)!1234
xri://@!a!b*(mailto:jd@example.com)*e/f
xri://@!a!b!(@!1!2!3)*e/f

I offer these without any real clue as to what they mean.

XRIs recently resurfaced on my agenda since reading the OpenID 2.0 specifications, whose service discovery documents include XML namespaces such as xri://$xrds and xri://$xrd*($v*2.0).

To the casual observer, it looks as though URIs have been given a work-over according to the old adage that nothing in computer science cannot be improved by adding a layer of indirection.

XRIs have risen to common attention again through being factored into the OpenID specifications, due to the desire to keep supporting inames.

The W3C’s point of view is that everything XRIs attempt to do can already be done with URIs (or their internationalized incarnation, IRIs).

I’m inclined to agree. There’s still more than enough confusion as to what a URI means, without adding more machinery. Perhaps in 5 years we might have attained widespread knowledge about using URIs — we may think REST has “made it”, but its dissemination isn’t that far and wide yet.

The web I know and love is a collection of small pieces, loosely joined. XRIs by contrast are heavy metal.

Call me old-fashioned, but the combination of URIs, DNS, and HTTP are all the web I can handle right now.

If You Don’t Need XML, Then Don’t Use It! - O’Reilly XML Blog

I think now would be a really good time for anyone who doesn’t “get” XPath to start getting it.

As an editor, it’s all to easy to spend a lot of time reading (and responding to) blogs and articles on the web, and as the editor for xml.com, I find my time is disproportionately allocated to trying to correct misperceptions and even hostile snipings about XML. One such post came yesterday, in Jeff Atwood’s excellent Coding Horrors blog, one that (my comments in this article not withstanding) is a must read for anyone who programs for a living.

In The Angle Bracket Tax , Jeff lays out what he perceives as the flaws of XML, and why they detract from the language. I could attempt to refute him point by point, but in the main, he’s pretty much spot on with regards to his particular complaints - XML is a verbose, complex, oft-misused language that has been pushed into areas where it has no business belonging (more or less) and there are far better formats for doing everything from describing data structures to writing processes to sending messages. Yet he also misses a major, perhaps even a fundamental, point -XML is not a programming language.

i had the honor of meeting Guido Sohne when I attended the Microsoft Technology Summit this last March. I feel both honored and privileged to suggest that we quickly became friends.

Guido was a good friend. Guido was a good person. I only wish I had more time to get to know him better.

He will be sorely missed.

Newsvine - Rest in Peace, Guido Sohne

I guess I am suffering from *OA overload. Trying to understand landscapes which are still developing and where religious and partisan statements happen frequently can be hard. Or maybe it’s impossible. I am seeing too many *OAs these days. This is probably a subset of what’s around:

• SOA: Service Oriented Architecture
• EOA: Event Oriented Architecture (sometime also called EDA for Event Driven Architecture)
• WOA: Web Oriented Architecture
• ROA: Resource (or REST) Oriented Architecture
• SynOA: Syndication Oriented Architecture

And of course all these *OAs are not independent. WOA = SOA + WWW + REST and similar equations can be found in a number of forums. Eveybody is fighting for his/her favorite *OA. Why not build a crawler that collects all these statements, then feeds them into Matlab, and symbolically computes the true *OA landscape?

Of course, IT is a huge market and as usual it’s all about branding and owning a brand and hopping on the right brandwagon, and it is not a mystery where all of this comes from. Sometimes I just think it would be so nice if all these *OAs were actually well-defined methodologies and we could spend less time trying to understand what the latest *OA is all about and if and how it actually introduces something new, and spend more time actually comparing these well-defined methodologies and their strengths and weaknesses.

My personal bet is on SynOA, one of the less popular *OAs. Atom (and more so AtomPub) needs a little overhaul to be less focused on time, the Atom landscape needs a couple more features such as feed query features, but then Web-style syndication is ready to become the main abstraction for information dissemination in loosely coupled systems. I have no idea how SynOA relates to ROA or WOA, but to me it looks like the right thing to do if you are thinking about Web-style cooperation, which of course is not the only possible scenario where *OAs are needed.

Anybody wants to submit his/her favorite *OA equation so that we can figure the *OA landscape out automatically?

The recently published HTML 5 draft does not change anything regarding HTML fragment identifiers. They are still limited to IDs only (with <a name=”"> as alternative for backwards-compatibility). This means that any reference into an HTML page depends on how the page is using IDs.

But wouldn’t HTML 5 be a wonderful opportunity to bring a little bit more hypermedia back to the Web? XML had XLink and XPointer. Both were failures for a number of reasons, but I am still a big fan of trying to make the Web more hypermedia-like. So why not learn from XPointer and try to give HTML 5 a more practical and useful set of fragment identification methods than just IDs?

The whole fragment identification idea is a classic chicken and egg problem. Why use them when they’re not supported? Why support them when they’re not used? We had a lot remarks like that when we worked on fragment identifiers for plain text files, but I still believe it is good to have mechanisms like that. Assume Firefox had a feature where you just moused over a paragraph, right-clicked, and then you could send an email with a pointer to that paragraph. If the receiver had Firefox, the browser would scroll to and highlight that paragraph. I am still convinced a lot of people would find such a feature pretty useful. And things would not break in another browser, users would simply not get the scroll/highlight behavior.

Markup design fascinates me. What is it that makes one format easier to use than another? Why, even within that subset of markup that uses XML syntax, are some markup languages elegant and others unreadable? When is it best to use XML, when YAML, when a custom format?

Not all XML is created equal, and I think the biggest distinction between a good markup language and a bad one comes down to whether the XML was designed as a markup language or whether it’s a serialisation of a completely different model. Practically all the XML serialisations that I’ve seen of object-oriented models, or relational models, or graph models, have been dreadful as markup languages.

My wife is a rather amazing woman. In addition to being an amazing song writer, performer and guitarist, she is insanely smart, with a heart to help. This summer she will be helping out Girls Rock Camp. Austin also has a great program called GirlStart that aims to help girls get excited about math and science. What is so great about programs like Girls Rock Camp and GirlStart is that it provides young girls a chance to be feel empowered. What excites me most about these kinds of programs is the opportunity for perspective.

My wife and I play music together and have throughout our entire marriage. I can’t tell you how much I have learned from the experience. The biggest benefit is the constant exposure to a women’s perspective. I’m convinced that men and women are simply different in how problems are approached. If you’ve ever worked on a problem for a while, only to talk to a coworker that provides an entirely different approach that is more elegant and better in all measurable ways, then you get an idea of what it is like to have a women’s perspective. It is something totally different from your own, and the vast majority of the time it is enlightening.

I recently read an article on why girls should consider a career in IT. The author mentions programming as art and it made me think about my own experience creating something artistic with my wife. I’m confident that without the perspective my wife brings to the music, the songs and band as a whole would be nothing compared to what it is now. I can only imagine what the software landscape would look like if a women’s perspective were more prevalent.

Assuming there is a wave of ladies infusing this male dominated software industry, us guy should get prepared. The first step is losing the derogatory comments. Anytime you speak of a person as though they are an object or less than a person, you are training yourself for failure because you are refusing to step up to the challenge of acknowledging equality. The next step is recognizing when you are allowing a person’s gender or differences influence how you evaluate contributions. Becoming attentive to when your internal dialog includes derogatory comments can be scary. On a personal level, realizing that you have bigoted thoughts is a tough pill to swallow. But, if you are to move beyond your own insecurities, you must be in control of your views. Lastly, practice really listening. Learning to listen is a valuable asset, and what’s more, it places the attention on what is being said instead of who is saying it.

Hopefully in the near future more women will find their way into the software industry. I truly believe it could revolutionize how software is written in addition to helping change society for the better.

Today I took some time to quickly scan through a backlog in my feed reader. There were a good number anti-XML articles cropping up. This got me thinking. What do you think of when I say “XML”? I personally associate XML as a baseline technology in a large set of tools used for describing data. For example, I think of Atom and XHTML within the scope of RESTful web services. Next up would be document formats such as DITA and DocBook. This starts me thinking about linking data and technologies such as XInclude and XPointer. As I reflect on where my mind wonders when thinking about XML, themes of linked data and document resources quickly rise to the top. What does not come to mind is WSDL, XML Schema, object serialization, configuration files, or SOAP.

What do you think of when I say XML? What kind of context does XML succeed and where does it fail?

Update: len trumps his own QOTD with these two gems. I’ll let you decide which you feel is funnier/more accurate, cuz’ I can’t decide,

I try to tell him that no element is *really* non-terminating but he gets wrapped up in the language abstraction and forgets to breathe.

.. or ..

We made this a lot harder than it has to be in the name of “just in case.”

[Original Post]
Is it really that taxing… - O’Reilly XML Blog

Most of the time when I find a programmer struggling with XML, they are a relational database programmer or an object-oriented programmer, or both. We should have lined these guys up against the wall at the beginning of the revolution, really.

NOTE: I met Jeff Atwood for the first time a month or two back. Nice guy. Obviously an OO-trained programmer. But a nice guy, none-the-less. ;-)

Jeff Atwood mentions the Angle Bracket Tax and not surprisingly, I don’t agree. XML can be difficult and painful at times, but I think the reasons are not entirely technical. Recently, I had the opportunity to work with XML in Java and it was definitely “taxing”. Even though the process was frustrating, it really had little to do with XML. The biggest pain was actually Java.

After working in Python/Ruby for a good portion of time, declaring types, long CamelCase variable/class names and overly complex Object Oriented patterns feel painful. I’m not much of a Java hacker, so most problems could be chalked up to my lack of experience in the language. While a better understanding of Java would have been helpful, in learning XML and C# I had a very similar (and frustrating) experience, which makes me believe it is not necessarily the XML. After working with XML in Python (and Ruby to a lesser extent), it is clear that the real problem is not an “angle bracket” tax. In fact, I would argue that I got a “tax return” by understanding concepts like DOM, which I became exposed to through JavaScript, rather than Python.

It is fine to think XML is “hard” and as I said before, it can be frustrating. But to consider XML the source of frustration probably is not considering all the factors. Jeff’s blog generally focuses on programming and human factors and this is a great example of a human factor. If you expect the static typing, wasteful OO patterns and require IDE support, then you have accepted the struggles as normal. When you then are forced to deal with XML, the common links to OO ideals and patterns don’t match, leading you to the conclusion that XML is hard. The people who breeze through XML and enjoy the technology are simply those who have invested a little time in learning basic tools. Of course, these people also have set their own expectations regarding XML, but therein lies the secret.

Programmers are supposed to be logical people who make decisions using reason. The reality is programmers are people with irrational feelings and emotions that impact decision making. There is a good chance your criticisms of XML are rooted in the thousands of blog posts saying XML sucks. There is an even better chance you promote tools such as YAML without ever actually having used it heavily. Opinions obviously have a place in software, but so does logic and reason. The next time you deal with XML, take a minute and try to learn something new before complaining. When I recently worked with Java, I went ahead and put aside my frustrations for a bit to get a good handle on Ant. Lo and behold, it was interesting and I learned something new that helped improve my perspective on Java. If you take the time to learn XML basics with an objective mindset, you might still say it sucks. On the other hand, you might realize it is not so bad.

Hello all.

My name is Griffin Caprio and I’m a new blogger here on xml.com. Apologies are in order to Kurt, my fearless editor, since this is my first post and I actually came into the O’Reilly fold back in March. I am very excited to blog here, as the O’Reilly xml community is at the forefront of XML evangelism.

I’ll be blogging mostly about semantic technologies. However, this will not be just another Semantic Web blog. Most of the talk surrounding semantics is centered on the Semantic Web and it’s potential to usher in a new era of interactive / integrated web applications. Personally, I’m more interested in using semantic technologies to tackle the oceans of data companies are amassing internally. Some people call that Business Intelligence (BI) and some call it Data Mining (DM). I’ll try and stay away from those types of product categorizations and concentrate on the pragmatic application of semantics.

As is standard practice here at xml.com, please feel free to leave me comments and / or suggestions if there is something you think I should know. We welcome all feedback!

A few months ago I wrote an article for Government Computer News on the battle over Rich Internet Applications. At that time, I thought it was odd that the other major contenders, Silverlight and Flex, use XML and JavaFX does not. I wonder, if in the rush to push something out the door, Sun forgot about separation of concerns and the benefits of skill specialization to quality production. I see the trend towards declarative User Interfaces as a good thing - the proper domain of graphic designers - so, why did Sun seemingly take a step backwards? If you are a JavaFX guru, I am interested in understanding this. I found a good simple way to compare these techniques was the bubblemark site which programs a simple animation in all three (and many more) variants. What are your thoughts on XAML versus MXML versus JavaFX? In looking at the bubblemark application I still feel that MXML is the cleanest. This definitely will require some more looking in to… in the meantime, see you in the trenches. - Mike

I’ve had a couple of conversations lately with people I like that led to strange places. One was about the seemingly crazy investment I’m making in ducks, chickens, and all the infrastructure they need, and the other was what I get for saying that I think we need to take a serious look at one-room schoolhouses as an alternative to busing kids everywhere.

I’m doing a lot of things that might strike people as strange or even inconsistent, but the driving force in most of it is an expectation that energy prices are going to keep climbing. I could, of course, be wrong, but unless we discover fusion technology, I feel pretty comfortable arguing that this is going to be the case for a long time. Demand keeps climbing, while supply isn’t looking so great.

This doesn’t mean I’ve disconnected my house from the grid, built a barbed-wire fence around my compound, and set up a world that’s all about preserving me, whatever prices may do.

It does mean, however, that I’m looking at the world in a very different way. I’m trying to invest in things now that will serve us well later. The new roof, furnace, and insulation were key components of that, as is all the gardening, the apple trees, and terracing the yard. We’ve learned how to can and freeze food.

All of these are good things to do anyway, even if energy prices don’t climb. Okay, I do worry that converting my yard into a garden may make the house harder to sell in this lawn-obsessed country. On the other hand, the return on investment for some of them will take a long time. The furnace and insulation have probably paid for themselves by now, and, well, you always need a solid roof.

The rest of the investments are pretty much a gamble. I can certainly drive a few miles and buy groceries more cheaply than I can grow them here today. Setting up the garden, and especially the fenced areas for the chickens and ducks, is an expensive adventure, if an interesting one. How many eggs does it take to pay for a $200 fence? Plus a coop, and all the feed the chickens ate in the meantime? And the time it takes to feed and attend them?

If I run these calculations based on current costs, I’m pretty clearly going to lose money. If, however, I calculate these investments as advantages in a world where energy and food rapidly become more precious, I may well come out ahead. (Looking at the rapid climb in wheat and rice prices, it’s looking like a good bet sooner than I expected, though I expect they’ll come down again before climbing over the long term.)

I’m already coming out ahead in knowledge, since I now know all kinds of things that I had no clue about before. (I still have much to learn.) Because I’m showing these ideas off both on Living in Dryden and on my gardening site, I hope to plant seeds in people’s heads as well - the “I can do that” seed that develops into projects.

(I’m also taking a look at policy related to these issues through the lens of TCLocal.)

It’s quite a change from my earlier work and my job, where I focus pretty exclusively on technology - what comes next, what great new invention will let us solve all these problems. I do think at this point that Web 2.0 is for real, reflecting a major shift in how we use technology - but I don’t think that set of technologies is going to do all that much to address this major concern. Similarly, Rails is a lot of fun, and lets me do cool new things - but it’s hard to eat it.

So if I come off as kind of strange, pushing ideas that don’t all make sense at first glance, try to remember the context I’m working in. I live in the current world and do enjoy it, while planning ahead for a world that I expect will be pretty different. We’ll see what Sungiva, our one-month-old daughter, thinks of all this when she’s older.

[Cross-posted in slightly different form at Living in Dryden.]

In 1998 I gave up an extremely lucrative contract with MSFT to go and work — for free — as an intern for the Seattle Supersonics. My dream “job”.

That choice was a mistake. At least from a personal level.

But I learned some lessons from that gig I couldn’t have learned anywhere else….

Passion.

That’s what I learned. Above everything else, that’s what I learned about life.

Passion is what matters. It’s what makes the difference between a forgotten has-been…

… and an NBA champion.

Update: As per my follow-up comment to Norm Walsh’s comment below,

NW: >> Obscure? Stole? Mystery? I thought I was fairly candid about why I left and why I joined Mark Logic.

So here’s what happened: I read just the first portion of the entry in my feed reader and assumed that was it. I hadn’t noticed “Sun set” at that point because “Mark Logic” was written afterwards so therefore was at the top of the stack in my inbox.

So it looks like what I had thought was just a one line “I’m now with Mark Logic” post contained a bunch more information than I had realized. Didn’t mean to come across in such a way as to suggest I thought there was some sort of conspiracy, though it seems that’s how it came across. My bad!

Also, as it relates to the usage of obscure ‘XQuery Content’ company,

That’s probably the wrong word. I guess what I’m suggesting is the XQuery market at the moment is still pretty small, so by obscure company I really should have said something more inline with “small niche market industry.”

And lastly, if the back channel rumor mills are correct, my guess is that this isn’t the last big-name XML luminary we’ll see moving over to Mark Logic. Time will tell… ;-) I’ll write a new entry related to this topic if/when it seems appropriate to do so.

[Original Post]
… I’m wondering how in the hell some obscure “XQuery Content” company stole Norm Walsh away from Sun. Y’all know who Norm Walsh is, right?

Anyone care to provide some insight? Is Mark Logic really *that* good?

Mark Logic

Makers of an exceptional XQuery engine and XML content platform. And my new employers.

Anybody? ‘Cmon… There’s got to be more to this story than just… [Mystery]?

I’ve been working at Sun since 2001 as close architectural consultant on various pioneering uses of XML, SOA and Semantic technologies in a couple of divisions, first through Fourthought and now through Zepheira. Sun has been the practical landing pad for some of the topics I visit so often here and in my articles. I tend to not talk much directly about my work with them, in order to respect any commercial advantage they may deem, but I often wish more details were open, and not just the strategic overview as in the SwoRDFish paper. It delights me then that my long-time manager Kristen Harris, Director of Content Management Engineering, has recently been opening up some useful details as part of her group’s effort to reach out more to the community of Sun users, and to technical communities on the Web overall.

I have recently been getting back into XML processing for some upcoming work with business glossaries. So, I will be examining the relevant business glossary formats in forthcoming blog posts…

As a precursor to that, I would like to delve a bit into XML design by bringing up something that has troubled me on several occasions over the past six months. First, as I was writing a simple iTunes de-dupe utility, I had occasion to parse and process the iTunes XML format. There are many good articles on this format like this xml.com article entitled Hacking iTunes. The iTunes format is an XML data-dump of a data structure called “plist” which is a list of key/value pairs. In fact, “plist” stands for “property list”. While using XML to persist data structures enables some minimal benefits via text encoding, it seems to be harmful to the larger goal of XML being easily understandable and thus processable by many applications. So, while in a small way, dumping data structures to XML is not evil, it also is not recommended. The reason is that the data remains fairly tightly coupled to the program which produced it and thus the semantic value of the data, as a standalone entity, is diminished. In short: Better to design XML documents than dump XML data.

Besides iTunes, this cropped up again about a month ago when I was examining a Microsoft dump of system information on Windows Server 2003 - I looked for that format on the web but was unable to locate it. If you have a link to it, please post a comment with it. That system information format basically fell into the same trap of simply dumping a data structure to XML. The problem I have with this is that it shifts the semantics from a more stable element into the more variable element’s value. An element can have a unique ID. An element can be described via one or more schema types. An element can be reliably referenced externally. Thus, the semantics belong in the elements and not in the element (or attribute) value.

Do you agree?

Do you have other examples of data structures dumped to XML?

As a related reference, this xml.com article calls this a “dynamic document”. However, since I am arguing here that an XML document is an expression of design, I would say that is, at best, a misnomer. To me, dumping data structures to XML seems to be a case of “interoperability lip service”. So, do you design XML documents or dump XML data?
If so, why or why not?

Until next time, see you in the trenches. - Mike

last-ditch emotional plea � GNUosphere

No matter how the death penalty is debated, the dialog I often end up in boils down to something like this…

OK Peter, I understand what you’re saying but what if it was your kid that was raped and murdered? How would you feel? What would you want to do?

My response:

I might go insane with anger. Though I feel uncomfortable saying this, I have to admit I may feel like killing the perpetrator while inflicting a painful revenge.

Their response:

See?

My response:

We should adopt laws expressing insanity and rage with the intention of vengeance?

I recently received the following comment to my previous Amazon POD post from “Mary”,

I’m sorry, but anyone naive enough to think Amazon is not going for the monopoly on POD publishing needs to really read that letter. And, from what I have read, Booksurge not only charges up front fees, but produces a shoddy end product. No thanks. I am cutting all ties with Amazon and believe most writers would be wise to take a stand on this. Do we ultimately want Amazon to tell us what is and is not publishable? What we can and cannot write? Dictatorships start with small, seemingly harmless inroads as well….food for thought.

A copy of my follow-up falls inline below with further commentary just after.

Update: Not sure if it was this post, or someone else who may have emailed and/or someone at the W3C who took notice of the multiple 404’s being logged, but regardless of the reason, it seems this has now been fixed.

[Original Post]
So maybe I’m missing some sort of intended joke by Tim Berners-Lee, but in doing some brush-up reading on URI’s (doing some work in the fragment identifier space at the moment, so wanted to quickly refresh on a few things to ensure I wasn’t abusing or misusing the aforementioned identifier inside of an a/@href attribute) I came across the ‘See also’ section of TimBL’s ‘Axioms of Web architecture’,

Univeral Resource Identifiers — Axioms of Web architecture

See also

• Cool URIs don’t change -
  persistence in the HTTP space
• Hall of Flame — how not to do it

In clicking through to the Hall of Flame I was greeted with,

As per the title, I’m wondering how many of you have taken the time to read through the letter Amazon.com sent to publishers on March 31st of this year? If you have not, it’s probably worth your time to read through to gain a better understanding of what is seemingly taking place.

A couple of highlights,

One question that we’ve seen is a simple one. Is Amazon requiring that print-on-demand books be printed inside Amazon’s own fulfillment centers, and if so why?

Yes. <snip/> (for a reason… reason to follow)

The reason: If the above is all that you read, outrage I believe would be the only sensible response. However, from what I have read thus far from both the blogosphere and professional journalists alike, I have yet to stumble upon anything that provides a sensible overview of the reasoning behind Amazon.com’s new policy. I’m not here to judge nor really even make attempt to change your opinion. Instead, I just want to be sure that both sides of this story have been properly propagated.

With the above in mind, a Q and A style summary follows, and below that, a copy of the entire letter.

One thing I’ve noticed when designing applications of late is heavy use of URLs. Rather obviously, RESTful services have URL interfaces that expose the service to the web. I personally think RESTful service should be "mountable" at any other URL, which requires a bit of care in the URL arena. Beyond the obvious publishing of services and resources, URLs have been playing a helpful role throughout the applications I’m working on.

Namespaces are one area where URLs seem to always come up. URLs make for great namespaces because they are essentially infinite and often referencable, even thought they don’t have to be. Basing your namespaces on a domain you own is a reliable way to make sure you are avoiding collisions, while offering something a little more unique than a PURL address or something similar. Also, within the scope of a framework, actual addressable namespaces can provide interesting results within the scope of communicating with other services. Also, when I say namespaces in this context, they do not have to restricted to XML Namespaces. A namespace in this context simply provides uniqueness, similar to a hashing function.

This combination of uniqueness and referencability is a pretty powerful combo. In Python WSGI web applications, there is a dictionary that is passed along to the underlying application and middleware. The dictionary keys are often namespaced in one way or another to avoid collisions with other middleware. A simple prefix map using actual URLs could help the issue and make accessing items in the environment dictionary simpler. I know in the past I’ve used helper functions to get and set items in the environment dictionary, but having my own key that maps to a URL namespace could help reduce my own typing errors while making it explicit what I am doing.

I think what I like about using URLs is the implied linked nature. Using something like a UUID for some ID is fine, but when you utilize a URL there is an implicit connection with some addressable resource. When you select the IDs from your database and you get a few thousand UUIDs the reasoning becomes more obvious. Some might argue that depending on some semantics within an ID is bad idea, and to an extent I agree. With that said, when the service crashes hard and you are tasked to rebuild it from scratch, you might think differently.

When designing the electronic mortgage standard for Fannie Mae, my goal was to design a flexible container that could travel with the content, protect the content, log its use and render it in multiple views. Of course, I was leveraging my programming experience and encoding the MVC pattern in XML. In my latest book, I refined the concept into this diagram:

Most recently, at the FOSE conference I moderated a panel on Information Sharing. In that panel, I discussed some of the principles of “Information As Product” and compared these XML containers to the packaging and labeling of physical products. A panel member suggested I examine the LEXS standard as an example of such an XML container. I was pleased to see the strong similarity between this work and the electronic mortgage specification (now a MISMO standard called SmartDoc).

I see Information MVC as a key XML technique to enable information products that make particular content relevant to specific consumers. The difference between information and an information product is making information suitable for groups or segments of consumers. Enabling the content to expose a different view based on who is accessing it helps us increase the relevance, privacy and security of our content. For programmers this is probably old news, but for data architects and data architecture it is a new concept. Have you used this concept or seen it used? If so, let me know…

Also, for those interested in Information Products, I just posted the first chapter of my latest book online.

Until next time, see you in the XML trenches. - Mike

The U.S. Government is very leery of technology “fads” and that is why it often has a love/hate relationship with XML. You must remember that at any time, for EVERY technology that exists - the government has a huge legacy investment. So, while the corporate world may turn on a dime and quickly adopt the latest and greatest thing … the government MUST contend with HUGE legacy issues, a 2-year (minimum) budget planning cycle, and a horde of technologists actively engaged and personally invested in that legacy technology that you want to throw away!

So, in this blog, I hope to engage with you in many discussions about how the government is using XML technology, how they should use XML technology and everything in between!

So, bring me your Government XML stories and I will bring you mine … today, let me briefly discuss a program that I initiated when working for the Department of Homeland Security (DHS). The National Information Exchange Model (NIEM) started as a joint-venture between DHS and the Department of Justice (DOJ) to harmonize and speed up the process of information sharing between the federal government and state and local governments (actually State, Local and Tribal governments). Here is the website for you to learn about and experiment with this XML data model. The basic idea is that it combines a registry of standard data objects (modeled via XML Schema), a process for quickly producing an exchange message, a governance process for the model, and robust tool support. The model leveraged and extended an existing model called the Global Justice XML Data Model (GJXDM). It is widely used by law enforcement at all levels of government and now is also being widely used at DHS. It has multiple success stories behind it including the Amber Alert and the national sex offender registry. I highly encourage everyone to look at it and help make it better. So, what does this mean for Government Schizophrenia? For information sharing, XML is a favorite but is attacked continuously in relation to weak data modeling support, weak encoding of binary objects, performance issues, and many more. Remember, the roar of legacy systems has a long tail …

Until next time, see you in the XML trenches… :)

Lately I’m becoming more bullish on RDFa. I loose heart when things don’t materialize on my timeline, however, I was recently reminded of a scene in “Under The Tuscan Sun” wherein the actor describes how they built tracks over the Alps before there was a train in existence that could make the trip.

“They built it because they knew someday the train would come…“

That’s similar to what the semantic web community is doing. In my analogy the tracks are the specifications (RDF, OWL, RDFa) and data sources. The train is of course applications that use these specifications and information to make our online lives more convenient. And for me personally, the mountain is grasping how it all works and most important of all, applying it in the real world.

That brings me to what I want to share. If you are particularly interested in RDFa, http://rdfa.info/ is probably the best link to watch. I found a very well written and informative tuturiol on linked data that I highly recommend for anybody gearing up to apply this technology. The recommendation to “not define new vocabularies from scratch” struck me as particularly good advice. If you are wondering what the more common, well used and understood RDF vocabularies are, here are my top 5:

A couple of weeks ago, when I was in New York for the AJAXWorld conference, the city welcomed me with a major downpour most of the time I was there. The metaphor about storm clouds building over Wall Street was wrong - the storm clouds had opened up and were hitting Wall St. and Madison Ave. and Broadway all pretty much with equal vehemence.

The consequence of the more financial fallout on Wall St. is only now just beginning to hammer at the average American (and is at least on the horizon for the average Canadian). If you were employed in the housing or financial sectors, chances are rising that you’re no longer in the housing or financial sectors. As the credit markets continue to seize up and the government bailouts begin to soar into the hundreds of billions of dollars, this WILL have a negative impact upon employment in most other sectors, I’m not going into the details - I’m sure that you are probably already as attuned to the shaky state of the economy as I am.

However, this has helped to highlight for me an issue that I’m seeing hit with increasing savagery. We live in an impedance economy, an economy in which the value of goods or services is directly proportional to the scarcity of those goods or services. On the commodities side this is reflected in dramatically rising food prices as demand for staple crops - rice, wheat, corn, soybeans - rises … whether that rise be the rise of a Chinese and Indian middle class (with corresponding shifts in taste), reduced arable land due to climactic pressures, subsidization of ethanol, rising petroleum costs .. there are certainly enough vectors..

As we approach the atomic limit of silicon, and hence the upper bounds of Moore’s Law, our insatiable appetite for computing power and petabyte-scale applications will be increasingly met by a horizontal scaling of computer power: ever-larger server-farms.

The platforms are already being developed to handle this situation; Amazon Web Services products such as EC2, S3 and SimpleDB provide an important infrastructure on which to develop scalable applications.

Although the problem has already been partly addressed at the platform level, little attention has yet been given to how applications can easily scale across these expandable infrastructures.

Bill recently commented on another small flare up on the REXML front. It is too bad that Ruby doesn’t have a better set of libraries for XML. As Bill mentions, Python does a great job with XML. He mentions ElementTree, which is definitely better than something like pure DOM. Lxml is another option, which actually implements the ElementTree API and includes some pretty slick objectify functionality. Ian recently performed some rather unscientific, but still interesting, benchmarks on some Python libraries for parsing HTML. Ian found lxml to be quite the performer. There is also the 4Suite and Amara toolset that provides a very comprehensive suite of XML tools including an entire XML/RDF based document repository and full featured XSLT engine.

It makes me wonder why the Ruby community have not stepped up with some better options. The Python community is very similar in that XML has not been a hallmark of the community as compared to Java or .NET. One argument could simply be time, since Python has been around a bit longer. No matter the reason, I think it is time for the Ruby community to consider stepping up and producing a healthy alternative to REXML. My first steps would be to start with the libxml bindings and go from there. Lxml and Amara have both proven that utilizing a fast C library for the grunt work pays off in the end.

Lastly, I want to make it clear that REXML is still a pretty great tool. It meets the needs of many of its users, which is more than many software projects seem to accomplish. With that in mind, lets not stop there when we can do even better to make Ruby a great language for working with XML.

Update: via a recent follow-up comment from Rick Jelliffe, we have ourselves our QOTD,

If DIS 29500 mark II has been accepted, then the narrowness of the victory needs to be something that Ecma and Microsoft take very seriously: standards maintenance needs to be a budgeted, normal cost of doing business. They should be aware that they are being thrown a lifeline, to some extent. If this becomes a one-off publicity stunt, as is the dire warning of MS’ competitors (and therefore, their own publicity stunts!), and timely, real maintenance is not performed, I would expect OOXML would be de-standardized at ISO.

[Original Post]
Open XML appears to clear ISO standard vote | Tech news blog - CNET News.com

Early reports Sunday indicate that Office Open XML (OOXML) appears to have enough votes to be certified an ISO standard. An official tally is not expected until Monday.

Some of you may have noticed that I decided a while back to ignore the whole OOXML/DIS 29500 debate here on XML.com. Two reasons: 1) Too much cost, not enough gain. 2) Rick Jelliffe had things covered from top to bottom, someone *MUCH* more qualified and capable than I to provide a proper perspective of what was going on and what it all meant.

Readers may care to amuse themselves with this double think: Arnaud Le Hors Clarifiation about ASF and OOXML in which he says

In case anybody misunderstood my blog entry “Let’s be clear: The Apache Software Foundation does NOT support OOXML“, I did not mean to imply that the ASF has any official position one way or another regarding OOXML.

Err, except that the title of the blog was ASF does NOT support OOXML!

even stranger, le Hors is responding to a claim that he made up himself

OK, I’ll admit that nobody has claimed otherwise. Yet. But in these days and age you are never too prudent. It wouldn’t surprise me to see this or other similar fancy claim being published eventually.

So le Hors makes up a claim (that someone is saying ASF officially supports OOXML), then decries it (that ASF does not support it), then is forced to retract the decrial (that ASF has no position), then claims that that he never meant to imply what he had said in plain words in his own headline! O what a tangled web we weave! But quite a funny example of the mentality that seems to have possessed some people: truthy rather than facty.

I only read Arnaud’s blog because I got a mention. He repeats Groklaw’s decade old story about MS secret dirty tricks to maintain control of its proprietary binary standards as de facto standards, and somehow vaguely tieing me into it: daft given that I have been so open and my concern is with helping (force) MS out of its market-dominating proprietary standards. He also mentions Patrick Durusau’s change of heart I see, and repeats the IBM mantra handed down in IBM VP’s Sutor’s Critical questions that dissenters should expect their reputations to be at stake.

As part of IBM’s commitment to intimidation, le Hors reaches a new low, which kind of offsets the chuckles I mentioned at the top. Speaking in the context of me and other experts who dare dissent

the consequences when being caught to have failed to disclose any relevant affiliation could be far greater than they currently are. I’m not excluding judicial prosecution here.

Where are their heads at? People who wonder why I spend so much of time on OOXML issues in this blog, which is time spent at my own cost (and it is a real cost: I could be doing paid work instead of writing this) should recognize that it is largely in response to this kind of bullying and intimidation, that I saw glimpses of early on as the suits started to invade ISO in 2006.

Jeff Barr’s Blog � Save the Rovers!

Apparently NASA plans to cut one of the two Mars Rovers from it’s budget. But Jeff Barr has a plan to help save it…

The second plan is a bit different. What if we just raise the freaking 4 million bucks ourselves and simply give it to the team? What would happen if we showed up at NASA HQ with a big plywood check (backed up by real money of course)? Could they take it, and would this make a difference? Does anyone know?

I just registered savetherovers.com and would be happy to use it as part of an organized effort to do something remarkable. I would also be willing to pitch in $100 as my part of the $4 million. That means we need just $3,999,900 more.

I’m in Jeff! Anyone else care to join the effort? I’ll contact Jeff and see how he plans to facilitate the volunteer/donation effort and report back the result…

Stay tuned!

Twitter / migueldeicaza

if you tweet and nobody is following you, did you tweet?

Digg - Lawrence Lessig needs your help

Having some personal experience with the benefits that come along for the ride when volunteering your time to the — and I’ll be blunt — the single most important mind of the 21st century, I can assure you of the fact that you will gain more in return than what you will give.

The above is a personal quote. Please forgive my follow-up comment to Jain Berlor. But to be quite honest, I don’t give a damn. Lawrence Lessig deserves greater respect than what Mr. Berlor seems to be willing to both give and recognize.

I won’t stand for it.

If you can help, please do. As per above, I can assure you of the fact that what you will gain in return is greater than what you will give.

… are *bound* for greatness.

Update: And for those who seemingly believe that their “power”, “wisdom”, “intellect”, and “”character”" enables them to overpower and therefore overcome the same collective power, wisdom, intellect, and character held by those of us who are listening, watching, communicating, and promoting that in which we both will and, more importantly at this moment in time, *WILL NOT* tolerate as a human culture, let the wrath that is the power of our tightly connected yet loosely coupled Internet communities take its toll upon you. [UPDATE: *PLEASE NOTE*: As per my follow-up to Len** below, “let the wrath that is the power of our tightly connected yet loosely coupled Internet communities take its toll upon you.” is *NOT* referring to wrath in the physical sense and instead in the social Internet networking sense. In other words, “Share this with your friends such that they can better understand the subject matter and judge for themselves how they might choose to react with their *alliance* to any given candidate.“.

** Please see my SPECIAL NOTE below. Thanks![/UPDATE]

Please watch/listen/and share the following with *EVERYONE* you might come encounter with. Thanks!

[Original Post cntd.]

So just added Barack Obama’s Twitter feed to my “following” list, soon thereafter to find the following in my inbox,

Hi, M. David Peterson.

Barack Obama (BarackObama) is now following your updates on Twitter.

Check out Barack Obama’s profile here:

http://twitter.com/BarackObama

Best,
Twitter

So two things,

1) He has either a bot, or an intern/staff member who is adding anybody who adds Barack Obama to the their following list to his following list.
2) Who really gives a damn, because the bottom line is that even with an audience of around 15k followers on Twitter — or in other words, not even enough people to win him a spot as the mayor of a small city — he’s still paying attention to what’s happening out here in the *real* world.

As per point 2, that kind of stuff matters to me. Maybe it does to you too?

** SPECIAL NOTE: While I certainly disagree with a lot of the comments Len (Bullard) has on this particular subject matter, he also happens to be one the smartest, most well versed individuals I know, and the one person I believe most capable of convincing me that my current line of thinking on any given subject matter we might be debating could very well be wrong. And at very least his thoughts/comments require I step back and think things through a bit more. Whether you agree or disagree with his opinions, his blog is a wealth of knowledge and thought provoking dialogue. If you haven’t already, I would encourage you to subscribe.

I had a letter from a reader, R, today, which I thought I would share with you. R is a person seemingly paralyzed by fears, assailed by the uncertainties in life, and prone to the most violent of doubts. Anything I can do to help R out of this unfortunate state of mind, I think it is my duty to do.

R writes:

Dear Rick, I have been reading your blog now for about a year, and am a great fan. We do not always see eye to eye, but your continued moderation is admirable and an inspiration to all right-thinking people. However, I am really worried about this Geneva meeting: I cannot sleep due to worry that there has been no attempt to provide any extra security against viruses in Open XML. I thought I could trust my local delegation to pursue this, but I cannot see any progress

Well, R, I can report that the BRM has in fact accepted an proposal to reduce the scope for viruses in OOXML files: I am not sure whether I can talk about individual issues yet or not, but it is issue AU-9 or Response 12.

In 2001, the sky fell for web development. Everything fell as the dot-com bubble broke. In 2008, even though the US economy’s not looking so good, there’s more hope for the web to follow the economy’s course rather than shatter and fall below it.

The issue of the best rules for naming came up recently. I think the way of the future for real human readability is euphony. We want beautiful Code, so why not beautiful markup? With the current discussions on XML 1.0 version 5 and its naming rules buzzing around, euphony has surely never been a more critical issue.

So what might rules be? “Jelliffe” is an old spelling for jolly. My parent’s traced the familly tree back, they think, to Bosham, England a thousand years ago, which is when King Canute was there.

One of the interesting parts of the tree is how often the same names come up: John Jelliffe, William Jelliffe and so on. Common enough names, but relentless and I have often wondered what the euphonic principles were. I have been looking at various a genealogical sites recently, and I think I have cracked the rules.

We start of by discarding wives names. I am not sure many people picked their spouse because of their first name, as in The Importance of being Earnest (though there is an Earnest.) Here are the rules as I see them:

1) Any name that starts with J sound. But this must be a single syllable name (or single plus a diminuative.) John. Jane. James/Jimmy, Charles, Jessie

2) Any name that has an internal rhyme or sound in the same position. Emma, Thelma, Elton, Benton, Benson, Benjamin, Frederick, Fred, Terence, Preston, Elsie, William, Mildred, Helen, Ella, Illah (my favourite), Lillian, Ralph?, Pauline, Wellford, Edyth, Edwin, Jesse, Ethelyne, Elinor, Elward, Patrick

3) Any name with the same rhythm making dash dot dash dot: Catherine, Martha, Robert, Clara, Hazel, Joseph, Edith, David, Taylor, Martin, Emma, Roger, Myrtle, Walter, Waltby, Ella, Vincent, Arthur, Sarah, Eva, Richmond, Zada, Jesse, Elward, Rufus, Walter, William, Zalmon, Mahala, Thirza, Stebbins, Fannie

4) Any name with a soft sound that matches the ffs and soft J: Catherine, Martha, Howard, Edith, Nathaniel, Smith, Fern, Phoebe, Arthur, Eva, Virginia, Willis, Owen, Fond, William, Fannie

Of the rest, there is mob that seems to have surnames as their first name, in the American fashion (Burr). But these rules catch most of them. Some odd runs in New Jersey: Gustiss, Freylinghysen.

So a name like Estha really fits the euphonic bill: 2, 3 and 4.

And I guess it makes sense of the first name Luzon which crops up: rules 3 and 4.

So for people who are passionate about beautiful documents, I suggest that attribute names and namespace prefixes should be picked to be euphonic with the element names. We needn’t, indeed we mustn’t and shan’t, put up with names picked because of some arbitrary theory of utility or historical accident, someone’s mere whim or assertion. We need the science of euphony to get really beautiful markup and proper human memorability (surely a better goal than just readability?)

Lessig ‘08 - Change Congress.

From Lessig

This site hosts this video to explain the launch of two exploratory projects — first, a Change Congress movement, and second, my own decision whether to run for Congress in the California 12th.

I have decided I want to give as much energy as I can to the Change Congress movement. I will decide in the next week or so whether it makes sense to advance that movement by running for Congress.

Many friends have weighed in on that decision — both strongly in favor and strongly opposed. Many more have joined draftlessig.org and a Facebook group asking me to consider it.

Watch or listen and you will understand some of my reasoning. Feel free to send your thoughts or advice to lessig@lessig08.org (though please excuse any slowness in my response).

More ways to show your support to follow shortly…

I recently did an interview with Rich Seeley for SearchSOA/TechTarget on the relationship between eXtreme Transaction Processing (XTP) and SOA, CEP, BPEL and BAM. Here’s an excerpt -

In celebration of XML 1.0’s tenth anniversary, I signed back on to XML-DEV to suggest that it’s time to do to XML - just the core of it, please - what XML did to SGML around SGML’s tenth anniversary.

“I think [forgiveness] may be the greatest virtue on earth, and certainly the most needed. There is so much of meanness and abuse, of intolerance and hatred. There is so great a need for repentance and forgiveness. It is the great principle emphasized in all of scripture, both ancient and modern.

Somehow forgiveness, with love and tolerance, accomplishes miracles that can happen in no other way.”

–Gordon B. Hinckley, “Forgiveness,” Ensign, Nov. 2005, 81

It is time for us to forgive. Please consider just this very act.

Update: Thorsten has followed-up with some interesting comments regarding the current state of the virtualized industry, the problems he’s seeing a lot of companies facing, and various obstacles they’re running into along the way. Interesting stuff!

A word of advice, easy to give, hard to follow: design your system so you can relaunch any critical instance!

Amazon has thousands of instances available, just waiting for you to hit the launch button. If a current instance smells bad and your own troubleshooting doesn’t resolve it, launch a new one and bring your service up on it. Actually, if it’s critical, you should have two running so you’d be left with one while you replace the failing one.

All this should be motherhood and apple pie on EC2 or any other hosting facility, or also in your own datacenter for that matter. Systems fail.

Thorsten von Eicken, Posted: Feb 2, 2008 10:53 AM PST

BTW: Thorsten is one of the smartest individuals I have ever had the fortune of coming to know. *GREAT* guy, and someone in whom if you need help with Amazon Web Services-related consulting, in particular EC2, I would *HIGHLY* recommend getting in contact with his company, RightScale. Just the right combination of open source, open minds, and openly giving more than he/they receive in return, so I believe it’s certainly both fair and in-line with the ideals of O’Reilly, and therefore this blog to provide promotion.

Merger-mania is in full swing of late, which is rather astonishing given the current credit market problems. Oracle finally managed after months of trying to snare business services provider BEA, and Sun’s purchase of mySQL was both hailed as a master-stroke and derided as a last gasp hope by a fading giant to bolster its database claims. Today the announcement hit the fan that Microsoft had made a $45 billion dollar bid for Yahoo, something that’s made Wall Street happy but that has people in Silicon Valley scratching their heads.

I am not a CEO, nor am I a big investor … and I’m generally not a big fan of mergers and acquisitions (M&As), because, especially when the mergers involve two reasonably large, well-established companies, the results in terms of performance seldom justify the costs involved. This is especially true of tech sector companies, where so many of the assets of the companies are not tied up in physical capital but rather in abstract ideas carried around in smart people’s heads.

The overtures that Microsoft has been making to Yahoo have been obvious for some time. The integration of Yahoo and Microsoft’s IM formats, for instance, hinted that the two were playing footsies under the table, and certainly the announcements that Yahoo would be utilizing Microsoft technology (and Microsoft’s subsequent PR blitz to that effect) at least indicated that the relationship was serious. Thus, the signs have been around for a while, and both the tech and Wall Street press have generally been playing the role of matchmakers. Yet there are more than a few signs that this particular marriage, if consumated, may end up in divorce court nonetheless.

Karen Deane has an article in The Australian Battle on Microsoft standard push which includes some quotes from a background interview she asked me for, to give her the gossip ahead of the big MS journalist fly-in last week.

There has been a fair amount of chatter lately about defending the value of SOA projects or justifying such projects to the “C-level”. Many of these discussions will point at the business value of doing more with less and achieving IT cost reduction by reducing redundant systems and reuse of services in a SOA. Also streamlining processes in order to run the business more efficiently is a popular opinion,

NOTE: ILMOTD == IRC Leaving Message Of The Day

As seen in the scroll back on #xslt,

MikeSmith left the chat room. (”Less talk, more pimp walk.”)

Actually, there are plenty of reasons why F# ((F == Functional) == True)) *ROCKS*. Here’s a few from the previously linked F# site on Microsoft Research,

Combining the efficiency, scripting, strong typing and productivity of ML with the stability, libraries, cross-language working and tools of .NET.

F# is a programming language that provides the much sought-after combination of type safety, performance and scripting, with all the advantages of running on a high-quality, well-supported modern runtime system. F# gives you a combination of

* interactive scripting like Python,

* the foundations for an interactive data visualization environment like MATLAB,

* the strong type inference and safety of ML,

* a cross-compiling compatible core shared with the popular OCaml language,

* a performance profile like that of C#,

* easy access to the entire range of powerful .NET libraries and database tools,

* a foundational simplicity with similar roots to Scheme,

* the option of a top-rate Visual Studio integration,

* the experience of a first-class team of language researchers with a track record of delivering high-quality implementations,

* the speed of native code execution on the concurrent, portable, and distributed .NET Framework.

The only language to provide a combination like this is F# (pronounced FSharp) - a scripted/functional/imperative/object-oriented programming language that is a fantastic basis for many practical scientific, engineering and web-based programming tasks.

F# is a pragmatically-oriented variant of ML that shares a core language with OCaml. F# programs run on top of the .NET Framework. Unlike other scripting languages it executes at or near the speed of C# and C++, making use of the performance that comes through strong typing. Unlike many statically-typed languages it also supports many dynamic language techniques, such as property discovery and reflection where needed. F# includes extensions for working across languages and for object-oriented programming, and it works seamlessly with other .NET programming languages and tools.

For those of you unaware, F# is now a first class MSFT language, or in other words, this is no longer a “Hey, here’s an idea. Let’s research it.”-type project and instead a true-blue MSFT product backed by mean-green MSFT money, led by some of the very best and brightest minds @ MSFT.

If you were to ask me “What’s the future language foundation of the .NET platform?” I would first state “More than likely, XSLT 2.0++.” And then when you stopped laughing and slapped me upside my head to awake me from my dream I’d say, “What the F#!? was that for?” and you’d say “F#??,” followed by “Isn’t that for programming the way God intended for people to program on the .NET platform?”, and then I’d say “Okay, you got me on that.” at which point we’d move on…

So here’s the thing: While there are *TONS* and *TONS* of reasons why F# *ROCKS* (did I mention that F# is distributed as both an MSI and a ZIP, the latter designed to make it easy for folks using Mono to take full advantage of what F# has to offer?), the biggest reason it *ROCKS* is this,

It’s been a while since I’ve written a “non-directed” blog for xml.com, so while I will be covering a few XML topics here if you’re not interested in economic systems theory, then you might as well skip this.

As I write this, it’s about eight hours before the financial markets open in New York. The markets were closed today, Monday the 21st of January, for Martin Luther King, Jr. day, which may prove to be a bad thing in the morning. Today the average market loss globally was about 7% - here in Canada, the drop translated to a 605 point loss, or about 4.75%, on the TSX. The India Sensex fell 11% in two minutes before trading was halted. I can pull out other figures, but they say much the same thing.

It’s hard to say what will happen in the morning - I’m not even going to try, though I have my suspicions. Enough fire control may have been put into place to keep the US markets from getting too badly singed (though I have NO doubt that few people at any brokerage firm in the country were allowed to stay at home today), but what you’re seeing here is something that we’ve not seen in a long time … the start of a worldwide stock crash.

I’m happy to report that Ruby on Rails not only offers a comfortable way to develop web applications, but that a little-noticed feature makes some formerly theoretical open approaches to XML much more immediately practical.

… and then what will /. have to write about? *NOTHING*, I tell you! *NOTHING*!!!

Wait, that would be a good thing, huh?! *SWEET*! Please carry on…

Brian Jones: Open XML Formats : Mapping documents in the binary format (.doc; .xls; .ppt) to the Open XML format

I wanted to call everyone’s attention to a few interesting developments in Ecma’s proposed disposition document related to the Office binary formats. There were a few comments from national bodies that asked about the documentation of the Office binary formats and the availability of those documents. We had already been talking about these issues in TC45 where there were a number of existing experts in the binary formats (including Apple, Novell, and Microsoft). Based on the feedback from the national bodies, Microsoft decided last week to take some additional steps in this area.

Brian goes on to describe the fact that MSFT will be making it even easier to gain access to the documentation for the legacy Office binary formats, promises not to sue you if — you know — you actually read the documentation and then apply this knowledge to implementing support for these formats, and then takes it one step further by not only providing a mapping from the binary formats to Office Open XML, but promises to start a Binary Format-to-ISO/IEC JTC 1 DIS 29500 Translator Project on SourceForge.net in collaboration with ISV’s.

Hey Micah (Dubinko),

Why don’t they just ISO standardize their binary formats? That’s “backwards compatibility” for ya. -m

Micah Dubinko | January 24, 2007 10:13 AM

Okay, so maybe this isn’t standardization, but as it relates to legacy file formats, isn’t this good enough? And either way, nice work, MSFT!

It just makes them more burnt…

Java and doneness

Just because a standards body can’t recogonize done-ness and declare victory, that doesn’t mean the fruits of that body have any bearing on reality.

I don’t agree with everything Sean McGrath writes in his latest post as I think there are a lot of really smart people who have developed some really smart ways to handle the variable width nature of XML w/o turning to malloc() every time the length of an element or attribute name reaches past any given preset constraints. That said, I can’t help but agree with,

Memory-based caches of “cooked” data structures are your friend.

Absolutely!

For you .NET developers here’s a pre-written recipe that handles all of the dirty work of determining whether to create a new XmlReader or return the in-memory cached version based on the generated ETag for the source file (see Extended Overview below for a deeper understanding of how this works.) To use this recipe you need to do nothing more than create a new XmlServiceOperationManager when your application starts up like so,

XmlServiceOperationManager myXmlServiceOperationManager =  new XmlServiceOperationManager(new Dictionary<int, XmlReader>());

and then use the GetXmlReader method of the XmlServiceOperationManager, passing in the Uri (an actual System.Uri object, not the string value of the URI, though I guess it would be easy enough to create an overload that takes the string value of the URI. Another task for another day. ;-)) of the desired XML file to get an XmlReader in return like so,

XmlReader reader = myXmlServiceOperationManager.GetXmlReader(requestUri);

That’s it! Now you can use your “new” XmlReader however you might need and the next time that file is requested for processing if it hasn’t changed you save all of the time it would normally take to read the source file and convert it into an XmlReader which is fairly significant.

Source code and extended explanation inline below. Enjoy!

Oh, and stay tuned for the next installment of this recipe where we learn how adding,

1 Part memcached
1 Part ETag's

and

1 Part GZip encoding

… can turn your lame a$$ performance sucking web application into a lean, mean, kick a$$ performing machine. For a precursor, see Joe Gregorio’s AtomPub presentation slides from this past OSCON. I assure you, it’s worth every second you spend studying this gem of a resource.

I just saw a link in the del.icio.us hot list that caught my attention called Data Portability, that bills itself as the open standards stack of the ubiquitous sharing and remixing of data.

Nice, but RSS and not Atom Syndication? OPML and not XOXO (The XHTML Outlines Microformat), JSON or YAML? And no Atom Publishing Protocol? “THE” stack? What is the criteria here?

This all seems rather arbitrary in what is “open” that I am suspicious of a hidden agenda at play here.

I’m all for open standards, data portability and anyone who wants to advocate it, but advocacy of “open” should be just that — open and not so closed, arbitrary or apparently biased. (If it’s not biased then someone didn’t do their homework.) We have enough agenda disguised in open advocacy clothes already.

- Grid computing will grip the attention of enterprise IT leaders, although given the various concepts of hardware grids, compute grids, and data grids, and different approaches taken by vendors, the definition of grid will be as fuzzy as ESB. This is likely to happen at the end of 2008.

- At least one application in the area of what Gartner calls “eXtreme Transaction Processing” (XTP) will become the poster child for grid computing. (see Gartner Research ID # G00151768 - Massimo Pezzini). This “killer app” for grid computing will most likely be in the financial services industry or the travel industry. Scalable, fault tolerant, grid enabled middle tier caching will be a key component of such applications.

- Event-Driven Architectures (EDA) will finally become a well understood ingredient for achieving realtime insight into business process, business metrics, and business exceptions. New offerings from platform vendors and startups will begin to feverishly compete in this area.

Once again into the breach, dear friends. I’ve made it a habit over the last several years to put together a list of both my own forecasts for the upcoming year in the realm of XML technologies and technology in general and, with some misgivings, to see how far off the mark I was from the LAST time that I made such a list. The exercise is useful, and should be a part of any analyst’s toolkit, because it forces you to look both at the trends and at potential disruptors to trends (which I’ve come to realize are also trends, albeit considerably more difficult to spot).

So, without further ado, I bring up the list from last year …

As the anti-OOXML crowd’s technical and editorial objections evaporate, and consequently as the reasonable people increasingly see that ISO is delivering a good result for them and jump ship, the rabid anti-OOXML misinformation campaign is ramping up. The basic strategy is to say that things are so bad that no improvement is possible, and indeed that any improvement is complicity.

But it is quite possible for the different sides to engage civilly and constructively.

OOXML Forum: AM

On Friday last week UNSW CyberSpace Law and Technology Center organized a really good day-long seminar in Sydney on the technical and legal feasibility of implementing Office Open XML, to try to get people talking.

The morning was a technical meeting: I was honoured to be invited to speak first, with 30 minutes on ISO and SC34 standards and I will be putting up my slides later. Other invited speakers include Mathew Cruikshank (who was very active in New Zealand’s vote), and Colin Jackson (NZ government angle.) Also speaking in little 10 minutes slots were a representative of IBM (same old material), and Lars Rassmussen from (notorious VML-users) Google Maps (he has a meeting report here but I don’t know why he has an “issue” with me; in any case, like Matthew, I think he is a goody.) MS had a 3 man contingent, and people were obviously trying to be on their best behaviour: Oliver Bell from Singapore was there and had blogged. Not many fireworks, I was tired and grumpy from travel so probably just as well. I was pleased to also see Standards Australia’s Alistair Teggart there too; he made some good clarifications. Prof David Vaile was very interested in what people had to say, and frequently asked for clarifications or expansions. Gnome Foundation’s Jeff Waugh was lively (in fact, he called the technical issues boring…clearly a big picture man) matched only by UNSW’s Pia Waugh. (Clarification: Jeff’s point was, I think, that other issues were more important than the technical/editorial issues of the Australian ballot comments.) (I am sure I have missed some who spoke!)

I’d stereotype the various opinions as people who didn’t see why there should be two standards, people who didn’t see the value of even one standard , people who saw the value in their own standards, people who didn’t see the value of their competition’s standards, and people who thought there should be many more standards (err, probably only me.) (Updated: I originally had some names against these stereotyped positions, but as they are probably not even fair stereotypes I’ve removed them, they don’t help the gag. If you think I have misrepresented you in any of my blogs, please write and I will certainly try to fix things.)

OOXML Forum: PM

The afternoon was the legal session, and very interesting. Unfortunately it only looked at the OSP and didn’t cover any standards law (relation to law of fraud, anti-trust, etc.) probably because in many countries there has been no relevant case law, and in each national jurisdiction the situation will be difference. The US law is quite advanced (or, at least, explicit) here: Australia really needs some legislation to clarify the duties and rights of stakeholders in voluntary national and international standards processes. I have previpously posted some material on this blog Standards and IP for people who are interested.

First up in the afternoon was legal background material by Ron Yu, a very likeable guy. He has made a report that is available from the CybserSpace Law Centre’s website. It was mainly a discussion paper raising various issues that people had made, rather than a definitive position on anything. Then MS’ Steve Mutkoski gave a talk on OSP, mainly focusing on the similarities and differences from the Adobe, Sun and IBM equivalent. He was one of the legal team who drew up OSP and pushed for less legalize in it (using “promise” rather than “covenant” for example.) His main thrust was that the differences between the Sun, IBM and MS licenses were only cosmetic. Steve made his points well.

As it turned out, from discussions it emerged that there was really only one bone of contention, which was the meaning of “required” in the OSP. Now this is something that I have blogged about before: see the lengthy comment (search for “Matthew:”) and also here (search “Kurt #2″).

MS’ legal department have been absolutely hopeless in helping people figure this out, and if OOXML fails at the final vote, they and Steve Ballmer have the lion’s share of the blame. Many in the open source community react strongly to the memory of MS’s FUDing on Linux patents; rather than (as I tend to do) saying “oh good, at least in the standards process they are opening up their IPR” (i.e. due to things like standardization and the OSP) the MS FUD has raised suspicion to the level where people say “since they clearly want to enforce their IPR, they cannot be genuine about OSP” (i.e. there is some trick there). That Steve Mutkoski was so unprepared to answer questions about what “required” meant shows, I think, that this issue (which has been repeatedly and constantly raised over the last year) just has completely flown over the heads of the MS legal department. This part of the QA session was a pretty disappointing performance.

The issue here is that the OSP only covers “required portions” of the spec (MS, IBM, etc) promise not to enforce their patents unless you sue them. When I looked at the OSP, I went ahead and looked at all the other licenses to see what “required portion” meant, since it was clearly some kind of legal term. IBM’s license is better, because it spells it out; MS thinks it is unnecessary to spell it out since lawyers would know; they wanted to keep the OSP to one page. But they got it wrong: people think that normal language is being used.

W3C (and OASIS) dealt with this very problem. I think the OSP should be redrafted to follow their wording (and of IETF), and use “normative” rather than “required”. That aligns the promise with the language of the standards and clears up some potential for confusion.

Laypeople look at “required portions” and decide that this must be in opposition to “optional portions”. Here is the MS wording from OSP:

To clarify, “Microsoft Necessary Claims” are those claims of Microsoft-owned or Microsoft-controlled patents that are necessary to implement only the required portions of the Covered Specification that are described in detail and not merely referenced in such Specification.

Predictably, IBM ’s rep was trying promote this confusion. Quite a lot of chutzpah considering that IBM in fact uses the same legal terminology in its covenant

“Necessary Claims” are those patent claims that can not be avoided by any commercially reasonable, compliant implementation of the Required Portions of a Covered Specification. “Required Portions” are those portions of a specification that must be implemented to comply with such specification. If the specification prescribes discretionary extensions, Required Portions include those portions of the discretionary extensions that must be implemented to comply with such discretionary extensions..

Intel v. Via Technologies

IBM’s wording is much clearer and better, and “required portion” is indeed a common term in these licenses. However, what if Microsoft turned around and said “We didn’t define it as a required term, and now we want to charge licenses for patents”? Lets put aside the common legal usage of required portion in licenses. Lets also put aside the small likelihood that there could be non-junk patents in the area of document processing formats (considering the maturity of Unix Publisher Workbench, TeX and so on from the 1960s to the 1980s, not to mention ISO SGML (IS 8879:1986) and its applications since 1986 and before. And lets put aside fraud issues, given the consistent public representations by dozens of top-level management from MicroSoft.

What happens with an ambiguous licenses? During the session I asked if anyone knew of any case law where “Required” was discussed, having a nag in my memory. I have looked it up again and it comes up in the case Intel v. Via Technologies, 319 F.3d 1357 (Fed. Cir. 2003 which has a discussion here. In that case, the judgement was that “required” must be given the widest interpretation (to include “optional”)

Although we agree with Intel that its reading of the plain meaning of “required by” is a reasonable one, we disagree that its reading is the only reasonable one. First, the words “required by” without any clarification could mean either non-optional protocols of AGP 2.0 or electrical interfaces or protocols that are required to perform any specification “described” in AGP 2.0, including non-optional protocols for an optional specification. For example, books “required by” a school could mean books needed for (1) “required” (non-optional) classes; or (2) any class taken, including optional classes.
…
The word “optional” does not occur anywhere in the license agreement.
…
Thus, we conclude that VIA’s and Intel’s interpretations are both reasonable readings of the license agreement. The district court erred in holding that VIA’s reading of the agreement is the only reasonable one. Nevertheless, it was harmless error because, as there is ambiguity in the agreement, the district court properly granted summary judgment of noninfringement relying on contra proferentum.
…
When a contract is ambiguous, the principle of contra proferentum, under Delaware law, requires that the agreement be construed against the drafter who is solely responsible for its terms.
…
Contra proferentum has been held determinative in resolving ambiguity in a contract that, like the agreement here, is drafted by one party and offered on a “take it or leave it” basis without meaningful negotiations.

It would be interesting to know in which other jurisdictions would also allow contra proferentum: according to Wikipedia it also includes Europe, California and international arbitration. Here in Australia, there are multiple cases that endorse the principle in various circumstances.

If I may throw a spanner in the works, the thing that I see missing from all these licenses is that they only seem to cover the of patents where use of the patents is unavoidable to implementing the spec. In other words, if there are multiple ways of implementing something, you have to use the way that is not covered by the patent. I think this is unacceptable, and something that MS, IBM and the others should fix. Don’t compete at this level, boys and girls, it is counter-productive: open up.

So it was a really enjoyable day, and I enjoyed catching up with Greg Stone, Matthew Cruikshank and the others during the breaks. I think Pia, David and the CyberLaw Policy Centre organizers did a really good job.

Consortium Disinfo

But outside in the world, confusion is still rampant. OASIS lawyer Andy Updegrove has never been known to say a positive thing about MicroSoft nor a negative thing about IBM, and most of the time he is happy to be one link down the S-bend from Rob Weir’s mischief, however he is really valuable when commenting on law. But it is interesting to see the level of misinformation of some of Updegrove’s readers.

A case in point. While I don’t see this issue as primarily IBM versus MS (that is one aspect but there are also open source people and industry people and governments occupying all positions: pro, neutral, con, don’t care, be fair, make it right, etc) nevertheless I think IBM’s strategy has long been that since they cannot prevent DIS 29500 being fixed and adopted, they need to shoot the messenger and blacken ISO’s name. In fact, IBM’s Bob Sutor is quite open about who IBM are really interested in, no shame in that: when asked about Open XML and ODF and ISO he replies

I think we have collectively educated and permanently changed the policies of procurement people in many organizations around the world.

Recently, IBM marketing guy Rob Weir has not had much to blog on, since according to JTC1 rules (which they are trying to get strictly enforced following their meeting in Australia last month) ballot resolution discussions are private. This rule is intended to stop hysteria and allow the participants the full range to describe options without outsiders citing proposals and options as done deals. (I.e. exactly what Rob has been doing, such as his complaint that the early issues dealt with were the trivial ones, to try to prop up the crumbling argument that there would be no changes to DIS 29500 in response to the National Body comments) People who are interested in participating have had a year to join their national standards body’s committees and come to grips with all the issues and procedures. So Rob came up with a great spin: Microsoft is bad because, PERFIDY! they are following the rules…

Anyway, Andy’s blog on this was a real classic according to the formula. He picks up on Weir’s message of the day, and links to both Weir and MS’ Brian Jones, which is some balance. Then, in further imitation of fairness, he quotes “Pamela Jones” (Groklaw) but her article is also just a riff from Weir’s tune. Pamela manages to find some minor wording issue based on some material on the SC34 website (the FAQ is really clear on the issue, I thought): Pamela because most people in ISO committees do not have English as their first language, it is not a good idea to try to find the worst meaning in phrasing: material on a general webpage is just general material and you are wasting people’s time by trying to read too much into them. Then, based on the spurious idea of the vote being taken at the meeting, off she goes with imagined opportunities for conspiracies and so on. Again, it is the basic strategy: FUD. Take the most lurid interpretation possible, try to discredit the process.

In politics, this kind of spin is called “innoculation”. What you do is try to get ahead of your opponent by coming in early with responses to them. For example, “Six lies the Democrats will tell you” or whatever. The intent is that then the public will hear any statements through the framework established by you. (Indeed, IBM’s Bob Sutor even talked of having a competition for this.)

Andy’s blog has a similar comical moment: after given some of the smaller details of an ECMA press release, where ECMA rolls over on some of the most contentious issues which previously the Echo Chamber had told us would never be changed, Updegrove says

Despite the meagerness of the sampling of recommendations described in the press release, it is possible to get an idea of the degree to which Ecma and Microsoft are willing to go in order to secure a final, favorable vote.

What Andy: no “How great, we got what we asked for!” instead a complaint about the press release being meagre (oh no, not enough material for a convincing spin) and then the corker that this shows “the lengths” they will go to! PERFIDY! They are refusing to act unreasonably! PERFIDY! They are giving in to user demands! So despite this utterly clear evidence that the process looks like it is working (ECMA proposes a standard, national bodies consider it and make comments, ECMA and the national bodies work out constructive solutions to them, the way that every other fast-tracked ISO standard procedes), instead it is supposed to be bad news. It is hard for me to think why it isn’t kind of pathetic.

Anyway, the thing that grabbed my attention about this blog item was that there had been four small discussion threads. And all of them were based on wrong information.

• The first makes a statement about maintenance, but the maintenance regime has not been decided yet. (See the SC34 Kyoto minutes 8.1
• The second is some FUD on copyright. Yet the ECMA copyright is very clear.
• The third claims that ISO requires two implementations. (Actually, now that Apple has about 10 different independent implementations of parts of OOXML released or in the works, claims about the impossibility of implementation are looking increasingly implausible.)
• The fourth is that OOXML could not have gone through the PAS process, as ODF did. (I made a reply giving the actual JTC1 directives on the issue: Appendix M) This last claim is often based on the idea that OOXML is a really rotten spec from the technical writing POV: ODF is contrasted. However, the fact is that all specs when looked at in detail have a large number of things to improve. Look at the Japanese defect report for ODF which has finally surfaced: 98 problems ranging from trivial editorial (’An’ not ‘A’) to the incorrect (#68) to the incomplete (#75) to the inconsistent (#88). The thing to do with errors is fix them, not augur calamity or incompetence cheap points from them.

A world of confusion. The emperor’s new clothes are tremendously well-ventilated.

The XML 2007 Conference has come and gone, with as usual a number of thought provoking talks and controversies. During the evening of the first day, there was a special XForms Evening, with a number of the industry gurus in that space providing very good examples of why XForms is a compelling technology and here to stay.

In the final keynote session, though, Elliott Rusty Harold sounded a somewhat more alarming note, indicating that while XForms does have a huge potential, there are no killer apps out there for it, and without significant support from the various players in this space it will be dead within the year.

I’ve known Chime and Uche for a while now. We all have. In fact, if it wasn’t for the Ogbuji family I doubt much XML would or even could have been much more than a passing fad. Fortunately, that’s not how things played out.

Every generation has their revolutions, and each revolution has their revolutionaries. Our generation has Chime and Uche, and to be quite honest, if that’s all our generation were to ever have, it would be more than enough. Our generation is one of the lucky ones. Sometimes that’s just the way things work. And that’s certainly how it worked out for us this go round.

Two recent entries, one in the form of a blog entry from Dare Obasanjo, the other in the form of a post to the FeedSync list from Steven Lees, both in the last 24 hours,

ADO.NET Data Services (Astoria) Transforms SQL Server into an Atom Store

This is sick. With Astoria I can expose my relational database or even a local just an XML file using a RESTful interface that utilizes the Atom Publishing Protocol or JSON. I am somewhat amused that one of the options is placing a RESTful interface over a SOAP Web Service. My, how times have changed…

It is pretty cool that Microsoft is the first major database vendor to bring the dream of the Atom store to fruition. I also like that one of the side effects of this is that there is now an AtomPub client library for .NET Framework.

Of course, I’m sure there will be many who will contend that GData, and therefore Google were the first to bring the “dream” of an Atom store to fruition my bad. Dare stated “first major database vendor“, which as far as I know is a true and fair statement. That said, I’m leaving in my props to Joe Gregorio cuz’ he deserves both the credit and attention, regardless of the fact that he isn’t a major DB vendor either. and to be completely honest, Joe Gregorio not only brought forward the original dream of the Atom store, but was the originating dreamer that brought AtomPub into existence, quietly building both the client and server pieces of this dream while at the same time acting as the (lead?) editor on a two man *ROCKSTAR* team, and backed by some of the brightest minds in the industry to ensure that the final result was what it needed to be. But let’s try and set aside differences in perspective for now and take a look at what Steven Lees has to say,

… or Girl. Pick whichever most closely resembles your gender and then apply this selection to the following…

Mike Linksvayer? The major political issue of today?

Music distribution companies are only one of the forces for control and censorship. The long term issue is bigger than whether private ownership of 21st-century printing presses should be permitted. The issue is whether individuals of the later 21st-century will have self-ownership.

The trouble with standards is that there are not enough of them. There is a strong public interest in having the interface technologies of market dominators (which would include near monopolists and long-term super-profit-takers) out in the open, unencumbered, zero royalty, non-discriminatory licensed, and with the documentation QAed by an independent group which may include experts and rivals and stakeholders.

And there is indeed a great system set up for this: ISO, the International Organization for Standardization.

First, I should clear up a misunderstanding that many people fall into: a technical standard (from ISO) is not a regulation. According to ISO, It is always the adopter’s responsibility to look at the available specifications and see which ones are useful, and in which contexts.

Second, for ISO standards, multiple standards for the same area the norm not the exception. In fact: look at the dozens of standards for graphics formats, the multiple standards for programming languages, the multiple standards for operating systems, the multiple standards for schema languages and so on. Putting a proprietary standard through the standards mill does not prevent other rival technologies becoming standards, nor does it necessarily obsolete an existing standards. Standards are not a race.

Next, I would like to mention that ISO has a very wide range of publication types, from International Standard to Technical Report to Publicly Available Specification, and it certainly may be the case that some technologies (such as obsolescent or fast-changing technologies) would be better made available using a lesser type.

My key point is that once a company’s success in an area brings it to the point of market domination (or long-term super-profit-taker) then anti-trust regulators need to ensure that their interface technologies are open enough for the usual level-playing field concerns to be addressed. It needs to be just a cost of doing business, once you reach a certain point.

Obviously this applies very directly to Microsoft. But IBM is also a market dominator (indeed, monopolist) in the mainframe game. And Google looks similar for search; Apple with the iPod; Adobe with PDF; and so on. There may be non-American companies that it applies to as well, I suppose. (And there are technologies that achieve market dominance outside a dominant vendor: PKWARE’s ZIP for example. These need to be standardized as well.)

The current standardization effort for DIS 29500 (Office Open XML) provides a good backdrop for this. For almost two decades, independent software developers have been calling on Microsoft to open up their interface and document formats: the SAMBA developers for example. In 2004, the European Union recommended to Microsoft that it should put its document formats forward (in an XML version) for international standardization. As Microsoft has done this, first through ECMA then at ISO, it is prompted a vicious campaign against the effort, lead by business rival IBM but also by stakeholders allied to some open source software substitutes for Microsoft Office. (In particular, stakeholders allied to the Sun lead Open Office application and the ODF format; note however that other Open Source stakeholders, notably those allied to Novel have welcomed the standardization effort.)

Some of the objections to DIS 29500 are in fact objections to the idea of a Microsoft-derived technology becoming a standard. In fact, several standards have come that way. The recent ISO Open Font standard, for example, it based on MicroSoft and Apple’s Open Type fonts (True Type, etc).

Other objections to DIS 29500 come from the other direction: DIS 29500 is flawed not because of what is in it, but because of what is outside its scope: media formats, macro languages, printer driver configurations, and so on. The most extreme version of this argument is to find a fault with DIS 29500 that it does not describe the (50 or so different) earlier binary formats for Office and its component products. (When this complaint is made in the same breath as saying that the 6000 page draft is too large, it does smack a little of insincerity.)

I usually respond by pointing out that a standard needs to be scoped, that standard is a work-in-progress, that it is impossible for ISO committees and National Bodies to have enough volunteers to do this work (both because experts are scarce and valuable, and because idealism is not stimulated by contact with market dominators.)

But I should have been articulating this: yes, there should be documentation for the binary formats, and indeed any format, API or protocol used for interfacing computer systems which gain market dominance. There does need to be more rather than less, and the cost (which are ultimately costs of decent QA and education, once the spurious controversy that has unsuccessfully attempted to derail DIS29500’s progress is over. (Of course, the lack of a tangentially related standard is no reason to reject a standard, we need to start from somewhere.)

Now many businesses are naturally coming to opening up their technologies in a similar way: look at Sun, for example, with its Open Solaris, Open SPARC, and its steps towards opening up Java. Public policy makers need to foster a procurement and regulatory environment where the winds of change for openness also blow refereshingly on market dominators.

The European Union was completely correct in asking for MS to adopt standard notations (XML) for the Office documents, and to standardize the schemas (through DIS29500), just as they were right to ask OASIS to standardize ODF (IS 26300). However, I hope this is the start of a larger movement for more: all document formats, all APIs, all protocols which have significant market domination need to be made available through one of the ISO standardization processes. All these interfaces are objects of legitimate interest for public policy for reasons of information ownership, level-playing field access, anti-trust, and even just from a procurement angle to ensure that systems are adequately documented and have had adequate attention paid to internationalization, harmonization, accessibility, and conformance testing: basic QA. And where the market dominating technology belongs to a market dominated by a single player (or cartel), then that player needs to bear the cost of the standardization effort, as a normal cost of business.

Now, my proposal here is not a total package: issues of conformance with external standards still need to be in place. After the statement “Here’s what we do” comes the natural question “Is it good enough?”, and that belongs in a separate blog.

ongoing ? On Communication

You see, if you draw the right graph, maybe you’ll see the gaping hole in it, the Next Big Thing.

I don’t know, Tim, but am I the only one that sees blips on a radar screen? ;-)

Who wouldn’t want to expand the human communication spectrum?

Absolutely!

Why aren’t more people thinking about this stuff?

TheyWe are. But instead of thinking and talking (Update: That process has been going on for quite some time now), we’re building and delivering.

BTW… Have I ever mentioned just how much I enjoy releasing projects on January 1st of each year? ;-)

(Something tells me 2008 is going to be a big year.)

[2:15pm] elarson: xmlhacker: you should patent the use of capital letters in association with the ‘:D’ emoticon during communication in order to protect your communications intellectual property ;-)

The above quote comes from Eric Larson in a recent IRC conversation after his recent post on “Patent Reform” ensued a conversation related to the 1-Click ordeal from 7 years ago (I was working on the Microsoft Passport team at the time, so those aware of that overall situation will understand what I mean when I say I had a first hand look at just how much fun dealing with the ugliness of the patent system truly is.) Those of you who have ever exchanged an email with me or have been a recipient of an email from one of the mailing lists I have posted to might recognize what Eric is referring to. For those who have not, at the bottom of all of my email communications you will find the following at the beginning of my signature,

/M:D

… which is intended to represent my first and middle initial, my first initial being implicitly bound to xmlns:M=”urn:publicid:Peterson:Patronymic+Surname:EN:1.2″ (<- If you don’t get it, don’t bother asking. It’s not all that clever *OR* funny ;-)) using an XML-ish namespace extension-based syntax.

Of course to those of you who have emoticons turned on in your email reader the above will translate to,

There is something about XML that makes people go crazy: in particular, people trying to make standards: its that ol’ tag fever agin Maude. I think I know what that thing is: the emphasis on standards = good combined with the desire for complete schemas and the idea that organizing schemas by namespace is the way to shoehorn requirements (rather than being a way of expressing results).

The result: vocabularies where unnecessary order and structuring constraints are given. You can tell when a standard schema is over-specified, because people using it will just snip out the low-level elements they need and plonk these in their own home-made container elements.

I have noticed this in a few schemas I have been working with recently: in fact, the trend I notice is that people start off with their own home-made schema, then “adopt” the standard by finding any elements that have close semantics to their home-made elements, and changing the name of the home-made element to the standard name. SVG in ODF looks like an example of this, and there is another standard I have been working with recently that has the same issue: when you adopt arbitrary portions of a cohesive standard, are you really using or abusing that standard?

I suppose there is a case to be made that transitional schemas should be treated seriously.

One software engineering idea that has stuck with me over the last years (which I wrote about in The XML & SGML Cookbook) is the twinning of cohesion and coupling. Basically, that when some information is highly coherent (think of Eve Maler’s Information Units) i.e., it belongs together semantically and would not make much sense in isolation, it deserves an official container.

Conversely, you should try to reduce coupling of information that is not cohesive.

A rule of thumb for many situations is that industry standard groups (and, indeed, inhouse schema developers), may be well advised to standardize data elements eagerly but container elements suspiciously: standardize the jellybeans not the jars. The next bloke may likes your jellybeans but have his own jars.

Various approaches to do this come to mind: think in terms of creating a vocabulary rather than a language; split your industry standard in two, with the tightly coupled elements in one normative section and the loosely-coupled elements in another non-normative section, perhaps with different namespaces even; use open content models and order-independence for loosely-coupled elements.

Another upside for this approach, is that it reduces the number of trivial issues for committee members to get excited about.

Campaign Widget | Creative Commons

Help support CC by putting this widget on your website, blog, or social network!

You don’t have to follow The Oil Drum to know that energy prices just keep climbing. Even if supply holds up, huge demand will make prices a problem for a long time to come. Can the Internet help reduce that demand?

Watch to the very end to understand the title.

TED | Talks | Larry Lessig: How creativity is being strangled by the law (video)

Larry Lessig gets TEDsters to their feet, whooping and whistling, following this elegant presentation of “three stories and an argument.” The Net’s most adored lawyer brings together John Philip Sousa, celestial copyrights, and the “ASCAP cartel” to build a case for creative freedom. He pins down the key shortcomings of our dusty, pre-digital intellectual property laws, and reveals how bad laws beget bad code. Then, in an homage to cutting-edge artistry, he throws in some of the most hilarious remixes you’ve ever seen.

Ok. So perhaps this is not a conspiracy because it’s out in the open, but ebay’s role in keeping feedback ratings artificially high is something worth discussing.

My argument is not about retaliatory feedback, but let’s discuss that briefly. Anyone who has used eBay much knows that feedback retaliation happens. You get treated badly, you leave feedback that says so, and the recipient leaves you bad feedback, sometimes even lying. This is a disincentive for leaving anything negative in the first place. eBay could take steps to make the system more fair, but they don’t. In fact, they have an incentive to leave the system exactly like it is. Retaliation discourages people from leaving bad feedback, and less bad feedback makes the entire marketplace look more trustworthy.

But that could be a bit of me creating a conspiracy, and perhaps eBay has better intentions than the previous paragraph gives them credit for. I considered this a possibility until one of my most recent forays into the depths of their system.

I was sold a counterfeit item on eBay. I paid about $100 for it and when it arrived it was obviously fake. After the seller did not respond to my emails, I filed a claim with Papal (which is owned by eBay for those of you who are not familiar). They offer “seller protection” designed to make sure you don’t get ripped off. Papal sent some messages back and forth and after about a month told me I would need to get the item appraised and send them evidence that it was counterfeit. This can cost hundreds of dollars, and discourages a cheated buyer from proceeding with the process, but let’s allow that it is necessary. On principle, I continued and found someone to certify that I had received a fake. After more than two months of fighting, PayPal finally resolved the dispute on my behalf and sent me a refund. That is all well and good. However, what really got me was the email they sent notifying me of all this:

PayPal has received the item in dispute. A refund will be issued to your
PayPal account within 5 business days.

Protection Services Department

“Amicably”? What was amicable about this claim? I spent a lot of money and effort trying to get a refund that the seller refused me for months. Why would PayPal tell me to consider this claim amicable when I leave feedback? Well, they have the same incentive as before. A marketplace with no negative feedback looks safer. But none of us want to participate in a system where a seller who regularly sends out counterfeit items is ranked highly, simply because eventually buyers can get their money back by the actions of a third party.

I do not hold out any hope that eBay, who is thriving, will correct their ways. I think it could eventually lead to a third party system (and several have popped up) for creating real and honest feedback about buyers and sellers. I would certainly use such a service - I might even pay for it - because I want to truly know how much to trust people I interact with. I don’t want to be falsely reassured that everything will be ok, even though that seems to be the tactic eBay is betting on for continued success.

To be continued? That’s what I intend to find out.

More when it seems appropriate to report more.

Embedding XML islands inside HTML documents is an old idea, and lately the debate about how to standardise this in HTML5 has been heating up again. As someone working on an HTML to PDF converter with strong XML support, I have a keen interest in the outcome of this debate. It would be very helpful if HTML and XML could be mixed and matched as necessary. So let me throw my five cents into the ring. (It would be two cents, but in this decimal age that would round down to zero).

Living in a place that used to think of itself as the bright future of America, it’s strange to me how people think that particular places will be the bright future.

Efforts have been underway recently to develop a schema language for JSON, analogous to the XML Schema Definition Language (XSD) or RelaxNG languages in the XML arena. Similarly, a JSON transformation language is being proposed and bandied about in various AJAX circles as web2 developers attempt to take the best of what XML has to offer and recast it from the angle-bracket modality to the braced modality.

These efforts are intriguing, and for the most part people within the XML community are now affecting the same rather confused expression on their face that I remember seeing on the SGML generation as they watched the young turks of the XML movement push their view of the world out to the world - “Didn’t we already DO that?”

Yes, I know that looking 43 years ahead is ridiculous for technology. But might it make sense for a place?

So my paper was accepted at XML 2007. I look forward to seeing some of you folks there. The schedule looks interesting not just because I see topics that I enjoy, and some about which I want to learn, but also because I see a lot of stuff that makes me think: “Oh, it’ll be fun to debate that one”. Anyway for my part I’ll be presenting “XML Data modeling for Web publishing workflow”, which is a pedestrian, but accurate title. I’ve been proud of the architectural expertise I’ve provided to help manage content workflow for Sun’s main sites over the past few years, but I’ve never been given permission to discuss it much. It will be fun to talk about some of that with Kristen Harris, the director with whom I’ve closely worked in that time.

June 13th, 2006

Chances are zero for XSLT2 in Orcas. Orcas is spec frozen now, and we can’t do a lot of work hoping that W3C will finalize the spec before we finalize the code. Also, Orcas is a tools release, so we can’t take anything other than critical bug fixes in the .NET 2.0 “red bits”. We are actively working on XSLT2 however, and there will be CTP releases starting in roughly the Orcas timeframe. It’s not clear how it will ultimately be released.

January 9th, 2007

Microsoft XML Team’s WebLog : XSLT 2.0

Our users have made it very clear that they want an XSLT 2.0 implementation once the Recommendation is complete. A team of XSLT experts is now in place to do this, the same people who have been working on the XSLT enhancements that will be shipped in the forthcoming “Orcas” release of Visual Studio / .NET 3.5. Orcas development work is winding down in advance of Beta releases over the next several months, so there is no possibility of shipping XSLT 2.0 in Orcas. The XSLT team will, however, be putting out Community Technology Previews (CTP) with the XSLT 2 functionality and appropriate tooling as the implementation matures. The eventual release date and ship vehicles (e.g. a future version of .NET or a standalone release over the Web) have not been determined, and depend on technical progress, customer demand, and other currently unknowable factors.

So Orcas Beta 2 released a few months back. Depending on your interpretation of various announcements, VS.NET 2008, which is quite stable already in Beta 2, will release somewhere between November and February. So that should mean we should expect an XSLT 2.0 CTP somewhere between now and March?

I must admit that I am getting quite excited by the anticipation of the first CTP release. Of course I can’t imagine MSFT would do something as silly as not deliver on something that the community fought as hard as we did to gain in the first place,

Our users have made it very clear that they want an XSLT 2.0 implementation once the Recommendation is complete.

… would they?

Not the MSFT XmlTeam I know and love**, that’s for sure!

A team of XSLT experts is now in place to do this, the same people who have been working on the XSLT enhancements that will be shipped in the forthcoming “Orcas” release of Visual Studio / .NET 3.5.

See!!! :D

** DISCLAIMER: The MSFT XmlTeam you know and [love|hate] might differ from the MSFT XmlTeam I know and, at present time, [HEART]. ;-)

Hey MSFT XmlTeam: How about a time frame update? Oh, and while we’re at it, will the XSLT 2.0 CTP source be a part of the forthcoming MS.NET Open Reference release? That would be *FANTASTIC*! :D

/me is waiting eagerly to hear when we all get to play with the first MS.NET XSLT 2.0 CTP release. :D

The bumpy ride to ISO standardisation of Microsoft Office Open XML is receiving a lot of attention here on XML.com, and drawing out a lot of strong opinions on both sides of the issue. Frankly, the intense coverage given to every minor detail in the specification bores me to tears, even though I see the need for it, and I think that there is a larger story behind these events that is not receiving enough attention.

One of the most startling aspects of the last year, to me, really shows the disruptive potential of standards: bitter enemies are hard at work making systems that also benefit their enemies in pursuit of a higher goal. A world turned upside down!

Examples include:

• MS opening up their formats and taking them to Ecma and ISO
• ODF and Sun making converters for Word
• Microsoft paying for an open source converter between OOXML and ODF
• IBM paying someone to review the draft specfication
• Open source activists reviewing the draft
• An ISO ODF editor making a big contribution to reviewing the draft
• Microsoft blogs publicizing Gnumeric, Apple software and any applications that get any kind of OOXML support

All this competition and bile channeled productively! No wonder people are freaked out. :-)

But the paradoxes don’t just mean that enemy act like friends, it seems. Friends also can get accused of being enemies. There is a very interesting post ODF vs OOX : Asking the wrong questions (hat tip to Doug) on the blog Spreadsheet Proctologist which I like very much because it brings out that ease-of-implementation is just as much (and perhaps mostly?) a question of what your starting base is (i.e. your native data structures and functions) as it is a question of what information and forms the external format provides.

But the readers’ comments include statements like Your self-annihilating devotion to Microsoft is too evident., and Just by touching MS OOXML, you are playing their pawn in the only purpose for this exercise. To kill ODF adaption and therefore the threat of Open Office and others as a replacement for Microsoft Office products. Is to laugh! Now GNU developers are pawns and devotees of Microsoft! That GNU software, ooooooh, just another Bill Gates plot!

As a side note, but related to the theme of finding strategies so to make the acts of people’s enemies as productive as the acts of their friends, I think that Stephane Rodriguez’ comments (to that blog and, just as circumspectly, elsewhere) on the calculation chain should be paid more attention to. (Sometime I will look up whether it made it to any national body comments for the BRM, I hope so.) Calc chain needs to be reviewed with the question asked “Is the base case a little too complicated still?” It is a mild and productive question: I suspect programmers would be happy if some more leeway were provided. Now whether the issue is an Office one or a DIS29500 one, I don’t know; but the issue should not be dismissed just because it was deposited by an ostensibly rabid whirlwind! Quite the reverse.

I’ve just glanced over the 3549 or so comments put in by various national bodies for the recent ballot on DIS 29500. I’ve made a table listing the countries that commented, together with their votes and whether I think most of their issues could be resolved during the upcoming Ballot Resolution Meeting next year.

The bottom line: there are a few touchstone issues that may be tricky but it is difficult to see from the comments that DIS 29500 would not be successfully fixed and approved to be an ISO standard. The particular touchstone issues I see are that spreadsheet dates need to be able to go before 1900, that DEVMODE issues need to be worked through more, that the retirement of VML needs to be handled now, and that there needs to be a better story for MathML.

Apart from these, there is a sea of details that are eminently fixable: typos, clarifications, fixing schemas against closed lists, the use of more standard notations for fields, encryption, conformance language, refactoring the spec: editorial and syntactic rather than data model or wholesale semantic changes. On the other extreme, there are various non-starters which I expect have little hope since they run counter to the rationale for the spec: adopting SVG or adding various frustrating little things in the name of compatibility with ODF (Some NBs even call for ODF’s blink element, even though blink has been removed from HTML since it can cause epileptic fits!)

You can find a full list of national votes from the SC34 website. I was pleased to see that all the issues I raised ended up in Standards Australia’s comments (it abstained on the vote, but its comments still go in the mix.)

What is in the table

The thing that interested me in this table was whether I thought each National Body’s comments could be resolved enough to change their No vote to Yes vote. I am assuming there is no point to a standard that Ecma and Microsoft could not buy into. One of most interesting documents in the collection of comments from different bodies is Ecma’s own contribution: basically they accept almost all of Japan’s technical issues (which have a lot of overlap) which augers well for many of the other changes.

So I provide a rating as to whether I expect that a National Body’s vote will be definitely no, probably no, or probably will change to yes as a result of a successful BRM. Caveat: The NB comments do provide a much clearer indication of each National Body’s thinking than just the raw Yes/No/Abstain vote (which are utterly useless in predicting a finally outcome); however, I would be a little more confident in my ratings of the NBs if SC34 or ISO had released information about which NBs had ticked the normal box that says they might change their mind if the issues were resolved. I guess you would rate me as an optimist in general about the process, but still I am not saying that all these NBs will necessarily vote yes ultimately; but there is quite a bit of commonality to the comments.

I also have columns marked “Indie” which has an X if it seems the NB undertook independent review of the specification. And one marked “Parrot” where the NB is reproducing (perhaps with some localization or sorting or selection) the material, turning the standards review process into a form-letter campaign. I have mixed feelings about parrot items: on the one hand an NB is free to consider whatever issues it likes, and some NBs have procedures that may favour the garrulous, but on the other hand it represents a hijacking of valuable review time to obsess on the same issues, rather than give fresh eyes.

The reviews that seem to me the best are those where an NB focuses on its areas of expertise or national interest: Japan is very interested in schemas, Israel is very interested in right-to-left text, Ireland is very interested in correct references, Australia is very interested in clarity, Canada is very interested in assistive technology, Tunisia is interested in the application to mobile devices, Ghana (with a large Arabic influence) is interested in IRIs, and so on. The comments that seem least useful are the parrot comments, and the ones with vague recommendations. (I expect that this is the first comments that many of the NB committees or staff have sent in, so it is a good training exercise nonetheless.)

And there are some nice touches in there, where perhaps some cultural values slip through: Switzerland’s comments are a list of problems they actually have rejected and the details why, and Jordan and Turkey both have dignified documents that explain their positive reasons. Some of the parroted comments are unnecessarily ranty, but only a few were mad: the US comments in one place want to remove OPC because it is not present in the “pre-existing binary format” but then they want to get rid of compatability elenents because they are a “museum”:..they don’t need to worry about consistency because they are voting yes anyway: some of the comments are like that, they are there to only allow the cake to be had and eaten. I expect that several NBs are not really attached to some of their comments.

The second last column is “Off-topic” which is where the NB’s comments includes material that the BRM cannot discuss. These are typically issues concerning IPR. MS needs to spend a bit more effort on this: Switzerland’s comment is really interesting on this point.

The final column marked “radical” is where a National Body’s comments include something that I think will be a challenge for MS or Ecma or ISO to support. I don’t include things like changing minor notations or providing better text explanations for things: I think the Ecma comments show a willingness to have those. However, where some change involves a wholesale alteration of the technology or its implementation, I would be surprised if it were acceptable. This is because for every nation that is voting “No” because they really prefer ODF, etc, there are two who are voting in favour because OOXML is what it is.

Update: To ensure proper context is propagated, as per Douglas Crockford’s follow-up comment below,

The context of my statement was Ajax data transfer. In that specific context, XML is in fact being replaced with JSON. I didn’t say anything about doing dishes.

Which, I believe, is an absolutely fair statement to make. In fact, if you were to run a couple of quick queries on the two pages this article spans you could determine quite easily that yes, in fact, this article had a heavy tendency to use the word ‘AJAX’,

Page1//html:p[contains(.,’AJAX’)] = http://personplacething.info/service/proxy/return-xml-from-html/?uri=http://www.infoworld.com/article/07/09/07/crockford-ajax_1.html//html:html/html:body//html:p[contains(.,’AJAX’)]

Page2//html:p[contains(.,’AJAX’)] = http://personplacething.info/service/proxy/return-xml-from-html/?uri=http://www.infoworld.com/article/07/09/07/crockford-ajax_2.html//html:html/html:body//html:p[contains(.,’AJAX’)]

What about dishes?

Page1//html:p[contains(.,’dishes’)] = http://personplacething.info/service/proxy/return-xml-from-html/?uri=http://www.infoworld.com/article/07/09/07/crockford-ajax_1.html//html:html/html:body//html:p[contains(.,’dishes’)]

Page2//html:p[contains(.,’dishes’)] = http://personplacething.info/service/proxy/return-xml-from-html/?uri=http://www.infoworld.com/article/07/09/07/crockford-ajax_2.html//html:html/html:body//html:p[contains(.,’dishes’)]

… Well, once again this statement can easily be determined to evaluate to true. Thanks for setting things straight, Douglas!

On a related note, isn’t it cool how you can combine something as ubiquitous as a URI path segment with something as ubiquitous as HTML turning the entire *live web* into an XPath query-able dynamic database as a result?

Anyone ever tried to do that same mapping with JSON?

Just wondering… ;-)

Enjoy your XML and JSON enhanced WebDevWeekends, everyone! :D

Update: I should quickly point out that my comment below regarding the comparison to Dave Winer had nothing to do with Crockford’s contributions to the development community — Douglas Crockford is a *HELLAVU* hacker and his contributions are both obvious and beautiful (as in Beautiful Code.) My statement was directly oriented towards his statement “Fortunately, XML has been replaced by JSON” which, if Dave Winer had invented JSON is exactly the kind of statement you would expect for him to make. Suggesting that XML has been replaced by JSON makes about as much sense as would suggesting that JSON has replaced data. While JSON is great as a data serialization format it doesn’t cook your dinner and clean your dishes. It’s a data container. A *NICE* data container, but a container none-the-less.

Anyway, just wanted to quickly clarify what I meant in my comparison below. Douglas Crockford doesn’t claim to have invented things he didn’t and then insist he be given credit regardless of the fact. And he most certainly knows how to write code like very few people on this planet are capable of. My apologies for making it seem I was suggesting otherwise.

[Original Post]

Web, AJAX slammed for deficiencies | InfoWorld | News | September 07, 2007 | By Paul Krill

XML is complicated and inefficient, he said. “Fortunately, XML has been replaced by JSON,” Crockford said. “This gives me some confidence that we can fix the standards in the Web. This is our first success at that.”

Yikes! And here I was thinking there would never again be another Dave Winer. ;-)

I’m mostly happy about how Living in Dryden has turned out, and every now and then I try to encourage other folks to do something similar. Judging by the growing list of Dryden weblogs, I’d say Dryden is doing pretty well in building a community of people writing on a regular basis about things that matter to them.

A couple of months ago I did an interview with fellow technologist Jon Udell. He wrote up some of the interview (though I don’t think those are really ‘laws’), and the full interview is now available.

I’m also hoping to lead a panel on Creating Local Life on the Global Web at the SXSW Interactive conference next March. Talking about Dryden, New York in the middle of Texas may seem a bit strange, especially at a web development conference attached to film and music shows, but there’s a lot to do there.

When the Internet and the Web first appeared, they seemed like great ways to reach large numbers of people who weren’t already connected to each other. People who lived in California could talk to people in Germany, Bangaladesh, South Africa, and New Hampshire, about common interests they couldn’t have easily shared before. In the past few years, though, it seems that we’re learning about how these technologies can help us communicate on a much smaller scale, helping us look beyond the walls and property lines of our homes to connect with our neighbors.

There’s a lot to discuss here, and it’s not just about places like Dryden. Some ‘local’ communities are local once a year, at a conference or event, while many have members coming and going. Online communications let people who have left the community stay in touch with what’s happening, and even build new connections while they’re away.

If you’re interested in seeing this at SXSW, visit the panel-picker and vote for (or against) the proposal I’ve put in. (The Panel Picker is open until September 21st.)

As seen earlier today on XSL-List (this relates directly to yesterdays post: Opera 9.5 and XSLT document() Function),

Note that it’s entirely conformant (and sometimes even useful) for an
implementation of document() to reject every URI you throw at it as “not
recognized” or “inaccessible”. It’s not conformant, however, to fail to
provide the function.

Interoperability and marketability, of course, demand rather more than mere
conformance.

Michael Kay
http://www.saxonica.com/

In the first few days of September, just as kids were beginning to head back to school, something rather remarkable happened: Microsoft lost its hegemony. In a vote by the various members of the ISO standards committee, managing only to come up 53% of the votes it needed to fast track the Office Open XML format for consideration as an ISO standard.

Now, Microsoft and its partisans will, most likely spin this as a temporary defeat, pointing to the number of No with comments votes that indicate that it may be possible, with some extensive work, to make OOXML workable in the near term future if some rough edges are smoothed out. In the end of the day Microsoft will have won, and those countries that were too narrow-minded focused on insuring that the standard that emerged was close to being workable with efforts will in the end change their minds, once the spec is cleaned up (and perhaps a few more people were added to the appropriate committees with positions financed by Microsoft money) that surely the OOXML standard will in fact become the de facto one very soon now.

We will (re)create it.

Please see this movie and then begin to think about what your creation(s) will be.

Thanks!

As seen at the top of a Yahoo!/B-Side co-branded shopping cart check-out screen,

NOTE: Our shopping cart requires you to enter your shipping address when you place an order. Yes, you have to do this even if you are only purchasing a download. Yes, that seems a bit, well, stupid. Hopefully, you can forgive us this minor inconvenience. Just enter your billing address for your shipping address, and we can all put this behind us and move on. - The management.

You’ve probably seen it. IBM’s Rob Weir’s 2006 diagram comparing the number of pages of various standards versus the time they spent in committee. It makes its appearance unchallenged regularly: indeed IBM (business rival of Microsoft)’s Bob Sutor gave the diagram a prominent place in his blog this week with what, presumably at this last stage, contains the essence of IBM’s argument against DIS 29500 and Office Open XML.

At the Standards Australia meeting, the diagram was brought out again, and I protested that it was misleading, but seeing Bob’s blog makes me want to explain my criticism more. Here is the scary diagram:

Digression

The issue of page count and book size is prone to publicity stunts. If you look at this web page, for example, you can see two different printouts of the open XML Spec, The first manages to fit in boxes under a man’s arms (and we don’t know how full the boxes are) while the second manages to be taller than a man! What can account for this doubling of size? Perhaps it is the magic of single sided printing and thick paper :-) (In the 1990s I was discussing a book with a publisher who said “it has to be 1.5″ thick, but if you don’t have enough material we will use thicker stock”! ) Say we have 6500 pages, and we print it at the maximum common paper weight of 105 weight Bond ledger, that gives us almost 3 metres of print out (10′)! But if we print it at the other minimum common weight of 16 weight, that gives us a tad over 50 cm (20″). On average paper weights, this should give about 64 cm (25″).

But back to the main story. I’ll deal with the issues I have with in reverse order of their seriousness.

Apples and Oranges

If you are using page size to compare documents, you really should make sure the documents are typeset the same. I moved the Open XML spec down from its extravagent 11pt body font and large heading spacing to follow the ISO standard 10pt.

Viola, I estimate that about 1,000 pages can be reduced by this. (Added: I estimate this because I tried it. I saved 800 pages on part 4 alone just by moving to 10pt and more typical ISO clause spacing. Technically, this is because there is so much display content and two-line paragraphs that get pushed to the next page, cascading with many paragraphs taking one line fewer.)

Difficulty of Review

The diagram uses page size as a unit of preparation and review. However, not all pages are equal. A page that contains normative text requires much more review than a page of informative text. A page that contains auto-generated text requires almost no review at all: you sample enough instances to have confidence in the autogeneration and then skip the rest.

Now this is especially relevant for DIS 29500, because it contains enormous amounts of non-normative/tutorial text and of autogenerated boilerplate. ODF editor Patrick Durusau this week tried a an experiment where he removed this fluff, and he reduced the WorkdprocessingML specification from about 1880 pages to about 600 pages (and he thought it could go a few hundred pages more!) Most standards avoid tutorial and non-normative material because it increases the tedium of the review process and confuses readers. A good tutorial is usually a bad standard, and vice versa. DIS 29500 is a really extreme example of this.

So lets say that only a quarter of the text is normative and non-autogenerated (based on Patriclk’s results, and considering the impact of the normative Part 3 and so on, And that the non-normative text and autogenerated text takes about 1/3 of the review effort. That means that, effectively for review purposes, the document requires only half the effort for the number of pages.

So divide the effective page size in half. (The legend “Number of pages” becomes “Review effort expressed in terms of equivalent number of normative pages”)

Time spent in Review

Now lets look at the other axis. Wier’s numbers here seem to be based on the time spent in committee before coming up for a vote. That might be interesting a year ago, but it is positively misleading a year later. Why is it still being bandied about like this?

In the case of ISO fast track standards, there is the whole review process by ISO that is omitted: the informal discussions with SC34 before submission, the 1 month administrative review period, the 1 month contradictions response period, the 5 month technical review period just coming to an end, and the ongoing review where each national body looks at each other’s comments over the next five and a half months before the Ballot Resolution Meeting in Geneva, which I expect to happen. That is a full year.

So add an extra 370 days there.

Nature of Review

The work that a committee does in compiling or creating a standard for a pre-existing technology is very different from what the work that a committee does in creating or augmenting a standard. When the proprietary Torx screws became an ISO standard, one can imagine that the committee had little to do. By contrast, the committee that produced the ISO PDF/X standard had a bit more to do, but still no where near what they would have to do if they were developing a standard fro scratch.

The work is review and discussions of policy, relieved of what-ifs and who-needs-this? As a completely conservative estimate, lets say that development of new material takes half the time, and review takes half the time.

Since we are measuring this in pages, lets be conservative and say that this relieves the committee process of 25% of its workload, and express that in effective pages.

Since we are looking at the workload of a committee, what about where a committee doesn’t have to author much, but is presented with a selection of workable drafts from the pre-existing documentation of a product? That is obviously a lot less work than writing for scratch, especially for the editor.

So lets say that this makes a committee 25% more effective, and express it in effective pages as before.

The other standards

Now, of course, to compare apples with apples, we would have to do the same procedure to the other standards, and they would move in the same kind of direction to a greater or lesser extent. But none of their shifts would be anywhere near as much as Open XML’s because it has the quintuple whammy of typesetting, fluff, the BRM, the lack of need of development, and pre-existing editorial material.

Furthermore, these other standards are not standing still. ODF has moved to ODF 1.1 with 1.2 in with works.

I have two other additional reasons why I think the diagram (or, at least, the way it is used) is misleading.

Ex nihilo?

The first reason is related to the last segments above. It is really not fair to compare a markup language for an old technology with a markup language for a new technology merely on the basis of the committee time. Microsoft moved into documenting text formats for its standards when it purchased RTF from DEC around 1990. A lot of the documentation in Open XML is adapted directly from the RTF and DOC documentation. Its basic strengths and weaknesses are well-known and long documented.

There have been perhaps fifty different versions of the .DOC format, on six different operating systems over the last twenty of more years. To ignore this history and just use committee time as the metric seems to me to miss out something important. A new standard does not come with all this prior work (and baggage).

I am not sure how to diagram this. Perhaps a line indicating the time the technology and documentation was in development before the start of the committee process? Lets date that from the advent of RTF rather than from the first .DOC format.

VML is a particular issue here: it was introduced into IE 5.5 and presented to the W3C committee. To ignore that early development and attempted standardization work seems to miss something important, again which is why I think we have have to be careful not to be mislead by the diagram.

Separate Technologies

Finally, my other problem with the diagram is that people use it to say “this is so big it cannot be reviewed”. However, Open XML is made from five or more completely distinct sublanguages: OPC, WordprocessingML, SpreadsheetML, PresentationML, DrawingML, VML, and then the extensions mechanism of Part 5. One person iis not expected to review a whole standard, it is done in co-operation with a committee. India is a good example here: they had separate task forces working on each of the three major application schemas.

So while the size of the draft in total is large, it can be decomposed into smaller sections and reviewed. There have been over 2200 people involved in national standards bodies reviews, I am told: that is a lot. If I was being as free with numbers as some people are, I would say that this represents about three pages per person! But of course, that would be just as flawed logic as accepting Rob’s diagram at face value.

So lets divide up the specification into its parts, and see where they fit on the chart. I’ll take into account the extra time for review, but just use the current raw page count for OPC (part 2), and the individual languages of Part 4 and 5. We get a diagram showing the size of each distinct (and therefore separately reviewable) sublanguage in page size of the current draft.

(If you select “View Image” or the equivalent in your browser, you will be able to see this a bit more clearly: the OReilly formatting system may get in the way here.)

And finally, lets have a look at what happens when we look at these separate languages, but get rid of the fluff as I suggested in the submission I sent to my national body for their consideration on the Australian vote. For WordprocessingML we will use the number that Patrick Durusau found when he stripped out the fluff: about 800. For the other largest four, we will just say that half is fluff, being conservative. (Actually, in my submission I want to remove some lists of examples such as border art to another part, but border art is hardly taxing on the reader.)

So this is a diagram of the estimate page count of normative pages in the component language standards of Open XML, against the time spent in Ecma and ISO development and review (and assuming a Ballot Resolution Meeting).

Note that this diagram does not include the “effective size” considerations above, so the position of the new items can be compared directly with the other pieces of data on the page, as apples to apples. To the extent that the other issues raised above apply to each language, their star would move left (and up); however, for a good comparison the other standards mentioned would also have to have their position adjusted in accordance to the same factors: however, as I mentioned, because the other technologies consist largely of normative material, the adjustment would not be as great; the other technologies might also need to have ISO process time added too, I don’t know whether Rob’s numbers include that or not (the effect would be add six to twelve months in an upward direction to some of the blue points.)

Bottom Line

So that is seven reasons why I think the diagram is misleading. Or, at least, why the diagram itself does not give data that is particularly useful for anything other than mindless sloganeering.

What I don’t understand is why people are not on to these kind of tricks. Big standard, ooh scary. Have people never heard of Adam Smith and the division of labour? Have people never changed font size and had a different sized document as a result? Do people think that all text is equally taxing for review? Do people think that adapting a standard from pre-existing text is not easier than writing (and indeed) developing the standard from scratch? I suspect that many people see that on the original graph the OOXML point lies so far to the right, and because pages are easily countable, they don’t have any alarm bells ring.

So let me ring your bell, if I may: what the original diagram tells us is that the standard has a lot of text. And that one stage of its life in a committee took about a year in 2006. both those things are such a partial piece of the picture (where is 2007?) that while they are of some sensational value, the diagram can be misleading.

Fedora Commons - About - News

Fedora Commons today announced the award of a four year, $4.9M grant from the Gordon and Betty Moore Foundation to develop the organizational and technical frameworks necessary to effect revolutionary change in how scientists, scholars, museums, libraries, and educators collaborate to produce, share, and preserve their digital intellectual creations. Fedora Commons is a new non-profit organization that will continue the mission of the Fedora Project, the successful open-source software collaboration between Cornell University and the University of Virginia. The Fedora Project evolved from the Flexible Extensible Digital Object Repository Architecture (Fedora) developed by researchers at Cornell Computing and Information Science.

Nice! Congratulations, Fedora Commons!

The press release continues,

Update: *EXCELLENT* follow-up post from Wladimir in which he closes with the following,

I guess I need to thank Danny for so many great articles in such a short time. On the other hand, maybe instead I should remind him that denial-of-service attacks are illegal, even in the USA.

I’ll let you come to your own conclusions as to what that last sentence is referring to, though I will point out the fact that no matter who you are or what you believe justifies your actions, while blocking ads is not a crime, DOS attacks and other forms of Internet harrasment and vandalism most certainly are.

If you are guilty of any such crimes, please don’t turn yourself into the authorities (our prisons are filled with too many people who shouldn’t be there in the first place), but please stop, think, and then find ways to get over whatever it is you are hung up on in a peaceful manner.

Thanks! Our Internet will be a better place if you are willing to consider the above request.

Update: Wladimir Palant, the *WONDERFUL* developer behind the *WONDERFUL* tool AdBlock Plus recently left the following comment that I thought the rest of you would find interesting,

Thank you for this article, it is real fun to read it. Btw, the numbers you were asking about - I don’t have exact numbers either but it seems that no more than 2% of Firefox users have Adblock Plus installed. Which makes this campaign as ridiculous as ever.

Of course one can only assume that after all of this attention, the number of AdBlock Plus users have increased, but not so much as to drastically change the above percentage to the point where any of the legitimate sites on the net in which use ad revenue as their primary support are going to be noticeably effected. In fact if you think about it, it’s quite possible that, while ever-so-slightly, the reduced cost in bandwidth savings from those who have no interest in the ads being displayed will *more* that offset any potential loss in ad revenue.

In fact, if you *really* think about it, if all of the people in which had no desire nor willingness to click on the ads presented on your site were to install AdBlock Plus there’s an ever-so-slighter (is slighter a word? Probably not, but today let’s make it an honorary word just for fun ;-) possibility that the net result will be that of increasing your cash flow instead of decreasing it.

Okay, maybe thats a bit of stretch, but if nothing else it’s definitely something to consider. Of course if it turns out this theory were to actually hold any water you would have none other than Wladimir Palant to thank for your decreased cost structure and therefore increase in monthly revenue. And according to the following forum entry from about this time last year (which was in response to a question regarding Wladimir’s preferred charity), here’s how you can thank him for your new found cash cow, ;-)

I don’t favor any organization, feel free to choose the one you like

Edit: On the other hand… I do favor one organization: http://www.mozilla.org/foundation/donate.html

Seems reasonable to me. :D

Thanks, Wladimir!

Update: NOTE: For those of you who first read this update at the top of my last post, here it is again but this time at the top of the correct post! ;-)

—
I *LOVE* this comment from an article linked to from Yours Truly (a handle, not a self reference ;-),

Upon clicking the link to http://whyfirefoxisblocked.com/ I was met with a blank page. Interesting, I thought to myself. Let’s check this out in more detail… I bet they want me to wipe the dust off my Internet Explorer and access their site that way. Admit defeat? Go back to using Internet Explorer? Hardly. I simply opened a new tab in Firefox and went to Google. In the Google search field I entered the search term: site:whyfirefoxisblocked.com and then loaded the conveniently offered “cached” version of the page in question. It loaded smoothly in my AdBlockPlus-enabled copy of Firefox.

Absolutely *CLASSIC*! :D Thanks for the laugh, Yours Truly! Of course the real test would be to do the same for the site that you would have been redirected from, but two things,

1) Why waste any more of your valuable time.
2) The spirit of your hack is most certainly in place, which leads to one very important observation,

As mentioned already: Don’t Fight the Internet! There’s fame (the good kind) and fortune and good times for all in whom find ways to embrace the way the web *truly* works, not the way you think it should work. And if anything this is the point of the entire post.

Update: Based on the evidence that has been mounting up in my inbox and in comments I’ve done a quick research project and have come to the same obvious conclusion that everyone else has: That the content that follows that now has a strike through is more than likely a completely bogus attempt at justification. My apologies to each of you that were simply following Digg, Slashdot, Reddit, and other links for proliferating the garbage that is being fed from this guy.

Oh, and Danny, (AKA Jack Lewis),

You know what, nevermind. Why even waste any more of my time.

No wait, I’m sorry, I do have something else to say: You are not a victim of terrorism. You’re a victim of yourself.

Best of luck to you.

Oh, and one other thing: If you are bothered by the ads on this or any other site and would rather read this or any other *FREE* content without being bothered by ads you find annoying: I’ve heard that Ad Block Plus is pretty good. Of course you’ll need Firefox if you don’t already have it, but if you’re interested in my opinion, Firefox is as good as a browser gets.

In fact, maybe even better.

Enjoy your ad free Firefox browsing days, everyone! The content here on O’ReillyNet is free to read however you might choose in whatever browser you might choose. If you choose to reprint it (beyond that which can be considered fair use) please do so under the terms of the Creative Commons by-nc-sa. Otherwise, do what you want. That’s your right.

And as always, thanks for reading! :D

Update: via a comment from Danny Carlton,

It’s my site, and if i want to control how people view it, I’m not letting a bunch of terrorists force me into changing that–and when you attempt to change someone’s behavior by threat of harm, you are a terrorist. The vile, obscene emails and phone calls, they attempts to shut down my server with DOS attacks and bandwidth eating programs, are all acts of terrorism, and it’s really interesting how many people who seem to get offended at being called “thieves” have no problems acting like terrorists.

Folks, I don’t care who you are or what it is you think you’re accomplishing, as far as I’m concerned anyone who involves themselves in this type of activity is absolutely as Danny specifies,

A criminal.

That’s absolutely shameful to do that kind of crap. You mind not be a criminal for blocking ads placed in the content you read, but you’re certainly a criminal if you take part in any of the crimes mentioned above.

Whoever is involved with the above: STOP!

It’s not funny. It’s not cool. And it certainly isn’t justified. It’s stupid. It’s illegal. And it needs to stop.

[Original Post]

Don’t fight the Internet! I promise, you’ll lose.

Why FireFox is Blocked

The Mozilla Foundation and its Commercial arm, the Mozilla Corporation, has allowed and endorsed Ad Block Plus, a plug-in that blocks advertisement on web sites and also prevents site owners from blocking people using it. Software that blocks all advertisement is an infringement of the rights of web site owners and developers. Numerous web sites exist in order to provide quality content in exchange for displaying ads. Accessing the content while blocking the ads, therefore would be no less than stealing. Millions of hard working people are being robbed of their time and effort by this type of software. Many site owners therefore install scripts that prevent people using ad blocking software from accessing their site. That is their right as the site owner to insist that the use of their resources accompanies the presence of the ads.

Here’s the thing: If people are going out of their way to block ads via Ad Block Plus do you honestly believe they represent a significant percentage of the +/-2.5% of the people who actually ever click on web ads in the first place? Wait, hold up, I think you answer your own question in the next paragraph down, but first let me take a quick moment to point something out,

Don’t you just love Jeffrey Zeldman? I know I do for the simple fact that he has no problem saying it like it is and in many cases he’s right on the money,

Jeffrey Zeldman Presents : What crisis?

The glacial pace of the W3C has given browser makers time to understand and more correctly implement existing standards. It has also given designers and developers time to understand, fall in love with, and add new abilities to existing standards.

So the glacial pace can’t be the crisis. Maybe the problem is lack of leadership. One worries about the declining relevance of The Web Standards Project. (Note the capital “T” in “The”–people who believe in standards should also believe in and follow style guides.) One has worried about the declining relevance of The Web Standards Project since 2002.

Nicely stated! Of course, just a paragraph or two above Jeffrey asks the question,

As per a comment I made to a post from Eric Larson to the internal Vibe* mailing list regarding the usage of Mercurial instead of Subversion for our RCS,

Of course maybe someone will come along and create a BitTorrent-based Darcs or Mercurial plug-in. Now *THAT* would be cool! :D

My point was in relation to the fact that with a decentralized RCS (which in most cases creates an exact copy of the repository with each checkout), as the size of the repository increases so does the cost of hosting that repository with each new checkout. But if a BitTorrent plugin were to suddenly surface?

Like I said, “Now *THAT* would be cool! :D”

Anybody care to become the *WORLDS BIGGEST ROCKSTAR CODER*? This would certainly be one way of becoming just that. :D

Dare Obasanjo aka Carnage4Life - Google Working on Social Network Aggregator

What I find more interesting is being able to bridge these communities instead of worrying about the 1% of users who hop from community to community like crack addled humming birds skipping from flower to flower.

Over the last month I have been collecting examples for fun from the web where scuttlebutt on the websites of well-known commentators has claimed procedural or other irregularities at standards bodies or participants. I started this off on the luridly titled “Bribery Watch page, but it is more “Innuendo Watch.”

Here is a little map (drawn dynamically) with the countries mentioned in red.

Some of the claims have a French farce aspect. For example a mistranslation of “seat” and “chair” caused a great flurry.

However, one persistent theme is the idea that the industry people who actually want a standard should not participate in the standards process. Sometimes there seems to be some idea of neutrality floated, sometimes some idea that people who come late have less legitimate opinions than people who come early, othertimes that the process is flawed unless people are allowed late. But the basic idea is that if you agree with MS on anything or have had any business connection with them, they own you, perhaps even bribed you, and your every opinion is inappropriate. But never an acknowledgment that standards are community self-help efforts participated in, for the most part, by the parties who want to use the standard; and that the standards process is not a tool for cartelization.

I’ll use this entry as an anchor for my observations on the final day of Extreme Markup Languages. I’ll update it with a note each time a new talk begins, but I’ll add my comments on the talk in the comments section. There is a numbering scheme for the talks, to correlate to comments.

If you happen to be reading this in an aggregator, much of the meat is in the comments, so you might want to click through.

D4.1. “Lessons from monitoring the hedge funds: Markup identifies and delineates. Does it give your position to the enemy?”, Walter Perry

D4.2. “Declarative specification of XML document fixup”, Henry S. Thompson

D4.3. “Topic maps, RDF, and mushroom lasagne”, C.M. Sperberg-McQueen

Update: It just keeps getting better. Or is it worse? Guess that depends on your perspective. And with that, from a Wired News article from two days ago,

Crew Member: Previous AT&T Show Had “No Politics” Policy
By Eliot Van Buskirk August 13, 2007 | 10:26:44 AM Categories: AT&T

A crew member who worked on a show webcast by AT&T confirmed that there was a policy in place to remove artists’ political comments from shows before they were webcast.

“I can definitively say that at a previous event where AT&T was covering the show, the instructions were to shut it down if there was any swearing or if anybody starts getting political. Granted, they didn’t say to shut down any Anti-Bush comments or anything specific to any point of view or party, but ‘getting political’ was mentioned.”

The crew member went on to say that the order to mute political speech was issued by Davie Brown Entertainment, which had been hired by AT&T to produce the recordings.

Sure, the policy — which AT&T initially denied was in place — applies to all political speech, not just criticism of Bush. But most bands, when they get political, tend to lean pretty hard to the left (especially when they’re on the stage of Lollapalooza, which is trying to hang onto a rebellious, “alternative” reputation).

Randall L. Stephenson, the CEO of AT&T, is also the Vice-Chairman of the President’s National Security Telecommunications Advisory Committee, and has motivation to shield Bush from criticism. And as some readers of this blog have pointed out, AT&T is free to do whatever it wants to the audio on its webcasts.

But one has to wonder whether the same political filtering policy applied to AT&T’s webcasts could eventually affect to the company’s portion of the internet backbone, in the absence of the net neutrality legislation it actively opposes.

PLEASE NOTE: I believe it’s important I point out the fact that I personally am not Anti-Bush. In fact, I voted for him in both 2000 and 2004. Did I make a mistake in doing so? Well, that’s neither here nor there as there’s nothing I can do to change the past, only learn from it. Even still, as per a post I made a year ago last February,

Australian national body Standards Australia had an industry forum today on Open XML. The agenda and invitation for this is up at Tom Worthington’s website.

The Invitation

Here is the most interesting part of the invitation:

This forum is being conducted by Standards Australia as a courtesy to stakeholders. It is an extraordinary meeting that we are not required to hold, but do so to provide an open process. We appreciate your attendance and expect that you appreciate our effort in making this opportunity available to you.

Standards Australia values its vote as a participating member of all international committees, and does not exercise it injudiciously. We provide considered Australian viewpoints that are beneficial to Australian stakeholders, including industry, government, academia and the general community, through the facilitation of trade and the inclusion of clear Australian requirements in international standards.

The JTC1 process has established that the ECMA-376 document is not contradictory to existing standards and ECMA has responded to a number of technical considerations raised in the initial consultation period. This forum is not to debate the merits of the JTC1 decision making process or the validity of the ECMA response.

While technical comments are welcomed, it would be entirely counter productive to use this forum to reiterate technical comments that have already been raised and are likely to be debated in every JTC1 member body in some form.

We are looking for creative, positive contributions that emphasise our commitment to representing truly Australian views to the international community.

More on that later.

The Speakers

The meeting had 30 to 35 attendees (I didn’t count, oops), based on the membership of existing technical committees and people who had sent in comments to the process so far. It was not a voting meeting at all, just a meeting to help consensus and to give more information to higher committee members. (However, participants can submit comments by Aug 21 for consideration by the Australian CITeC Standards Sector Board.)

The meeting was a three hour affair, with the first half invited speakers and the second half question and answer and commentary.

The first half started with an introduction by Standard Australia’s Alistair Tegart, who provided good strong chairmanship that left most people frustrated that they had not had a chance to say more, but which gave everyone a chance to make their most important comments in the allotted time. The interesting thing was that discussion of technical minutae was strongly discouraged (wrong meeting for that), which is a nice break for me. Discussion was civil, everyone friendly in the coffee break, and frank in the meeting.

I had been invited to speak on the subject of General overview of the standards process because of my involvement as Australian delegate to (what is now) SC34 in the 1990s and my continuing involvement with standards. A nice comment afterwards (by a law professor!) was that mine was the only talk with new content. I tried to present an SC34-based perspective on standards: what SC34 standards are, how the preference for enabling standards rather than applications has been overtaken by the fast-track process, the basic standards posture for Australia in SC34 in the mid-90s (need for simplicity to suit our small development teams, I didn’t mention support for regional neighbours though it was important) and how each different country has different requirements. (For example, some countries have a requirement that they do not want to be blocked out from international contracts because of the lack of standards.)

Then a quick mention of some of the issues that I prototype in this blog: that ISO standards for documents are voluntary, that standards form a library of choices, that the mere existence of alternative standards does not prevent any group from choosing one over the other, that standards such as PDF and Torx are not open in the sense of allowing arbitrary change but nevertheless valuable, and so on. I emphasized again that the ISO process is a win/win system in which attempts by one group to stymie another’s needs does not fit.

That took about 15 minutes, then there were speakers on the case against the adoption of Open XML (the scheduled IBM speaker was hospitalized so we were treated to an emergency podcast from Rob Weir which was basically the same content as his Technical Case against OOXML.) and for the adoption: a quick tag team with an MS representative, then the local CompTIA representative, then a CEO. The CEO, Richard White from CargoWise EDI, was particularly forceful on how it would help his business.

The Discussion

Then after coffee we had over an hour of moderated discussion. By and large it went as expected: people from local industry welcomed it as solving a real problem, people from business rivals of MS didn’t like it, people who identified themselves with Open Source didn’t like it, people from academia or standards bodies seemed to think that having it as a standard would somehow force them use it (I didn’t get this.)

I had to gag myself a few times. The local Google Maps operation was represented, but I was quite surprised to hear Lars Rasmussen say how difficult it would be to implement Open XML…surprised because he had told me last year how Google maps used VML to deliver to IE and how format was simply not a problem. (Here is the first line sent for a Google map, for confirmation: note the namespace declaration and stylesheet reference:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml">
<head><meta http-equiv="content-type" content="text/html; charset=UTF-8"/><noscript>
<meta http-equiv="refresh" content="0; URL=http://maps.google.com.au/?output=html"/><<noscript>
<title>Google Maps</title>
<link rel="stylesheet" type="text/css" href="http://www.google.com/intl/en_au/mapfiles/86/maps2.css" />
<style type="text/css">body{margin-top: 3px;margin-bottom: 0;margin-left: 8px;}
#vp {position: absolute;top: -10px;left: -10px;width: 1px;height: 1px;visibility: hidden;}
#homestate {display: none;}
v:* {behavior: url(#default#VML);}   ...

It seemed strange to be saying that a technology was too big to implement when you were in fact using that technology successfully. Maybe the Google speaker didn’t realize that VML is a part, though obsolescent, of DIS 29500. I think what happens is that “implement” gets stretched to mean “implement all the parts of a specification”: so “It is too big to implement” means “It is too big to implement it all” in Googlespeak. But Australians aren’t going to implement a full new office suite. It is too big, even if you just used ODF; and Open Source people will more naturally join the existing Open Source and Free projects rather than set up new ones, it seems to me. For Australian requirements, “full implementation from scratch” is an imaginary and spurious requirements.

What has maybe slipped Lee’s mind is that most integrators will use DIS 29500 in the same way that Google Maps would be using it: just cherry picking the parts that are needed (in their case, the subset of VML.) And, in particular, when you are using it as an end-format, you only need to “implement” (i.e. generate) the elements that correspond to your input. Not the whole thing.

When I was talking to the local Google people last year, they told me that Google doesn’t actually have any fulltime people allocated to standards work in general. I gathered that was a little pedestrian for them, because they made their money by innovating not by following the pack: sounds like a recipe for QA disaster to me. I don’t know whether their foray into web-based applications will make them a little more savvy with standards.

Another Google guy (who turned out to be a ring-in: Georg Greve, initiator and president of the Free Software Foundation Europe, who gfim says was flown in from Switzerland by Google especially for the occasion) stood up and recommended we should track what the Indian standards body’s concerns about binary mappings. Again I had to gag myself (actually, Alistair did it for me) because I believe I actually was present at the meeting in Delhi where that issue was raised: by the Indian representatives of Sun and IBM. I am afraid I couldn’t help thinking this was the classic Colbert “Echo Chamber” effect, similar to Wikipedia’s astroturfing: one member of a collective puts something up in one forum, then other members of the same collective bring up the first as independent evidence. (In this case with the added twist that Sun and IBM were not mentioned: a lay listener could easily have had the impression that this was some kind of position adopted by the BIS, whereas, as far as I know, it has not so been. I hope the Georg will be a little more careful with attributions in the future, because people can so easily get the wrong impression.)

An interesting comment from local IT29? committee head, Jamie (surname illegible sorry), that for educational users, they needed to guarantee interoperability and could not force students to purchase particular programs. I didn’t uite see the logic of how this meant that DIS 29500 should not be adopted at ISO. Marcus Carr from Allette Systems (who I consult for and teach standards seminars by) sits on a local IT committee too and responded that students would be better served by PDF if guaranteed interoperability were the issue.

A few comments later, I got a chance to mention that none of the XML formats today provide guaranteed interoperability in the sense of visual fidelity, for the reasons that readers of this blog will be familiar with: every application supports different feature sets, has different fonts, hyphenation dictionaries, kerning tables, line break algorithms, and so on. Plus the formats are extensible, so can have all sorts of strange media types. Standard documents may perhaps be a necessary condition, but they certainly are not sufficient. What is needed are profiles, which restrict the features, requires certain application behaviours and require certain fonts. And for uncramped page designs that reduce the chance of page overflow on different systems.

Several speakers raised the issue of IP rights and worries about the MS Covenant Not to Sue and the Open Specification Promise. MS gave the usual response: we have run it past lots of external lawyers who say it is fine, and since OSP is so similar to Sun’s equivalent, why don’t you have the same concerns about ODF. I think Standards Australia has been playing a little coy here, because they are trying to be scrupulous not to be seen to take sides, I guess. I had asked them in email to have a clear position on standards and IP from the legal perspective, and they ended up saying, in effect, that for Standards Australia, it is JTC1s responsibility and competency to evaluate the IP issues of drafts submitted for Fast-Tracking, and not a technical issue for voting.

I think a better and more complete answer would be better. People who are interested in this are should first read the excellent webpage by OASIS lawyer and anti-Open XML conduit Andy Updegrove, especially on the Allied Tubemakers and Dell cases. Standards don’t exist in a vacuum, and MS standard’s participation and the very strong and constant statements that MS would be considered in any court.

Another aspect of the IP discussions is that typesetting and desktop applications are not now a new thing. With a 20 year limit, patents before 1987 have expired, which is well after the invention all of the basic ideas in office suite software. (Last week in Thailand, MS’ Oliver Bell was asked a question on this issue, and IIRC he said that actually MS only uses its patent portfolio defensively and has never sued on IP. Does anyone have a list on this?)

Jamie ? pointed out that participation in a standards body does not nullify the IP; however, the issue is the scant chance that submarine patents are enforceable. So add the two covenants, external legal opinion, vetting by Ecma and ISO/IEC JTC1, the age of likely patents, the recent stronger court awareness of junk patents, the difficulty of enforcing submarine patents, the multiple statements made by MS executives and staff to the highest level, and the basic fact that a document standard is more concerned with schemas and general description and no of methods or algorithms, and I don’t know how much more would be possible to satisfy someone.

One comment mentioned the idea that MS covenant not to sue etc does not cover external technologies. Of course, not, but that is no different for ODF and HTML.

Other speakers brought up a few of the usual suspects. autoSpaceLikeWord95 made its scheduled appearance, along with statements that made it clear that the speaker had never read the spec and was parroting. There is an easy way to tell a parrot in this area: they will say something like “The standard is full of compatibility elements like autoSpaceLikeWord95 which are undocumented and prevent implementation”. In fact, there are 64 compatibility elements, and IIRC correctly all but two are adequately document with explanations of their general functionality. AutoSpaceLikeWord95 is optional and is clearly marked as deprecated: it seems to be a warning flag that some document was originally created by Word95 with this bug and it had never been corrected. The bug is related to the treatment of Fullwidth character used in East Asian typesetting (zenkaku): certainly for Australian users it is utterly extraneous to our national requirements.

The issue of the definition of various functions in SpreadsheetML came up too, from a localization perspective. (Again, irrelevant to Australia.) If the moderator hadn’t been so tough, I would have liked to have asked whether the speaker wanted to remove or rename the existing function (and break everyone who used this function’s spreadsheets) or merely to add better localized functions (which belongs in a maintenance phase.)

On the issue of maintenance, I did get another opportunity to spout. I said that it is too early to tell what effective systems for maintenance will occur. When OASIS and ECMA submit their standards, they also submit information about how maintenance will occur. There would be collaboration with JTC1 SC34, for example. I said I thought this was only practicable for fast corrigenda (which don’t add functionality just fix the text and clear mistakes) and that the approach that OASIS seem to be taking, which would involve resubmitting ODF 1.1 and ODF 1.2 etc for fast-tracking each time, was probably the more realistic thing to expect. However, I noted that DIS 29500 has a quite strong extension regime, indeed a whole part (Part 5) and starts from a much more complete position that ODF: so one would expect complete updates to be rare events, perhaps aligned to the three-year product cycle.

The main Google guy at some stage made a good point about overlapping standards, along the lines that having multiple standards for programming languages was OK because the differences could be justified, but he had not heard arguments why Open XML was so different from ODF that it could be justified.

A few others also had comments that could be fairly reduced to “We don’t need it, therefore we do not support it becoming an ISO standard, therefore we appose it becoming an ISO standard” which is a non sequitur.

Baseline formats and downstream formats

A very interesting point was made by the National Archives representative. They don’t have the resources to cope with Open XML and ODF, he said, so they would adopt ODF for their future format and didn’t support Open XML becoming a standard. Again, I don’t see how the standardization of Open XML forces their policy in any way. Standards Australia is not even a government agency, and has no legal clout on the National Archives: moving to ODF where possible seems a reasonable choice (well, ODF 1.3).

Marcus Carr objected to this. He spoke from the perspective of document processing from the early 90s, and the difficulties in practice of dealing with Word documents (with the various hijinks: converting .DOC to the Rainbow DTD, converting .DOC to RTF then processing that, etc) and brought up the key processing issue that I think almost all the commentators on Open XML miss. He brought up the issue of the need for a full-fidelity baseline schema to allow the most flexibility in downstream processing.

Now this is a pipeline approach that has proven itself to work over the last 15 years we have been using it. Elephantine readers may remember a blog of mine a year ago:

A typical strategy when converting from XML into some structured text format is to have three transformations:

* first, convert the XML into ideal XML: resolve links as needed, remove extraneous elements and attributes, convert cases, generate headings and other things that need to be generated
* second, convert that ideal XML into an XML-ized version of the output format
* third, convert the output XML into the text format, delimiting and indenting as needed

If the input data is non-XML, then we have an additional stage where we first convert the data into an “baseline” XML format that maintains all the information from the data source (it could be a database, another format, a binary, no matter.) You never know what information you need, and you don’t want to trust someone else’s abstraction but work with as unmediated form of the data as possible.

So Marcus’ comments was that we (the system integration and document processing community need Open XML as an ISO standard, because it alone provides an adequate baseline format for subsequent transformations. So the Australian National Archives could well decide to archive data using ODF, but they may well decide to implement their conversion to ODF by going through Open XML. So one standard would be useful for one purpose (saving future archives), the other standard for a different purpose (opening existing files in the archive.) If the Australian National Archive is moving to ODF 1.0 fast, I hope they don’t throw away the original binaries…

Breadcrumbs

So all in all, I think the day was a worthwhile exercise, and a good opportunity to help us all escape groupthink.

I suspect, from the tone of the invitation and comments made at the meeting and elsewhere, that when Standards Australia looks at the comments that people send in (deadline August 21) they will be completely disinterested in comments that question JTC1 decisions and comments on issues that have no local relevance. I gather they may not be much impressed by arguments that can be refuted by precedent: for example, that there should be no overlapping standards. However, we shall see, and I don’t know anything about the CITeC Standards Sector Board.

The most enlightened part of their approach, and I think this is pretty novel, is that Standards Australia seem very aware that the role of an individual standards body in vetting a standard when there is a multi-national campaign to discredit it (on the one hand) and promote it (on the other) changes the requirements for a review. In the case of a normal standard review, you raise as many (sensible) flaws as you can, because you don’t know whether the issue will be addressed by anyone else. In the case of a global campaign, it is clear that almost every National Body has been mail-bombed with the same speil and that therefore those are issues that we can actually ignore, unless they have a clear national significance because we know that other national bodies will be examining them. I think that is what may be behind the last line of the invitation
We are looking for creative, positive contributions that emphasise our commitment to representing truly Australian views to the international community: they want to husband their resources to what is important for local industry and local requirements. They don’t want to succumb to a Denial of Service attack where by concentrating on sorting out edge cases and typos they miss out the big picture of national interest.

I’m preparing my comments to the CITeC Standards Sector Board at the moment, and I will put them online here too, if anyone is interested.

I’ll use this entry as an anchor for my observations on the third day of Extreme Markup Languages. I’ll update it with a note each time a new talk begins, but I’ll add my comments on the talk in the comments section. There is a numbering scheme for the talks, to correlate to comments.

If you happen to be reading this in an aggregator, much of the meat is in the comments, so you might want to click through.

D3.1. “Principles, patterns, and procedures of XML schema design: Reporting from the XBlog project”, Anne Brüggemann-Klein, Thomas Schöpf, Karlheinz Toni

D3.2. ” Enhancing AIML Bots using semantic web technologies”, Eric Freese

D3.3. “Converting into pattern-based schemas: A formal approach”, Antonina Dattolo, Angelo Di Iorio, Silvia Duca, Antonio Angelo Feliziani, Fabio Vitali

D3.6. “Relational database preservation through XML modeling”, José Carlos Ramalho, Miguel Ferreira, Luís Francisco da Cunha Cardoso de Faria, Rui Castro

D3.7. ” Mind the Gap: Seeking holes in the markup-related standards suite”, Chris Lilley, James David Mason, Mary McRae

One of the most odd comments that is coming up on DIS 29500 is that plain old XML is not human readable. I would love to hear an explanation of this. A string of characters saved in a text file with a .xsd extension is not human readable, but exactly the same string when cut and pasted into a word processor is human-readable?

(To forestall talking in circles, this is not about whether XSD is baroque, nor whether a human who can read XML can then necessarily understand the intended semantics of the markup.)

I’ll use this entry as an anchor for my observations on the second day of Extreme Markup Languages. I’ll update it with a note each time a new talk begins, but I’ll add my comments on the talk in the comments section. There is a numbering scheme for the talks, to correlate to comments.

If you happen to be reading this in an aggregator, much of the meat is in the comments, so you might want to click through.

D2.1. “Retiring your metadata shoehorn (For OpenOffice documents)”, Patrick Durusau

D2.2. “Localization of schema languages”, Felix Sasaki

D2.5. “Semantic resolvers for semantic web glasses”, Nikita Ogievetsky

I’ll use this entry as an anchor for my observations on the first day of Extreme Markup Languages (See also: Looking forward to Extreme Markup Languages). I’ll update it with a note each time a new talk begins, but I’ll add my comments on the talk in the comments section. I also added a numbering scheme for the talks, to correlate to comments.

If you happen to be reading this in an aggregator, much of the meat is in the comments, so you might want to click through.

D1.1. B. Tommie Usdin, one of the organizers of opens up the conference with “Riding the wave, riding for a fall, or just along for the ride?”

D1.2. “Easy RDF for real-life system modeling”, Thomas B. Passin

D1.3. “Writing an XSLT optimizer in XSLT”, Michael Kay

D1.4. “From Word to XML to mobile devices” , David Lee

D1.5. “MYCAREVENT: OWL and the automotive repair information supply chain”, Martin Bryan & Jay Cousins (Martin presented alone)

D1.6 “Advanced approaches to XML document validation”, Petr Nalevka, Jirka Kosek (Petr presented alone)

I’m still getting my Weblogger profile here updated, but this year I transitioned from one company I co-founded to another. Zepheira provides data architecture solutions, with a focus on semantic technology. I was early on the Semantic Web bandwagon, and I almost fell off at one point because I felt the useful, modest ideas at the core had been overrun by an academic brand of technological megalomania. This year I felt the timing was right to not only renew my interest in the technology, but to stake my livelihood on it. Part of it was timing: I was starting to see the more useful underpinnings of semantic technology take hold in corporations. Part of it was people: I found a group of professionals who I believed were capable of building practical semantic technology solutions, and, more importantly, selling them.

One of those people, Eric Miller, former chair of the W3C Semantic Web Activity, is especially well known for describing the benefits of semantic technology in terms executives can appreciate, and he’s featured in a new InternetWeek article “The Semantic Web Goes to Work”. The article says:

“You[’d] better figure out what the Semantic Web is and soon, because its concepts have graduated from academia and are starting to contribute to your competitor’s bottom line.”

I’m hearing a lot of that sort of thing, lately. The pundits, having written off semantic technology as so much pipe-dreaming for so long have switched into a level of hype overdrive. The reality is that as Eric puts it in the article, a consistent, universal system of identifiers and a layer of technologies for mapping these identifiers is the sweet spot of semantic technology. Semantic Web technology is the specialization that builds these identifiers on Web technology, and in particular URIs. This opens up the benefits of REST architecture, and for me that the third pillar is the universal writing system provided by XML. These are all, individually, modest technologies. Hardly nanotech, quantum mechanics or genetic engineering. But take these three and combine them with a skilled data architect and I do believe very special things are possible. There’s a large crowd of folks who still make the free association from “semantic” to “metacrap”, but that presents nothing but a ripe opportunity for others who know how to keep it simple, and thus get real work done.

The article also mentions Eric’s keynote at the Semantic Web Strategies conference, which is chaired by fellow XML-meets-semantic-tech pragmatist Bob DuCharme this October. I’ll also be on a keynote panel, and I’ll be co-presenting with Kristen Harris, long-time collaborator at Sun about how we improved content architecture for Sun’s mail Web sites using Semantic technology and REST.

This conference, organized by IT industry watchers Jupitermedia, is just another indication of how seriously folks are starting to take this stuff. I almost cringe that the stampede could end up ruining the crop, but that’s a test every worthwhile technology must endure at some point.

I’ll be off to Extreme Markup Languages 2007 on Monday. It’s will be my first time, and I’m excited because it’s always been one of those conferences I’ve wanted to attend, but August is usually not a good time for such things on my calendar. I’ve always heard that it’s a brilliant conference, and my French friends always tell me Montreal is a very fun city (doesn’t stop them from poking fun at the French-Canadian accent).

Some of the talks I’m especially looking forward to are:

* Writing an XSLT optimizer in XSLT
* Advanced approaches to XML document validation
* Retiring your metadata shoehorn (for OpenOffice documents)
* Localization of schema languages

There are many other juicy -looking talks, but the above really stood out for me at first glance.

I hope to meet up with many old colleagues, and make some new acquaintances at the conference, and I’ll be reporting often from this Weblog. I might even try a bit of live-blogging.

Update: As I pointed out in a follow-up comment to Woof, one of the things I absolutely love about blogging is that is encourages interaction and communication on important subject matter that would otherwise not take place if the medium did not exist. Often times I find myself having to reevaluate my position on any given subject matter because someone has forced me to do just that via a blog post they’ve written or via a follow-up comment to one of my own blog posts. Such is the case I am currently faced with,

But I groan over this particular post because you generally attack rules of clear writing, which is all S&W (and others like them) are trying to promote, with a class of “hey, man, WHO’S TO TELL ME” middle-fingered hubris.

Of course, as I pointed out in another follow-up,

Put this way your position becomes quite a bit more clear. And I can’t help but agree with your point.

… which is absolutely the situation I am currently left pondering. That doesn’t mean I believe the content of this post is no longer relevant, and instead that there is certainly more to this than meets the eye, a fact of which is forcing me to reevaluate my overall position.

Of course, life could be worse. I could go around thinking that my viewpoints were always and without a doubt the correct viewpoints and that everyone else who disagreed was, in fact, wrong. If there is one thing I have learned in life it’s that you don’t *ever* want to be “that guy.”

Thanks for helping me realize the flaws in my argument, Woof! Still thinking through this a bit, but once I have I’ll provide a follow-up comment with the results of my reevaluation.

Update: via Piers Hollot we have ourselves a new quote-of-the-day, week, month, and possibly even year,

To their credit, messrs Strunk and White had no way of knowing that semicolons, hyphens and parentheses could also function as winking faces.

*YES*! :D Thanks for the laugh, Piers! :D

[Original Post]
Coding Horror: Google’s Number One UI Mistake

Strunk and White urged us to Omit Needless Words:

Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell.

If you were to ask me who I believe to be the greatest writer of our era, that answer would be immediate and definitive,

Tom Robbins

Of course I doubt Strunk and White would agree. “Too wordy. Too much personal expression. Too much social and political undercurrent. Too much. Too much. TOO MUCH!” would of course be five too many too’s for Strunk and White’s liking.

A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts.

Of course if this were truly the case, there would be no need for erasers. In fact, there would be no need for pencils. Everything would simply be written in permanent ink.

In the same sense, there would be no need for code refactoring tools and there would only be one programming language. I mean, why mess around with Ruby or Python when you can write everything in assembly, or better yet, machine!

Hell, for that matter there would probably only be one spoken language if Strunk and White had things their way. Why waste our time describing things differently than someone else when it’s much more efficient to describe them exactly the same way as everyone else? With such efficiency we could spend less of our time communicating and more of our time…

– doing —

… Hmmm, I don’t know… Staring at the wall? Watching the paint dry? Or would the paint already be dry in such a world? For that matter, why even paint the wall! What’s wrong with the wall the way it was in the first place?! In fact, why do we even have walls! It’s just wasted space!

Of course, in an efficient world such as this that has no need for walls and with such efficiency in our written and spoken languages I’m not exactly sure what we do with our time. But we’d have plenty of time to do whatever it is we wouldn’t be doing, that’s for certain! ;-)

Hey Strunk and White**, here’s some elements for your style: As hard as it would obviously be to actually submit to, go to your local library or favorite offline or online retailer and pick up a Tom Robbins novel. I’d personally recommend Half Asleep in Frog Pajamas or Fierce Invalids Home from Hot Climates. Then again, Jitterbug Perfume or Skinny Legs and All are both excellent works of literary art, as are each and every one of his other titles.

Oh, and while you’re at it, take a week off and go visit the Louvre. It’s a beautiful, wonderful, and thought provoking place filled with LOTS and LOTS of lines. No, not those kinds of lines. These kind,

–

** Yes, I’m aware of the fact that William Strunk Jr. and E.B. White are both dead.

NOTE: I should also point out that I have always found,

This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell.

… to be a pathetic attempt at saving face. By who’s definition makes the determination as to what words are necessary to *tell* a story, and what words are not? Theirs? Yours? Mine? Which, of course, in and of itself is the entire point,

One mans trash is another womans treasure. This same rule can be applied to *ANYTHING* and *EVERYTHING*. If you don’t like it. Don’t read it. But don’t tell others how to define what is trash and what is treasure…

*They* can make that determination on *their* own.

I’m sitting in a quiet coffee shop on the mist-shrouded Oregon coast, taking a much needed break from family in the wee morning hours to put down some thoughts on the recent O’Reilly Open Source Conference in Portland. I’ll be heading back to Victoria over the next couple of days, nursing my poor, ailing Saturn back to the island and no doubt stopping in Seattle to indulge my daughter’s mania for all things Japanese anime related. She tells me that she’s a dedicated Otaku, regaling me with the plot-lines from half a dozen Japanese comics, many of which she’s now reading (more or less) in the original Japanese (”They always get the translations wrong, Dad!” she says with the conviction that only a fourteen year old teenager can have).

The conference itself was immensely enjoyable, and very eye-opening. I did get a chance to meet with Simon St. Laurent (an old friend and acquaintance that I haven’t seen in nearly a decade) and hung out with M. David Peterson, Kevin Farnham and James Turner, all of O’Reilly, spent some time talking business with Jason Gilmore and Terry O’Donnell, Managing Editors of Apress and DevX respectively, and sat in on some very good presentations (and hopefully gave a good one, though its always hard to tell when you’re on the stage side of the presentation).

The issue of handling legacy binary formats is one that impacts much more than old Word documents, especially for governments who have long-term archiving requirements.

I think governments should simply legislate “After 20 years, the documentation for all formats used in government data should be made public for access on government archival websites and is deemed unencumbered by IP considerations for the purposes of information retrieval of government data” as a matter of public policy. Hand them over or get a fine for obstruction or bad record keeping!

Of course, the regulations would need to say more than that to cope with industry churn and the ravages of time. For example, what if the vendor or product has been onsold and no-one knows where the documentation is now? What if the local sales body is no longer the sales body for that product, or the development organization is defunct. But that need not stop the general case.

Of course, for contemporary and future data, standard open formats are the thing.

I’ve been making some presentations this week on XML Governance. The aspect of governance in particular is the promotion of evidence-based management, with governance involving higher-level management asking lower-level management “What objective evidence do you have that you are taking care of issue X?” The trouble is that it is very difficult to come up with a good list of Xs.

So the approach I am suggesting is that as well as the top-down approach, there also can be a bottom-up approach where you invert the question so that we ask “Given that we have these technical artifacts (e.g. XML), what information can we extract from them and what issues can it be used as evidence for?” In this way, we come up with a list of the issues for which there can be objective evidence, and management can cherry pick the issues which are useful,

One case-history I gave was of a markup operation who installed context-based full-text indexing system. They started using it in an entirely different way to the way we had expected: they went through the list of words that had been marked up as keywords and then looked for every instance of that word where it wasn’t marked up as a keyword. This allowed them much better consistency.

But taking my bottom up suggestion, it also can be used as a governance input: the technologists first report “It is possible to get evidence (a measure or metric) of the words that have not been marked up correctly” which then allows the managers to trace to or add the business requirement “All keywords should be marked up” which in turn leads us to the governance requirement “How to you prove that this business requirement that all keywords should be marked up is being met?”

Things like the Extensibility Manifesto may help formulate useful issues for governance, but it is top-down. And the trouble with top-down is that sometimes tracing from issue to evidence peters out or stalls on the way down: a fine sounding abstract requirement that is unmeasurable. Now this is, of course, the basis of many of my company’s tools and my work on validation and metrics: concentrating on the possibilities that XML allows for evidence gathering, and then trying to progress this upstream to management questions and the governance issues.

Roger Costello has been making a series of “best practice” papers on Schematron over at XML-DEV recently. While these are very important to think about and to gather intelligence about, in a sense bests practices represent a kind of middle-out or context-free approach, which I think can be criticized because abstract statements of principle move away from the worlds of evidence (at the low level) and governance (at the high level). For example, at the moment there is a discussion on whether it is better to embed Schematron schemas in XSD schemas or to have separate documents: a good question. (I have posted to say “well what about XSD types inside Schematron rules too?”)

But my main comment was that perhaps whether a constraint is bundled with the grammar is kept independently should perhaps follow organizational lines: database people can look after static kinds of storage requirements, and analyst people can look after the business rules -checking. It may be that dividing constraints between schema documents should be based on who is looking after them. Now this would correspond to a management requirement “A separation of concerns should be implemented to reduce the intra-organization dependencies on data and applications.” And the relevant governance question would be “How do you prove that you have a separation of concerns in your data and schemas?” And the evidence would be to trace from each constraint in a schema to the driver for that requirement, and showing that a particular schema only has traces to a requirements set by a single organizational entity.

So in summary the bottom-up approach starts with technical artifact (e.g. XML) then finds out what it can evidence (”what can I measure?”) , then extracts potential management requirements which could be analyzed using that evidence, which then suggests possible governance questions. The bottom-up approach never descends in airy-fairy handwaving or impossible to implement abstractions: it seems a practical approach. The result will be partial: if you start with XML as the artifact you will get “XML Governance” issue raised. And the issue of “What can I measure?” leads directly into the worlds of complex validation (e.g. Schematron) and the need to develop good metrics in general.

In the spirit of truth and reconciliation, and to calm the situation down, I thought I’d keep a running list of new accusations on the Web of bribery and corrupt procedures in the document standards world, for the instruction of readers. Of course, follow the links for the context behind the money quotes.

I’ll add notable or novel examples I find for the next week, hopefully none! Sorry no comments on this one; the quotes and links can speak for themselves, And I want to plainly emphasize that there is a big difference in saying that a process is bad (see this interesting Karl Best blog and the interesting post by Marbux referenced below) and saying that some people or group is corrupt.

Here is a little map with the countries mentioned in red.

create your own visited country map

(Added) From NOOOXL.ORG

After the swiss cheese, you can taste the smell of the bitter cacao from Cote d’Ivoire. The Chairman of the Technical Committee in Cote d’Ivoire is Roger Kouadio, from the company Inova Formations. I let you guess from which vendor he is a business partner.

which leads to

Microsoft paying National Bodies to suddenly become P?

(Added) From “Anonymous Coward” on Slashdot

Just yesterday I was sitting in the relevant meeting of SNV/UK14 (http://www.snv.ch/), that decides how Switzerland will vote. The chairman (Hans-Rudolf Thomann) explained the following rules:

- we are here to create standards, not to reject them

- if we reach consensus (>=75%) to vote for Microsoft, we will vote for Microsoft

- if we only reach a majority (>=50%) to vote for Microsoft, we will vote for Microsoft

- if we reach a majority to vote against Microsoft, we will vote for Microsoft

- if we reach consensus to vote against Microsoft, we will abstain

The present spin doctors of Microsoft and ECMA managed to convince Mr. Thomann to reject every serious technical and general concern we had regarding OOMXL by pointing to compatibility reasons. At the end we had a majority _against_ Microsoft but which (giving the unfair rules) results in a Swiss vote _for_ Microsoft. “

(Added) From NoOOXML website from FFII

Brazil also in the way to be hijacked by Microsoft

…

ISO and ABNT are being hijacked by Microsoft and its partners.

(Added) From NoOOXML website from FFII

A generous contributor gave us the list of members of the Technical Committee in Colombia. The vendor is also invading Colombia.

(Added) From NoOOXML website from FFII

Romania votes “yes” for OOXML

Our correspondent writes, “…our politicians here are corrupt as hell, the general view around is that they are “yes-men” when it comes to lining their pockets or lack of knowledge of the technology…