In the first few days of September, just as kids were beginning to head back to school, something rather remarkable happened: Microsoft lost its hegemony. In a vote by the various members of the ISO standards committee, managing only to come up 53% of the votes it needed to fast track the Office Open XML format for consideration as an ISO standard.

Now, Microsoft and its partisans will, most likely spin this as a temporary defeat, pointing to the number of No with comments votes that indicate that it may be possible, with some extensive work, to make OOXML workable in the near term future if some rough edges are smoothed out. In the end of the day Microsoft will have won, and those countries that were too narrow-minded focused on insuring that the standard that emerged was close to being workable with efforts will in the end change their minds, once the spec is cleaned up (and perhaps a few more people were added to the appropriate committees with positions financed by Microsoft money) that surely the OOXML standard will in fact become the de facto one very soon now.

This is a technical blog, and as such, its not always appropriate to express political ideas, especially those ideas that run counter to what will make software vendors bit money. Yet in this case, I think that it is in fact exactly the forum for such an opinion, because in all of the details about whether or not this standard was technically solid or whether in fact Microsoft could make this more solid by some minor compromises, there is an ethical question here that I suspect may have influenced the decisions of participants in this process far more than the technical ones.

Is it right for any company, no matter how noble their intention, to put up as a candidate for a global paradigm a standard that is effectively a reflection of their own products, expressed (perhaps indifferently) in an “open” format and hence supposedly endowed with the magic of being an “open standard”?

Let me say up front that I’m glad that Microsoft produced OOXML (though I think the name is such an absurdly blatant attempt at deliberate obfuscation that it deserves to be censured on that point alone). Microsoft Office (especially Word) has become a de facto standard, in large part because of fairly draconian licensing that served to lock out most proprietary competitors. We still pass around DOC formats and XLS documents in emails (though as scripting has become an increasing concern curiously enough such formats are beginning to disappear in favor of “safer” alternatives). Professional writers still use MS Word (though not as much as they used to), and accountants and business planners still work out their spreadsheets as Excel documents (though even here that’s changing, as the ability to unlock such formats in a useful manner has proved ever more difficult in a world of XML or JSON).

In other words, the de facto Microsoft Office world is now noticeably shifting into a less “facto” one, as these staples of the late 1980s are giving way on the one hand to open alternatives such as Google Office or Open Office at the low end and are giving way to high end dedicated layout packages on the upper end, and facing a constant barrage from XML all up and down the line.

Despite having been one of the pioneers of XML, Microsoft has been dragging its heels at exposing the core of Office as XML for nearly a decade, not because it was technically hard to do (hell, I’ve done it more than once using VBA and walking the Office model without even trying) but because they recognized that once XML entered into the equation, it meant that MS Office became simply another anonymous data provider or consumer in a web of such, with virtually no lock-in capability to speak of that a talented XSLT developer couldn’t exploit over a weekend.

The acceptance of the Open Document Format by ISO was a terribly rude awakening. The only significant producer of ODF at the time, Open Office (OOo), was underpowered in comparison to MS Office, but more importantly, OOo was free. You cannot compete in a market where the competitor drops costs to nothing - unless that competitor is frankly not out to compete in the capitalist game altogether. They could console themselves with the idea that OOo was inferior, that it was seen as being geeky, and that it didn’t have the installed user base to compete at the enterprise level.

Yet with the acceptance by ISO of ODF, the power of open standards began to emerge. Google turned around and made an office suite that uses ODF as a primary delivery and input format, is free for the average user (and far less than office in most enterprise settings) and could be accessed anywhere from the web. ODF is beginning to appear on the Macintosh, it has become the primary word processing format on Linux, and every other Microsoft competitor is now recognizing that ODF represents a point of commonality that gives them a huge leg forward in an often contentious marketplace.

Zipping up XML, what a concept - combine two of the most widely used formats on the planet into one package, and you can transport documents or even sets of related documents, complete with scripting if appropriate. Note that it doesn’t mean that the internals of the application need to use ODF within the app - the application developer has a responsibility in creating the most efficient internal representation possible, after all - only that the access points into the application need to be ODF aware. ODF’s value thus becomes one of serialization, and such a common serialization similarly implies that such a standard, even if not necessarily used to save local files, becomes far more common in messaging systems just at the time when the realization of the web as more than just a static delivery platform enters common consciousness.

OOXML is an example of the dangers of miscalculation, and of arrogance. Does the world need another office serialization format? No - it has one in ODF already. Moreover, ODF emerged out of a presence through a standards body that has gained a reputation for both being relatively democratic and having a very clear cut process for developing standards (OASIS-org) that are mirrored upon those used to produce ISO specifications. ODF was a comparatively easy sell because it followed the rules, satisfied the available niches, and reflected a relatively unbiased development process.

I want to focus on this a bit. The argument can be made (and has been, even in this venue) that ODF was no less biased than OOXML in terms of its vendor support … clearly Sun had a stake in getting ODF approved as a technology. However, the specification was NOT in fact the native format used by Open Office initially; indeed, development of Open Office 2.0 actually was changed to shift to the new ODF format from the older SXW (et al) formats because of this realization that the product would be far better off adopting a community developed standard, and the community consisted of both people and organizations far removed from Sun that nonetheless could see the benefits of the development of such a standard.

Develop a standard that people in general are comfortable supporting, then building your tools to take advantage of the new standard, lies at the heart of the open standards movement. It can be argued that in some respects ODF may prove to be harmful to Open Office in the long term because it means that other companies or open source projects can come along (a la Google) and build ODF based products that are better … which in turn provides an incentive to improve the OOo interfaces in a virtuous cycle.

Compare that with Microsoft’s approach with OOXML. The OOXML standard was pushed through a standards body known to be friendly to Microsoft in the past (ECMA), on a Fast Track that allowed for very limited review of the standard, despite the proposed standard at the time being still somewhat incomplete, containing many references to established proprietary technology, and weighing in at several thousand pages. Note that OOXML supporters have pointed out that if you take SVG, XForms, and other W3C and related specifications that ODF uses collectively, that they come to much the same length. However, it should be noted that ALL of these related technologies went through their own formal standardization processes independently of a single over-arching one, while in essence the OOXML specification perforce also implicitly included analog formats that were all meant to be approved as an entire package.

ODF, like all OASIS and W3C standards, also requires that there be at least two “test” implementations for a given standard before it can be moved into the fully approved category. This acts as a brake to insure that no one player in the market automatically has a monopoly in that standard space. Obviously, this does not hold true with OOXML, and indeed it raises an objection that I feel lies at the heart of this debate. Suppose that a company should decide to do the unthinkable, and create a word processing program that is superior to Microsoft Office and utilizes the OOXML specification … will Microsoft in fact permit this to happen without trying to raise significant legal barriers? If a Chinese entrepeneur should create a RED MOND Office suite that differs from Microsoft Office only in that it doesn’t save to the binary DOC/XLS/PPT zoo but instead chooses to save to OOXML, can they do so with impunity? With ODF, the answer is an obvious yes. With Microsoft …

Microsoft had to win this, and they made sure that as many organizations as possible that were sympathetic to the Microsoft vision (or the Microsoft billions) were “upgraded” to the requisite voting levels in ISO. If nothing else, ISO has certainly received a significant infusion of new money from membership fees, but even here that effort fell short. At the end of the day, political considerations come into play at this level, and one of the calculations of such politics is whether in the end the approval of a standard is in the best interest of the people who are part of that organization - in this case the member nations of ISO.

Fast track processes are sometimes necessary - there are times when an emergency threatens, and you have to get a standard out as a way for people to agree upon how to accomplish critical tasks to solve that emergency. In some cases, such as that of ODF, fast track can also occur because the principles used in the initial formulation of the standard mirror that of the considering body, and as such it becomes easy to ascertain whether such a standard proposal is both viable and desirable.

However, fast track processes can also bypass scrutiny that may prove when such standards (or legislation in a legal body) are not in fact up to a sufficient and necessary level for the public good (defined by the best interest of the standards body delegates). That’s why any attempt to fast track should be looked upon with a heightened degree of skepticism as well. The only emergency here is that Microsoft is losing its dominance upon a market that long ago would have succumbed to commoditization without its presence. This raises the question about whether in fact the OOXML standard was in fact both solid enough to withstand close scrutiny and fair enough that others could implement it without fear of legal recrimination.

At the end of the day, Microsoft was unable to meet these twin burdens of proof. What this means in the short term is that while it may in fact be thrown back into a longer (and more protracted) review process, in the longer term, the distrust that Microsoft has engendered worldwide by its practices is now coming back to haunt it. From both reviewing the spec and watching the commentary, while they may be able to salvage pieces, the OOXML specification, and Microsoft’s ability to push its product vision into the world’s standards organization, is badly wounded if not dead.

It is very likely that at this stage there will be a solid push on the part of ISO to get Microsoft to play ball, perhaps by agreeing to work to merge OOXML with a future version of ODF (this proposal was in fact made by the Irish delegation recently). I’d be delighted to see that, myself, as I think that there are areas where ODF is deficient, and when they are not trying to push bad standards Microsoft is actually a good partner to work with in the establishment of good standards, if only because they can play the role of the devil’s advocate remarkably well.

This is a crossroads moment for Microsoft, and the direction that they choose at this point will likely be an indication upon the management by people such as Ray Ozzie, Bill Gates heir apparent, on the direction of Microsoft in the future. At one time Microsoft may have had the clout to push this through, but the market has changed, the technology has changed, and while they will continue to be influential, Microsoft can no longer expect to create “standards” simply because it is Microsoft.

It’s an interesting lesson to take to school.

Kurt Cagle is the webmaster of the XML News Network and blogs at Metaphorical Web.org.