Why would Open Source developers want to support Open XML becoming an ISO standard? Isn’t it from Microsoft, the great Satan? Isn’t is some kind of trick or trap to stop nice Open Standard ODF? Are we going to let this chance to overthrow the monopoly escape?
Now there have long been two different camps in the Open Source movement: those who think that it is important to have independent APIs and those who think that it is important to have Open Source clones of the most important proprietary APIs. This latter group is of course associated with Novel and the Mono effort is a good example: on their history, I don’t think they have much problem with Open XML going through ISO (Gnumeric’s Miguel de Icaza fits into this latter camp.) So this blog is more addressed at the first camp.
First, I would like to set the scene. I think the reasons for supporting Open XML at ISO become a lot clearer if we take a fairly hard-headed view of what is possible. Which is perhaps a nicer way of saying that I think some of the anti-Open XML case has been built on naively faulty assumptions about the miraculous power of ODF to disrupt Microsoft.
- No office suite or utility can afford to ignore any important format for long. So in a year’s time, every major office suite and utility (whether open source or not) will support ODF and Open XML, whether or not Open XML becomes an ISO standard.
- No vendor will adopt, as their default save file format, a format which does not support their particular feature set. So the only way that MS would make ODF its default file format would be if (when) it is improved to support Office’s feature set. (Support for Office’s feature set was not a design goal or activity of ODF’s development. See the second item in the minutes of the first ODF meeting for example. Or see ODF’s Gary Edwards "http://about.diigo.com/about/show?url=http%3A%2F%2Fwww.consortiuminfo.org%2Fstandardsblog%2Farticle.php%3Fstory%3D20070629070544217">comment that “There is no possible way anyone can claim that today’s OASIS ODF TC would welcome Microsoft and make accommodating changes to the specification!”)
- The poor state of ODF implementation by Open Source applications means that a too-fast adoption of ODF will backfire for Open Source developers. So paradoxically, supporting mandatory ODF too soon would be the kiss of death for open source ODF: bureaucrats will test applications and, finding them lacking, be forced to buy into a new generation of closed source tools (MS Office, IBM Lotus, Word Perfect, Sun Star Office.). If governments mandate ODF, it won’t exclude MS Office, in particular, and MS thinks their new GUI and features are competitive against other implementations, of course.
- No matter what format you use, the only way to get 100% page fidelity (apart from some good-as-read-only format like PDF) is to save in the native save format and re-open the same file using exactly the same application on exactly the same operating system configured exactly the same. ODF won’t give instant interoperability in the sense of full page fidelity; I’d expect the same would be true of Open XML on different platforms too.
Putting it all together, it means that there was never a chance that Microsoft Office would or could adopt ISO ODF 1.0 as its native and default format. So the real choice that faces us is whether we want Office to generate files in a format that MS controls with very few external checks (with all respect to Ecma) or to generate files in a format that MS instigated but which has the extra checks and balances that come from being an ISO standard. ISO standardization is not an Aladdin’s cave of democratic rights, and it is not a Pandora’s box for Microsoft, but it is way better than nothing. Because that is what the anti-Open XML people would achieve: no controls on Microsoft. Under the guise of supporting ODF.
If you, like me, are in the position where you don’t use MS products in your normal work lives, then you may not feel any urgency to support Open XML, but I think we at least should not oppose it. It is a good step forward.
We often read that Microsoft is doing this as some kind of sinister ODF spoiler. However, Open XML is a path they have largely been forced to take (though obviously they will try to make the path as beneficial to them as possible) in order to fend off continuing anti-trust problems in Europe: Microsoft went down this path after a very strong hint from the European Union: in the same report that recommends that ODF be submitted to ISO, the EU’s ‘Telematics between Administrations Committee’ recommended that Microsoft should consider the merits of submitting XML formats to an international standards body of their choice as well as improving the documentation, IPR issues and going all the way with XML.
I certainly support governments mandating that public documents should use standard formats: HTML and PDF being the primary two, and ODF after that, but also Open XML as a second source. However, having an ISO Open XML does not prevent any government from preferring ISO ODF. ISO standards are what are called “voluntary”, which means that they are not like laws where you have to adopt them. In my view, the drivers for ODF will continue unabated even after/if Open XML becomes a standard.
So, in my jaded view, ODF will not make Office go away, ISO ODF will not make Ecma Open XML go away, and ISO Open XML will not make ISO ODF go away. So I see no downside in Open XML becoming an ISO standard: it ropes Microsoft into a more open development process, it forces them to document their formats to a degree they have not been accustomed to (indeed, the most satisfactory aspect of the process at ISO has been the amount of attention and review that Open XML has been given), and it gives us in the standards movement the thing that we have been calling for for decades (see my blog last week that compared what Slashdotters were calling for in 2004 with the path that MS has taken).
In the 80s, there was a hilarious wrestler called George the Animal Steele. He was incredibly hairy, especially on his back, which was supposed to be emblematic of his sub-human state. His great flaw as a wrestler was that just as he was winning he would be distracted by the turnbuckle, often trying to eat its stuffing while his opponent recovered. I guess this is how I feel about the attempts to stymie Open XML at ISO: just as we have victory in our hands, with MS prepared to go XML and standards, along comes this distraction, ODF, which is great in its place but dumb, unworkable and counter-productive as a Microsoft buster.
I’d like to make two other minor points. The first is that ODF, HTML and Open XML all do markup “wrong”: the more that markup is done in semantic terms, the more that it has value and can be repurposed. While ODF, HTML and Open XML have various mechanisms to allow the addition of nice XML fragments (XForms, Custom XML Controls, etc.) their basic markup is highly presentation-oriented and low value. So we shouldn’t get confused that somehow one is particularly more virtuous or high-level than the others.
The second point is that what the XML-in-ZIP formats do is allow much easier pipelines, where the office suite or application is just one stage in the process. So when thinking about these formats, the primary user of ISO Open XML is not a rival suite developer, who pretty much has to be underwritten by a large corporation, but it is the small developer and integrator. What Open XML represents is a way for small Open Source (and other) developers to get more reach into the systems that MS for years tried very hard to keep out-of-bounds.
Sir I like your. I want to learn more XML and want to use it in PHP MYSQL application. XML is very important for the open source community. It will help developing more application with different platform.
"So the only way that MS would make ODF its default file format would be if (when) it is improved to support Office's feature set. (Support for Office's feature set was not a design goal or activity of ODF's development. See the second item in the minutes of the first ODF meeting for example. Or see ODF's Gary Edwards"
You pick something from the "first" ODF meeting -- how many years ago was that? MS later joins OASIS but then doesn't help them at all...great on MS's part. So, OOXML is not a standard. A standard is not "better than nothing", it is judged on merits like multiple vendor participation, whether it's freely implementable, etc. One really has to question deeply the things that OOXML "references" as well, mostly Microsoft proprietary technology. MS's recent GNU/Linux patent saber rattling has not done you any favors in that department.
Jaykumar: Yes.
Jamie: See my blog Is our idea of open standards good enough for a general critique of the boutique standards bodies (Ecma, OASIS, W3C). MS got rolled at W3C on VML, and they learned from that not to expect friendly co-operation on terminal formats. Indeed, that is one reason why it is regrettable that terminal formats (rather than "enabling technologies") are such a headache for standards bodies as soon as there is any idea of mandating particular standards. The idea that joining one of the boutique standards bodies will necessarily be a win/win situation for all or even the public is bogus.
ODF did not start with a goal of supporting Office, it was not developed with that in mind, and some members of the OASIS TC have demonstrated their antipathy towards MS, as in the Gary Edwards quote.
If you don't like MS' sabre rattling on patents, why aren't you welcoming Open XML, given that they have their covenant not to sue and are submitting to the Ecma and ISO process to make sure that the IPR situation is clear and open?
Why is "references" in quotation marks?
Mr. Jelliffe,
Where is the MS covenant not to sue that you refer to? I have not seen this, and I wish to read it. Sadly, my research in uncovering this item has come to naught. Thanks.
William: The Wikipedia entry on Open XML has lots of good background material from all angles.
There are two sets of licensing for Open XML: the Covenant not to Sue and the Open Specification Promise.
Re: "ODF did not start with a goal of supporting Office, it was not developed with that in mind, and some members of the OASIS TC have demonstrated their antipathy towards MS, as in the Gary Edwards quote."
Gary's opinions on this are only that; I think most of the ODF TC would disagree him.
I personally would welcome MS participation in some kind of harmonization or interoperability effort. I've said as much, even as I rejected the basis on which Gary was arguing a particular list proposal for ODF that he claimed was all about interoperability with MS file formats. My point at the end of that note was not at all anti-MS: it was to say that it's silly for both TCs to be operating as if they were in a vacuum, and that there ultimately needs to be some formal cooperation going forward.
I've also, BTW, talked to people at MS and via the ECMA TC 45 comment list about some details I care about (bibliographies and citations) in the spirit of constructive comments, and been disappointed to see little if any substantive changes result.
I don't think it's productive in the long term to have two ISO standard XML office document formats, any more than it would be to have two fundamentally different models for web documents ((X)HTML and something else).
Rick. The OSS project's I've been involved in (Ant, Axis) like best things they can adjust as they go along. Instead of the waterfall of OASIS process, home-rolled, de-facto standards are nice because they are agile and adaptive.
if we do have to use standardised stuff, we like standards that are broadly adopted (justifies effort) and easy to implement. That means small specs, reuse of existing things (like SVG) and a really good, public, test suite you can verify your implementation.
OOXML has none of these. It lacks the broad user base (remember, we arent talking about the binary formats), it has lots of wierd stuff that nobody has implemented (VML), and there's no decent test suite.
This is why ODF is being used in places like apache forrest, as a format for content. Anyone can enter the food chain, its easier to implement support. And, if there is a problem, there's a process for fixing it that works with OSS.
OOXML is just a support problem from a single under-deployed product. More interesting would be a formal documentation of the binary formats.
Steve: But there is a difference between standards you want to use, and standards you want to allow/encourage/not-block. Having Open XML as an ISO standard does not force you to prematurely adopt a standard, nor does it force anyone away from using ODF, or Apache somehow into the bosom of Microsoft.
The thing that is notable about the Open XML debate is the number of people who wouldn't use the ISO Open XML standard themselves in a pink fit but feel perfectly happy telling the people who do need it (because they have to fit into that world) that *they* cannot have it either.
Bruce: Given my assumption that MS cannot therefore will not adopt ODF as the default save format (1.0 at least; presumably ODF 1.4 will have a much better chance), the question then becomes, is it more likely that ISO standardization will make Microsoft more receptive to user requirements or less? I think it cannot make them less responsive, because it slots them into a maintenance system where there is an extra pressure and which allows a more concerted forum to achieve change.
When we look at the kinds of statements that members of the ODF TCs are putting out against Microsoft and Open XML, I don't think any dispassionate observer (let alone me!) would think they would find much joy or real welcome at OASIS.
You write that ODF, HTML and Open XML all do markup "wrong". Is this something that can really be attributed to the file formats, or isn't it much rather a flaw with the applications that we use the most?
As long as you don't take away people's Italic button in Word, and replace it by a Term Definition or Latin Phrase button, the temptation to simply mark up for presentation rather than semantics will be just too big to ignore...
Apart from things that are relatively simple to infer from formatting, like headings and bulleted lists, is semantic markup in Office files really something that you think can happen unless "word processors" turn into "meaning processors"?
[...] will make Microsoft more receptive to user requirements or less? I think it cannot make them less responsive, because it slots them into a maintenance system where there is an extra pressure and which allows a more concerted forum to achieve change.
I think what it boils down to is that many people want community, not vendor, driven standards. They want independent verifiability, as Steve said. They want the pressure to be on Microsoft to work with others to create solid, complete community-based standards, not for others to be pressured to work with Microsoft designed and driven "fast track" standards. I think, in short, that many people want to see consensus-driven development, not Microsoft in the driver's seat.
All of this is very likely to happen if OpenXML standardization fails, and I think that is why you see so many people against the current approach.
Marcus: Indeed, the file formats reflect the class of application.
I'm not completely against direct formatting in casual documents.
But at the simplest level, one feature that would be great would be if file formats (and templates) can tell the WP applications to disable direct formating, so that only styles can be used. That would allow document lifecycles where direct formatting cannot creep in 'by accident'.
At the next level, word processors should support Schematron-style schemas, so that information items marked up using styles can be constrained and required.
Kevin: I think you are right. But it is woolly thinking that will result in the opposite: MS being completely outside any standards process with ODF at the margins.
The primary pressure to use MS formats comes from the market share and product success, independent of ISO fast-track processes. That pressure has been overwhelming for most users in the past and will continue to be so whether Open XML goes through ISO or not. ODF may be able to overcome it in a couple of markets (public and regulatory documents) but not generally, in its current state (I never rule out ODF 1.4). That is why I think the ODF versys Open XML dispute is a sideshow that distracts from the real issue of the best practical strategy to rope MS happily and productively into the world of document standards.
Kevin: Oops, that wasn't well expressed. I should have written
I think you are right (that that is what many people want.) But...
I'm convinced this issue has nothing to do with standards or standards organizations and processes. Those threads are well-documented and understood. It is about this:
http://en.wikipedia.org/wiki/Other
I don't think there is a big change in the wind here. 40 years of markup left a long history of failed consensus for document formats for WYSIWYG systems. As long as the ecosystems are healthy even if competitive, there won't be.
Convergence for its own sake is not a compelling argument. XML succeeded wildly because there was a real need for it. For document interchange, gencode systems such as HTML do the job adequately. For everything else, the exchange is on such a scale as variant solutions are tolerable and preferred to sacrificing features for the sake of acquiring a bigger scale, scope or reach.
The rest is the politics of other.
But it is woolly thinking that will result in the opposite: MS being completely outside any standards process with ODF at the margins.
Really? That's a matter of opinion, I guess. Microsoft primarily needs to keep existing customers, so it would seem very risky to go against its customers' wishes and completely pull out of standardization efforts. A much safer route, I think, would be to provide excellent ODF support while promoting a superior user experience to OO.org.
ODF may be able to overcome it in a couple of markets (public and regulatory documents) but not generally, in its current state (I never rule out ODF 1.4).
Of course, but it would improve a lot quicker if it had companies like MS putting more of their energy into it. Perhaps they'd even hire you to help out on the effort. :-)
Kevin: The trouble is not MS "pulling out" but MS being *pushed out*, which is what we are seeing. The result will be no extra victory for ODF, no loss of sales for Office, the losers will be the people who would be advantaged by having MS at the table: i.e. users and integrators.
When people say "Why don't you use standards?" they will say "You can export to ODF, and we tried to make our formats a standard but we were rejected"...leaving them scott free to do anything they like. The anti-Open XML people are guaranteeing that the political relationships between MS, the minority vendors, and the public remains the unchanged.
Jeff,
The only reason for wanting ISO approval of Microsoft's XML format is to ensure Microsoft's future access to government contracts and with that, government suppliers. With ODF in place, ISO compliance becomes a check-off item that Microsoft has to meet without supporting competitors - hence the Open XML push. You are right, Open XML does not prevent ODF from being used - that's what the lobbyists are for, but they need the ISO approval to work effectively.
Connecting openness to the standards process is misleading as use of XML and publication of the description are orthogonal to the the ISO approval process.
Customers always want to know the file formats. They may not act on it, but they do want to avoid lock-in. Right now many users want to access the billions of documents in the proprietary Microsoft formats without using Microsoft products to do so, but Microsoft continues to ignore them.
Suppose for a moment that Open XML is approved - how will that constrain Microsoft from adding elements to it's productivity products that are not described in the standard? How will it prevent deprecating major elements in favor of new elements that only Microsoft products can handle? How does it control Microsoft in any way they are not already controlled?
The down-side to ISO approval is it gives Microsoft the appearance of participating in the open, while not roping or forcing them into anything.
Only ODF can force Microsoft to be open, and that has already started change. ISO's Open XML approval will stop that change.
PS George "the Animal" Steele did win - every match. He got paid and maybe even more than his on-stage competitors. Not bad for a teacher and coach with a Master's degree. A more apt comparison is between Open XML and the Trojan Horse.
ODF may be able to overcome it in a couple of markets (public and regulatory documents) but not generally, in its current state (I never rule out ODF 1.4).
But when will ODF 1.4 be approved by ISO? 2017? Never?
Some major problems found in OOXML can be found in ODF as well and there are issues specific to ODF. If ISO takes its business serious it won't approve a new version of ODF without the same amount of scrutiny as OOXML currently receives. This could mean substantial changes to the standard or straight rejection - especially so if OOXML gets rejected by ISO. This is where the anti-OOXML strategy might start to backfire.
Support for ODF 1.0 is lacking and ISO ODF 1.0 is lacking as well. It's a rather poor standard that seriously needs improvement. OASIS is working on the problems and OpenOffice.org and KOffice (at least) are currently working on implementing ODF 1.1 which is not approved by ISO and will continue to implement newer versions of ODF. It does not look like there will be a "strict ISO OpenDocument 1.0" mode in those applications.
In some markets public and regulatory documents will have to be in ISO ODF 1.0 a standard with very little (and therefore expensive) tool support. All the others will use some non ISO file format (be it OASIS OpenDocment or Ecma OOXML).
From a user perspective the choices could easily be ODF 1.x (with x>0 = not approved by ISO) and OOXML 1.0 (not approved by ISO). In this case there might be no reason for not choosing OOXML even more so as OASIS (as an organisation) offers no advantages over Ecma (as an organisation).
Kevin: The trouble is not MS "pulling out" but MS being *pushed out*, which is what we are seeing. The result will be no extra victory for ODF, no loss of sales for Office, the losers will be the people who would be advantaged by having MS at the table: i.e. users and integrators.
When people say "Why don't you use standards?" they will say "You can export to ODF, and we tried to make our formats a standard but we were rejected"...leaving them scott free to do anything they like. The anti-Open XML people are guaranteeing that the political relationships between MS, the minority vendors, and the public remains the unchanged.
Well, this is disappointing. Now we're back to attempts to de-legitimize complaints about the process and create 'bad guys' by simply lumping everyone's positions together as "anti-Open XML". How about if we call MS anti-open-verifiability? anti-vendor-neutrality? Would you prefer to add those terms into the discussion as well? It will only hurt the discussion, and in the end the more we try to de-legitimize the others' opinion, the less time we take to try and understand it.
Anyway, for me it boils down to this - there are plenty of standards out there that do have open reference implementations, that are vendor-neutral, and they generally work quite well. As an implementer, I'd use one of these in a heartbeat over alternatives, because I've used both proprietary APIs/formats and open formats, and overall the open formats are just better designed, reviewed, and tested. MS has an open format, but still a mostly proprietary, vendor-controlled process. So in my mind, asking MS to do the same things so many other organizations and standards do right now is not so unreasonable. Not only that, it may even benefit MS considerably. In fact, the irony is that the "anti-OpenXML" people may just be pressuring MS to do what is, in actuality, in Microsoft's own best interests. Nah, that's probably a crazy idea. :-)
Dave S: Yes, I had forgotten that! Thanks for the encouragement.
My favorite memory is when they took him to some professor who attached what was clearly a kitchen colendar to his head and turned on some bogus Frankenstein power supply, leaving him with ability to speak lucidly and fluently. Alas, the effect wore off. (Indeed, I once walked past Kamala the Ugandan Headhunter, who was without his Handler at the time scarily, and I heard him speak in good English too: perhaps they were still experimenting with the colendar!)
Of course, central the the George persona is the idea that primitive people have thick bones and therefore are impervious to attacks to the head. This was widely believed of Africans by Whites early until early last century: IIRC there was a world championship boxing match in Australia (1920s or 1930s?) with a local white against a black American, and the local guy didn't even try to make hits to the head because he believed it would do no good! He lost. The myth found its way into the tropes of professional wrestling, but usually in a thankfully deracinated (that is not the right word) form.
Kevin: No, I am only lumping together the anti-Open XML people and calling them "the anti-Open XML people". And I am certainly not saying they are the bad guys: where do I say anything like that? Good, naive, bad, nutty, cynical, clear-thinking, everything in between: the normal mix. And I don't want to denigrate ODF or the impact that it is having. It is just that I don't believe ODF's impact (or capabilities, for ODF 1.0) are anywhere near enough to support the claims and hopes that crypto-marketing people have attached to it, for busting Microsoft's monopoly.
"How about if we call MS anti-open-verifiability? anti-vendor-neutrality?" I am sure sometimes they have been, sometimes are now, and sometimes will be in the future. But there is, as far as I know, no verifiable vendor-neutral standards body in the terms that I am suggesting in other blog posts; it needs to become a kind of formally mediated and attested procedure, where standards can be rated on the numbers and weights of their supporters and detractors: if governments must mandate single standards (which I am generally against on the grounds that plurality is good for the ecosystem), it should only be for standards where all significant stakeholders accede (and reviewed regularly.)
Having and Eating Cake
What I see in all of this is that even those advocating a pro-OOXML stance see it as essentially providing some form of leverage to force Microsoft to come to the table at some future point to agree upon a universal open format. I applaud Microsoft for bringing an XML version of their format out; despite having nearly a decade in which they could have done so they chose not for some time, largely due to the potential loss of vendor lock-in that such a format might bring. That they are doing so now, at a time when another format is clearly being adopted by some of the largest potential customers in the world, does not speak to altruism but to the realization that they would stand to lose far more if such a format gained ascendancy than they would gain by maintaining their lock on the format.
The question that this raises for me is one I've raised previously - simply because it is in the best interest of Microsoft to move to an XML format is it necessarily in the best interest of the rest of the world to have the Microsoft format be recognized as a canonical standard? I've seen no indication that Microsoft is prepared to engage with other standards bodies beyond its immediate pet ones (ECMA, for instance), nor to deal with any other vendors in a standards body that they do not otherwise control. This means that for Microsoft, an ISO imprimature is primarily a marketing tool rather than an overt move towards "playing well with others" - with it in place, they can continue to ignore the ODF standard, can effectively steer clear of any homogenization efforts, and, what's worse, have an effective leg up in asserting their format over ODF in what I see as ultimately a zero sum game.
Here's where I part company with many people, especially those in the Microsoft camp. I do not agree that the most feature rich technologies are necessarily the best; rather I see a standard as a floor, one that should determine the minimal necessary conformance and provide a sufficiently robust extension mechanism that lets implementers build iterative technology above that floor that can then be implemented natively in the next version. This is a technique that has worked reasonably well within both the W3C and within OASIS, and has had the effect of insuring that all implementations that subscribe to these standards have the chance of pushing the envelope in a consistent fashion.
Microsoft frequently claims it has been pushed out of various standards bodies, but my recollections covering those bodies over the years has usually been quite different. Microsoft left the W3C largely in reaction to that organization refusing to accept RAND licensing (largely due to the anger by many people involved in the standards process discovering the manipulations going on only at the last minute and responding vehemently against the move). The WS-* efforts similarly have gained the adherence of Microsoft "vassal" organizations and few others.
In other words, the past history of the organization has generally been that they have been extremely hostile to compromise, and I have seen few indications to believe that this will end any differently with the adoption of OOXML an an ISO standard.
I am not subscribing to the Microsoft is evil camp here; they are clearly motivated by the business concerns of insuring that they neither lose existing customers because of governmental mandates nor lose the ability to gain new ones because there are competitive products and standards. However, I think it is reasonable to ask why it is that Microsoft should in fact be accorded special status regarding a proprietary technology that they continue to control.
Kurt: Yes, it certainly does remain to be seen how maintenance and evolution will pan out: MS has historically been very keen to keep "control of the API".
Indeed, that is something almost entrenched in Open XML's extension mechanism: it allows MS (or anyone) to add new attributes and so on without putting them through the standards process, but the format just says whether an application that is unaware of these new attributes can ignore them or should complain.
As far as it being unique, there are at least two other examples of large companies putting up their family jewels at ISO. The first is Sun, with ISO Java: in the end they decided that giving up control was not in their best interest and pulled out before standardization; the JCP was formed instead. The second is Adobe, with ISO PDF/n, which is a series of profiles of PDF suitable for various vertical markets; they allowed the minimal standards, then freaked out about PDF in Office despite it being a standard, and now they seem keener on having the whole thing standardized. A certain amount of skittishness should be expected.
And yes it is really important to keep standards free. But it is not necessary to think that Open XML is wonderful in order to think we would be better off with ISO Open XML through than with MS Open XML.
No, I am only lumping together the anti-Open XML people and calling them "the anti-Open XML people". And I am certainly not saying they are the bad guys: where do I say anything like that?
It's hard to know who the "anti-OpenXML" people are, though, when they're not defined. Not being defined, each reader ends up guessing your meaning from the context. That's why I suggest it's not good to use terms like this, regardless of what you want it to mean. As for 'bad guy', sorry, I meant to say that you held them responsible for 'pushing out' MS. (But to be sure, pushing people around isn't an image you'd cast on a group you see as helpful or constructive!)
But there is, as far as I know, no verifiable vendor-neutral standards body in the terms that I am suggesting in other blog posts; it needs to become a kind of formally mediated and attested procedure, where standards can be rated on the numbers and weights of their supporters and detractors: if governments must mandate single standards [...]
I see your point, but I have seen many standards 'done right' and I don't see how the standards body does not have the power today to ask for those things from Microsoft. As an implementer, a missing reference implementation is a pretty glaring and critical omission. Of course, I have a good idea of why one isn't there, and why there probably isn't one forthcoming, but it's only going to make life that much more difficult for implementers. Implementers, in fact, who weren't given the chance to suggest better ways to implement parts that might significantly reduce their workload. And who may have no choice but to implement support to meet the demands of users asking for standards-compliance. Do you really see no fundamental issues with that approach?
Personally, I think there is a unique opportunity here to get MS to start working with the broader software community in a way that treats other players as equals. OpenXML does not reflect that mentality, in my mind, and we have an opportunity to tell MS that 'that's not how we do business around these parts'. If MS decides to opt out of standardization because it does not want to be 'just another vendor at the table', that will be their decision and I'm sure there will be consequences for them and perhaps for others as well. What would be far worse, though, in my opinion, is if people didn't stand their ground on principles such as equality, transparency and independent verifiability.
Kevin: I think reference implementations are certainly a good idea, especially open source reference implementation. Of course, Office 2007 does act as a de facto reference implementation, and anyone seriously implementing a competitive product would be using all sources of knowledge.
However, ISO rules do not allow reference implementations, and SC34 has been quite strict on this. This is part of their methodology that constraints should only be specified once. When there are multiple sources for constraints in a standard, it increases the difficulty of editing and runs the risk that items can be out of sync. (Standards already have a problem that often constraints are split up so that you need to look in several places to get the full picture, but that is the nature of text.)
Recently I see that Martin Bryan, in editing the ISO DSDL spec called Character Repertoire Description Language (CRDL) (which standardizes a technology that W3C's then head of Internationalization Martin Duerst created, but which didn't fit into W3C's priorities), has added a sample implementation as a non-normative annex. This is a good sign, I think, as long as readers are aware that the non-normative reference implementation acts to help disambiguate the normative text and not to compete with it.
For my standard, ISO Schematron, it also largely just documents a pre-existing application: I call the open source Schematron implementation (a new beta version should be available next week at schematron.com) the "unofficial reference implementation". When creating a standard from an existing technology with a single dominant implementation, you are largely just documenting it; after the standard exists, it becomes a market power issue whether discrepencies between the standard and the dominant implementation get reflected in the standard or the implementation. This will be the interesting question with Open XML, and why it is important that the standard and the current dominant implementation start off in synch.
However, I don't want to go too far along the route of talking about documents standards solely in terms of helping or thwarting Microsoft's plans for continued monopolization. ISO is not an anti-trust court, for better or worse, and the focus should be on whether there are users who need it.
Standards at ISO are intended to address market requirements not to dictate market requirements: in other words, a standard gets created because some credible group says "we need this" and it is not for the ISO process to say "We don't serve your kind here." Standards at ISO/IEC JTC1 SC34 are concerned with "Document Description and Processing Languages" and the focus of the SGML-derived material is on how to promote "rigorous markup", which means documents that use text-based markup (e.g. XML) that has clear structures that can be validated against a schema. So the filters through which technologies go through is loosely "Are there people who say they have a market requirement" and then "Does this use rigorous markup?" [and then "Is this documented properly" and "Have the IP Rights been looked after?" (though in the last case, there is case law that if a participating company does not disclose, they lose their rights to claim on the IP, so it is probably more important to check external IPR)]
We are used to the word "anti-trust" being used for Microsoft's monopoly or near monopoly positions, but it is also used in standards for when a cartel blocks a competitor's technology from getting standardized. Standards organizations like ISO and ANSI are very clear that the standards they make are "voluntary standards" (where you have to opt-in to the standard) not laws or treaties per se (where they apply automatically.) The parrots who talk of "there should only be one standard for anything" often leave out this over-riding legal context, that standards cannot be used for cartelization. Standards are agreements, and if two camps cannot agree at a particular time, it is better to have two standards than one. (Which one gets chosen? the cartel's? the first to register?)
in other words, a standard gets created because some credible group says "we need this" and it is not for the ISO process to say "We don't serve your kind here."
No, but the standards body would be remiss if it didn't not consider what the likely overall impact on the market would be. You talk about the target market who 'needs' ISO standardization for OpenXML. Where are they? I'd like to hear from them, hear how large they are, why they need ISO standardization, etc. They seem nearly totally silent in all of this, at least on the public forum. I think I have yet to hear anyone say "Hey, I really need this" in all the debates I've read.
On the other hand, due to the standard's wide reach (how many office documents exist in the world?), we all must really consider the standard's impact as if it were to be adopted by many organizations around the globe. The potential negative effects of this standard seem emmense. It could literally be a major setback to truly open software and services, and force a lot of people back to buying proprietary software just to read and write (standardized!) documents. (Not to mention I think a lot of people will learn from this that ISO standardization doesn't ensure solid interoperability.) I clearly see the benefit for MS here, but I don't quite see how this is a huge boon for the community. So what is this huge benefit I'm not seeing?
Standards are agreements, and if two camps cannot agree at a particular time, it is better to have two standards than one. (Which one gets chosen? the cartel's? the first to register?)
For me and most critics I've read, it's not about OpenXML vs. ODF, it's more about open process vs. closed process. ODF is frequently used as a 'counter-example' to issues with OpenXML but I think that actually does muddy the argument. We should talk about OpenXML's issues in isolation to clarify what the real issues are. ODF has its own issues (and benefits), but those should be the topic of another discussion. (One, I think, that most certainly should be had.)
As for cartels, again, I don't like the labels. For the record, I don't care to call MS a monopoly either, which you may or may not have noticed. I think the focus should be on what's best for the community, and who/what these companies are matters only in so much as what influence they have over the community, and more practically, what influence they exert over the community. So let's talk about their actions, but let's not label them using terms that suggest things about them that may or may not be true.