Patrick Durusau, editor of ODF, asked me to restate my thoughts on what “contradiction” should mean at ISO. I had mentioned my views in an SC34 meeting last year. This topic is, of course, of interest right at the moment, because the Ecma proposal for OOXML is at the stage in its acceptance process where the process says it should be checked to make sure it doesn’t contradict other standards.
I take a fairly strict view of “contradiction”. Anything else works against fairness of process.
A contradiction is where
- One standard attempts to redefine another, or is a rival standard for exactly the same named thing but is different in some aspect. The precedent here is the contradictions alleged between two standards for UNIX-derived APIs that arrived at ISO by different routes ISO Linux API and ISO POSIX. The remedy is withdrawal and harmonization.
- One standard disrupts another. The precedent for this is the IEEE 802 WAPI issue in which the claim was that the changes would make existing conforming implementations non-conforming. The remedy is withdrawal of the contradicting standard and future harmonization. (However, the WAPI issue does not seem to have been resolved; certainly the Chinese approach (compared to the UK body) is that procedural issues should not be used to block standards, in particular that obsoleting old standards for allowing a “contridiction” to block the progress of the standard if the issues have been raised before and not disposed of. But it does suggests that fairness is a more important criterion than contradiction, at the end of the day. Or, at least, that using procedural tactics to block something may not find much favour in the eyes of the ISO Secretariat and many ISO participants. )
- One standard pretends to be another. The precedent for this at ISO are the UK objections to C++/CLI. The remedy for this is a simple name change, and other associated editorial changes.
- One standard incorrectly uses another. For example, if a standard said it used ISO SGML but allowed that to be invalid for no intrinsic reason. (I have not found any precedent for this, but it seems the obvious case.)
A contradiction may have negative effects, such as user confusion, but it is not the negative effects that cause that there is a contradiction; a highly technical standard will confuse anyone. It is the direct contradictions int he text of the standards that is involved.
So what, in those terms, are not contradictions?
- Overlap. There are many overlapping standards. For example, there is ISO 12803, a schema for books: if it is an absolute requirement that existing standards be re-used, ODF would not be allowed unless used the ISO 12803 conventions, presumably. Another example, there are two to five (depending on how you count them) standards for grammar-based schema languages at ISO (SGML/XML DTDs, Architectural Forms, RELAX NG, RELAX NG compact syntax, namespace-aware DTDs) let alone considering W3C XML Schemas as a standard schema language.
- Using a profile. A profile is a subset. Unless a standard forbids profiles, they are obviously allowed. Of course, a standard that does not take the requirements of a particular locale seriously riskd being voted down if that locale has a national body that is concerned. However, national bodies may be more lenient on standard coming in by fast-track, in that they may be works-in-progress where localizations are a stage subsequent to the initial standardization. ISO ODF is probably an example of this: ODF 1.1 may not handle every requirement of every writing system at the moment, but that is no reason not to have as a standard. And, in any case, it is not a contradiction.
- Doing your own thing within a specification. Specifications being fast-tracked that come from a different technological tradition may indeed have different conventions. For example, identifiers coming in from W3C will have identifiers in IETF formats (URI, MIME media types, etc) rather than necessarily adopting ISO formats usch as ISO 9070 Formal Public Identifiers. ISO ODF was not required to follow the prior ISO DSSSL formating model. Making up your own format or functionality is defensible (i.e., It may need to be defended) but it is not a contradiction. For example, ISO Schematron uses simple keywords for the Query Language Binding attribute, not URLs or ISO 9070 FPIs (plus most of them are reserved for future possible use, and recommended but not adequately explained, and one of them is for something that doesn’t even exist…”xslt1.1″ but is there due to historical reason. (We may take it out, but it doesn’t actually cause any harm, it is just reserved.)
- Reserved-but-unused or deprecated keywords with inadequate explaination of the operation that the keywords would have if they were allowed-for-use and not deprecated. Especially if the standard has not been divided in a way where there is an external registry of keywords, it is not unlikely that a fast-tracked standard that encapsulates the current state of a mature technology will have deprecated or reserved keywords. The best way to handle these is unclear to me, but the presence of such keywords are not contradictions.
(I should stress that even if you do accept that “contradiction” has the kind of very specific meaning that I describe here, and even if under this meaning the supposed contradictions detailed by Groklaw fall down on inspection, and even if it means that OOXML can pass beyond the Feb5 adminstration periood into the next phase unmolested, it says nothing about whether OOXL should be defeated (or accepted) at the final ISO vote by member bodies. The issues that national bodies use are their own, and while they certainly include technical ones they may include issues of national interest; whether their own developers would be better with OOXML as a standard, whether their public bodies are capable of distinguishing a standard for document interchange with a standard for exposing the documents of the leading brand, whether they feel bound by international trade treaties (a particularly novel argument, that one), whether it would in fact lead to an interoperable paradise, etc. )
(Note: this is a new entry, and I may update it, Wiki fashion, if improvements come to hand, especially concerning precedents and the results.)
Added 2007-0103 Another aspect of this, a mistake I didn’t realize people would make until I reviewed some of the comments people are sending to their national bodies, is that contradictions are limited to contradictions with ISO/IEC standards. The JTC1 directives, the JTC1 resolution both limit it to that. Even the supposed WTO treaty requirement does not extend itself to industry consortia like, say, OASIS, Ecma or W3C.
Rick, do undefined elements, licensing-related issues, or dependencies on a particular vendor's other products have any effect on these processes? I believe that these are the kinds of issues that are stirring up opposition to OOXML (along with the sponsor's stubborn refusal to implement the already-approved standard, ODF).
What a great objective analisys done by Miguel de Icaza on http://tirania.org/blog/archive/2007/Jan-30.html
W^L+: I wish people would scrutinize ODF for any particular defect before claiming that something similar is fatal to OOXML. In particular, ODF skated through an oppositionless process while being clearly underspecified in several respects.
Rick: Thanks for the analysis. The number of bugs and misunderstandings in the supposed "contradictions" is a great reminder that as humans we find what we are looking for. Starting from a presumption of evil is a self-fulfilling activity and a risk of sin in itself. Careful analysts work hard to uncover and correct that sort of thing in their work. I hold you in the careful analyst category.
(Great example of finding evil: Confusing the special-function highlight color settings for WordML with the treatment of the range of colors in DrawingML which, apart from some of the spellings, are identical to the SVG ones. I suspect this kind of thing tends to impeach the work in the eyes of experienced standards workers and the various standardization bodies.)
W^L+: Not on the particular issue of contradiction. For the final vote, member bodies *should* satisfy themselves on licensing and so on. But no-one just blandly take the word of one side, especially by buying into the premise that being pro-ODF requires you to be anti-OOXML.
Stijn: That is an extraordinary piece, I simply was not aware of most of his issues!
He (a leading open source developer) says Considering that ODF spent a year receiving public scrutiny and it has holes the size of the Gulf of Mexico, it seems that the call for delaying its adoption is politically based and not technically based. which also aligns with what Sun's Tim Bray has said in his Ongoing blog: that the "Big Picture" is corporate competition.
There are billions of dollars at play; but what is at stake is not MS' dominance (an ODF-only world will not exclude them, they have their plugin) or open source's inexorable rise, but whether the other commercial software vendors can survive being squeezed between MS and open source. They need ODF to be a lifeline: they cannot survive by fighting open source so their only option is to try to carve out some of MS' action.
I'd like to make a point about Catholics and Protestants. Well, not all Catholics and not all Protestants. And it could be applied to liberal Jews and orthodox Jews too, and undoubtedly other religions with Holy Books. There two views (in stereotype, please don't flame me, I'm not taking sides) can be characterized by their attitude to authority: in the Protestants, truth comes from the Bible which is given, frozen, complete, self-contained, unambiguous and authoritative; in the Catholics, truth comes symphonically from multiple sources, including the writings and lives of the Saints, the Pope (speaking ex cathdra on matters of faith and morals) and the Scriptures, and its themes develop and clarify over time as more work is performed and so on.
Are standards more like the Protestant view of Scripture or the Catholic view? I think the reality is that the goal is a Protestant-like ideal of a perfect revelation, but the reality is the through-a-glass-darkly Catholic-like approach, where standards may be works in progress, where they may refer to works outside them, where they may be ragged on the edges, and where you may have to access humans (shock!) to understand them sometimes. (A standard cannot be written with knowledge of what confusions of preconceptions its readers will have; consequently, especially in the early days, human liason with editors and committee people by implementers etc is important.)
So the fact that ODF (and OOXML) may have holes, is not a deficiency necessarily. It is just a standardization strategy. ODF chose incremental standardization, OOXML chose big bang standardization. But a new standard will always have things that need correction. Hopefully as few as humanly possible. Islamic carpet-makers supposedly put a flaw into their carpets, because only God is perfect...standards are more complex than carpets.
Orcmid: My understanding of the Groklaw/IBM(Weir)/OASIS(Updegrove) campaign strategy is to quote each other enough (while not making the paid alliances explicit at the citing end) to claim there is a groundswell of expert opinion, and to try to scattergun so many claims that no-one will bother to check much. Then they start off a letter-writing claim to national body committees based on this.
I used to vet draft ISO standards and the national body comments for the Standards Australia committee I served on in the mid-90s. I've done if for probably scores of standards. The Groklaw comments don't seem up to scratch, from a superficial skim. For example, I've mentioned before that the make some claim about it being impossible to validate bitmasks with Schematron because it doesn't have a bitset datatype, but this is complete crap: you don't need a datatype to validate, you can parse each bit (using XPath string functions) into its own variable, then have an expression that tests all the variables. And Ken Holman, the SC34 chairman, long ago published XSLT code for doing this kind of thing.
I don't know whether I'll look at the Groklaw piece seriously. Other people are starting to look at it. The shortness of the 30-day contradiction period indicates to me that ISO will look for *glaring* and unambiguous contradictions, not the kind of petty things the Groklaw list has; at least in the technical section, which is pretty much the only part that interests me.
Groklaw is a funny site. They claim to have aa legal background but when Microsoft had a reputable legal firm examine their CNS (and for OOXML Micrsoft grants both the CNS as well as the OSP) they never refer to that document.
It seems to me that an analysis by a legal firm is worth more then an analysis by Groklaw a site that says it provides no legal advise. Especially since Microsoft was the party that asked for the analysis so that is will hold up in any court as being an officially prvided legal view provided by Microsoft to it's relations and customers.
I found the analysis by Groklaw very disappointing. It also claims to be a site that is antiFUD but the OOXML analysis is almost entirly made up of FUD which I find very damaging for it's reputation. The newsletter from Groklaw which asked their readers to try to influence the ISO votingbodies of their countries I found to be shamefull even.
Oh, I think Groklaw provides a good resource for collecting counter-Microsoft material. I have emailed with Pamela before, and she seems nice, and she would cheerfully admit that she doesn't have a "neutral point of view", I am sure.
But a lawyer has no special advantage in knowing how to construe the word "contradict" in ISO terms. There are frequently lawyers involved in standard, SGML's Charles Goldfarb being the most notable, and their legal training helps them argue for a particular position no doubt, and it certainly helps them read and gives them a good vocabulary of concepts to argue a position from, but it is of no value in helping them draft or construe standards and procedures. If they went to the University of ISO and took a course in ISO procedures, that would be different; but the lawyers who are experts are not experts because they are lawyer. They are experts because they turn up to meetings, study the procedures, look at the precedent, and figure out what is fair and consistent with previous decisions.
Oh, I think there is nothing shameful about campaigns to influence standards, per se.
But the narrative being given is that this is a fight between Open Something and Microsoft, as if it were between rule by Democracy and rule by King. However, this fight is just as much a fight between rule by King and a rule by Oligarchy. It is Survivor, where the rest of the Tribal Council wants to vote out the leading contender so that they can remain on the island.
But yet another narrative is possible: that document processing people and archivists would benefit from a public standard for the major product in the market in a standard, non-binary, highly documented form, something that people in the industry have been crying out for for years. The document processing crowd is the core of ISO SC34 committee membership: the SGML/Topic Map/DSDL people who have been working for years on getting data exposed.
The uses of XML for blind data interchange between the office applications suites of the rich and powerful is nice for them, but it is simply not the only game in town; document processing (and archiving) may be a niche area for people who only use shrink-wrapped applications, but it has been in fact the core concern of ISO/IEC JTC1 SC34 (in particular for WG1) since its inception.
In fact, to an extent the office application suites have been the antithesis of what SC34 is about: non-extensible, loose structures with low-semantic labelling, lack of enforceable styles, lack of tools to enforce document consistency, emphasis on visual rather than structures, and so on. I think it would be unfortunate if, when MS has been forced to give us exactly what we have been asking for for years, the requirements of our traditional constituency of the document processing (and archiving) industry were treated as second class.
For readers who are interested in the ISO process for Fast-Track, MicroSoft's Brian Jones has a article up. (Microsoft is the stakeholder most closely associated with developing and pushing EOOXML to industry standards body Ecma and thence to ISO. You can tell he is a Microsoft employee because it is written in a banner above his blog that he is a "program manager in Office".)
For balance, OASIS's lawyer Andy Updegrove has an article which gives a much broader claim. (OASIS is the industry standards body that ODF has been standardized through. Some ODF people think that OOXML would compete with ODF. You can tell he is the OASIS attorney by clicking on the heading at the top, then scrolling down 11 pages...on my PC at least. Or, at least, I think you can tell; he is the attorney in charge of standards work for a law firm that has his name and which has OASIS as a client...I think that makes him their attorney. However, he writes that the blog is his opinions not necessarily his firm's, and he has many other clients too.)
Updegrove reports that INCITS/ANSI basically agrees with my analysis above about what constitutes a contradiction. Good.
Jones makes the point that the 30-day period is short because it isn't a detailed technical review. There is six months for that.
Updegrove says I am singing from the same hymnal as Jones, but it is the same song I expressed in Seoul last year. Updegrove says of me But isn't the question of the hour what ISO/IEC says a contradiction is, rather than what one outsider or another thinks the word should mean? then proceeds to quote an outsider giving their opinion, before giving his own opinion. (The quoted article does have one good citation from JTC1 [unverified, sorry], in which it mentions "evident contradictions" and equates them with "inconsistencies": this is hardly saying that any problem at all is a contradiction and showstopper.)
By the way, the other thing about the 30-day contradiction period is that if it takes a week to get hold of a draft standard, then many comittees have a deadline that material being discussed hs to be submitted say a week before to allow review, then another week procedural time to get the national body committee report to ISO, you easily might only have one week to conduct contradiction reviews.
So this really suggests that the 30-day administrative period is a coarse seive, not a fine one,so that really obvious contradictions can be raised and dealt with immediately rather than waiting for the final vote.
What would I expect to see in a contradiction claim sent by a national body to ISO?
1) A reference to the proposed ISO spec, giving section and hopefully page number.
2) A reference to the international standard that it supposedly contradicts (Not sure which specs: international? Class-A liason?), including section and page number where possible.
3) Brief description of the issue
4) An example, if it were needed
5) No external references: irrelevant
6) No references to products or business strategies: irrelevant
Rick,
I am yet to hear any good reason from you, or Microsoft, as to why such a complex 'standard' should be fast-tracked with so little time to properly analyze its technical merits.
The only argument both you and Microsoft can seem to muster is pointing the finger at the ODF approval cycle. While you and I could argue at length about the appropriateness of its approval cycle, it nevertheless fundamentally misses the point.
There is absolutely no valid reason to rail-road Microsoft's 'standard' as an ISO Standard. If I understand correctly, the ISO is meant to serve the best interests of global citizens, not the best interests of a specific multi-national organization, even if it happens to be very wealthy thanks to past illegal activities (for which it has convicted by both the United States of America and the European Commission).
Rick,
It's already a public standard, namely ECMA 376. The information you describe which the archivists require has (arguably) been made available, and (I presume) can never be retracted from the public domain.
The other objectives you describe do not seem to be fulfilled by either of ODF or OOXML, so while it's a well-made point, it's sort of a moot one.
The only value that seems to be added by OOXML to the existing body of ISO standards (Miguel's commentary on spreadsheets potentially notwithstanding) is its alleged usefulness to people who already have documents in a previous Microsoft Office format.
Why should any vendor be allowed to have an ISO Standard specifically for themselves and their customers? Regardless of technical merits or anything else -- if a specification has been expressly designed for the use and benefit of ONE vendor and that vendor's customers (past, present, and future) -- why should any international standards body elevate that spec to International Standard status?
I suspect that this falls under the "elephant in the room" umbrella though, which I suspect you will continue to "resolutely ignore" (as Tim Bray put it recently).
This whole thread is a hoot and I'm saving it for the amplifying remarks that you've provided. I'm one of those document-processing folk and my heart goes out to the SC34 folk on this one. I also trust that they have more than enough technical smarts to understand the irrelevance of the Groklaw compilation.
Frank: I don't speak about the time period much because it doesn't interest me. Traditionally, National Bodies abstain if they don't have enough resources to review a standard, not vote no; if N.B.s voted no for every standard they didn't have an expert to review, almost no specs would reach IS stage.
One of the reasons that non-fast-track standards are scrutinized carefully is because often you don't know if they work or not: they typically are improved versions of some existing technology but not yet fully implemented. So a standard where there is an definite implementation (dare I say, a "full implementation") requires a different kind of scrutiny; there is less scope for internal contradictions when dealing with a spec for something that has been implemented already, for example.
Claims that the 30-day admin period are not enough are really claims about ISO process, not about OOXML as such: they have nothing to do with contradictions. The detailed technical/format/national interest/wordsmithing review is still a full 6 months.
ISO allows "Fast-track" and that is what they've got. I am sure the ODF people will be moving to get Fast-track altered; this will then give them the opportunity to say "oh, Ecma were using a second-class process" which should be quit effective spin for people wanting to be spun.
On the issue of ISO serving global citizens, if blind interchange between competing mature and full-functioned office suites to ensure a level playing field was the only use of documents, then you would have a stronger point. But global citizens do more than that; in particular, the industrial publishing/archiving sectors that SC34 traditionally caters to does more than that.
ISO is simply not an anti-trust court. Extensible standards are necessary but inadequate to guarantee level-playing fields or blind interoperability. I think people are wanting ISO to do things that belong in the realm of regulation and legislation.
Alan: That is indeed a good point. Ecma need to make their case clearer for why OOXML should get to ISO, and I suppose that will be one of the things they will address over the next months. But that is for Ecma's champions, not me.
Orcmid: I've gone over the Groklaw material in more detail now. It is an interesting exercise to cross out everything that does not relate to actual ISO standards (including things that are criticisms of ISO procedures such as the 30-day thing), which is I expect the first thing that ISO people looking at claims must do, and to see what is left. I make about 5%.
Orcmid,
The primary message of the anti-376 movement on Groklaw is simply that the specification should be subjected to a thorough and complete technical review rather than just being catapulted through on the fast-track.
Is that really so wrong?
Doubtless there will be many unhappy folks around there (myself included) if the thing gets a full review and *still* passes, but at least there will have been the due care and process, rather than a rocket-propelled rubber-stamp as seems to be the risk here.
The primary goal (AFAICS, anyway) of the Groklaw compilation is to *get them* to look closely at the spec. That's all -- and even if they never so much as glance at the Groklaw page, as long as they're studying the EOOXML spec in detail and using those technical smarts to reach their own unhurried conclusions, that's good enough for me.
Having said that, I still cannot see any legitimate reason why we need two separate, incompatible ISO standards that cover the same ground.
Alan: That is certainly a very respectable concern, without buying into it. But I don't think the "broad spray of shot" approach as adopted by the anti-OOXML people helps technical people at NBs, ISO and SC34 work through the issues, as they ultimately must; it may help emotionally, but 100 weak arguments don't add up to a compelling argument in these kind of situations. However, if you think that standards people are asleep or unaware, you should read things like WG1's Alex Brown's blog OpenXML vs ODF in SC34: The Phoney War from June 2006.
A useful comment from an ODF supporter Stephen Walli (a former SunSoft and MS employee turned open source developer, with experience in IEEE and ISO standards processes), who seems to want the anti-OOXML side to have substance rather than slogans:
"I would offer, however, that a contradiction should not be defined as a simple overlap with another standard. This is economically a poor yardstick to use. We all saw this coming last Spring. At the time I observed:
"While ISO certainly doesn't like to encourage competing or overlapping standards, they will not necessarily prevent them. They are a standards development organization in place to ensure that the rules of development are transparent and followed. It is not their role (nor do you ever want it to be) to manage the marketplace through determining the economic viability of a standard.
"By all means send the Ecma International specification back for some of the egregious internal conflicts, and ugly artifacts like date redefinitions. But let the market decide which standard has the best value proposition to solve customer problems with the most implementations. [We already know which will win.] Consider the IEEE 802 standards family. If they didn't allow standards that overlap in functionality, we would still be living on star LANs."
Now the original posting he mentions has the following opinion: "Then there's the issue with respect to competing standards. This too is a red herring.
"While ISO certainly doesn't like to encourage competing or overlapping standards, they will not necessarily prevent them. They are a standards development organization in place to ensure that the rules of development are transparent and followed. It is not their role (nor do you ever want it to be) to manage the marketplace through determining the economic viability of a standard.""
Err, sounds like me doesn't it?