Oh, I guess that would make me me pretty much right then…
The excellent Erc Lai at ComputerWorld is reporting that JTC1 has accepted Ecma’s responses to the recent contradiction review, and that matters are proceding to the next phase of the voting as normal. It is worthwhile reading these responses, for people interested in openness or getting the other side to the story. (The rest of the post is based on the assumption that his report is correct.)
I think I can speak a little more frankly now. I didn’t want to inflame the situation before. I think certain stakeholders quite cynically or gleefully inflamed passions and expectations unnecessarily by utterly inflating the importance and urgency of the contradiction phase. I mentioned in my blog my opinion that contradiction will mean something quite limited and specific, that consistency with previous practice is a guide to what to expect, that a 30 day review really is only intended to catch the broad problems that could be found by a quick skim not detailed technical issues, that detailed technical issues get dealt with by the six month review, that JTC1 would not take on economic or political arguments, and that the whole process is geared towards resolving issues (win-win) not win-lose. The big vote is the FDIS national ballot
What we don’t see from these stakeholders is “Oh I guess we were quite wrong on what a contradiction is; oops.” Instead, we are starting to see a shoot-the-piano player reaction; will we now see attacks on the individual officers, on the secretariat, on the ISO procedures, blame on everything except the bad advice? I see somone lamenting “Have we no standard for standards?” but that is what the JTC Directives are. I think they will now have to whip up emotions that the process is flawed, so that their true believers won’t start thinking “oh, perhaps there are two sides to this story” or even “oh, perhaps being anti-MS and pro-ODF does not require me to be anti-ISO OpenXML”.
The thing is that the contradiction review is actually a small formality to prevent wasting time and to escalate certain problems to JTC1, not THE ONLY CHANCE TO STOP THE MONSTER! The focus of the process is first to get issues raised, ascertain that Ecma has responses that indicate the matter can move forward using the normal process, and then to get to the more detailed technical review stage.
So the Ecma spec is now at the “Draft International Standard” stage (DIS).
In five months time, as I understand it, there will be a ballot of the national bodies that are P-members of SC34 (participating members, rather than observers). In ISO procedure, there are votes for “abstain”, “yes”, “no” and “no with comments”. I expect that OpenXML will have a lot of “No with comments”, which happens sometimes. These comments give all the technical (and perhaps IPR) issues that have been found that are showstoppers, together with other misc comments.
After this ballot, we all wait two or three months. This gives everyone time to draw their breaths, gird their loins, examine each other’s positions, and prepare responses. Then there a ballot resolution meeting, at which all the issues are dealt with; the meeting may respond with a fix to the DIS, or with a comment that this is an issue for further study and enhancement, or they may to decline to fix the problem, or they may say that it is not a problem in their opinion. Then Ecma
The FDIS then gets sent around to the national bodies, and a vote is taken after 30 days. So we are talking, 1 month admin review (contradiction) + 1 month (Ecma response) + 5 months (detail review) + 2month (collection) + 2 months (resolution) + 1 month (pre-vote). A Fast Track standard with any controversy actually has about 13 months of review time before the final vote minimum! Now contrast this with the panic from the “30 days is too short” crowd. I cannot believe the instigators didn’t know better: even a fast-track procedure is lengthy, open and serious.
So does that mean that NZ, Canada, Singapore, Kenya, and so on were wrong to raise their issues? Not at all. It is good to have all the issues on the table, and opinions can legitimately differ, especially in the absense of explicit guidelines or precedent on what contradictions are. I expect that the JTC1 secretariat and the ITTF would have a very limited class of issues that would come into their perview, and that otherwise they would punt to the normal mechanism. Indeed, as was the case with Australia, it is good that they pass on the various comments earlier rather than later, so that they can be responded to.
What is important to realize is that even though JTC1 may have found that Ecma’s responses to the contradiction issues are satisfactory enough to let matters proceed, it does not mean that therefore the issues themselves go away. It is not JTC1s job to make technical decisions but procedural ones, is one way to put it. I expect that some of the issues will be re-put at the DIS ballot, some will have been answered by the Ecma response already, and there will probably be some more. The DIS ballot resolution process looks like being a long and difficult job, but it will make the claims that OpenXML had no input from an open process fairly untenable; probably ODF should have had the same amount of scrutiny.
For people interested in ISO procedure, there is a little summary of Fast Track Procedure prepared by or for Jim Mason (SC34 chairman) and Ken Holman (SC34 secretary), which I have been relying on. SC34 is the ISO/IEC JTC1 committee that looks after Document Description and Processing Languages, and you might know them from such standards as SGML, Topic Map, RELAX NG, Schematron, and from participants over the years such as Charles Goldfarb, James Clark, Murata Makoto, Sharon Adler, Anders Berglund, Patrick Durusau, Martin Bryan, Yushi Komachi, Jon Bosak, Elliot Kimber, and so on.
The process may work as you're describing. However, Marbux mentions a rule change on February 20th to enable the forward movement without delay or derailment.
I have to admit that ISO standards have, for most of my life, been pretty irrelevant. Thus, I am admittedly unfamiliar with the process and review that leads to standards. However, I think Marbux' earlier analysis of the issue, namely that international law (as in the UN, GATT, and other approved intergovernmental organizations) constrains governments to follow such standards in general, conforms to the outlines of my experience. Such rules would, of course, have economic and legal / political purposes, much as the Sherman and Clayton acts in the US (meant to prevent and break up anti-competitive business groupings). By this, I mean that it is in the interests of non-US governments to ensure that locally-owned competitors have the chance to compete for software buyers' money. It is in the interest of US state governments (at least in tech states like Massachusetts and California) to do likewise.
It is encouraging to see state governments in the US discussing and considering policies meant to place ownership of user' documents back into their hands.
Whether it is now or in the future, the ISO process will change. The new Congress in the US will stand up for consumers and seek to ensure that internationally-approved standards consider the needs of end-users and not just those of vendors. The only question is when–before the PAS process ends or later, after revisiting the "net neutrality" issue–and whether ANSI will give in to Congress or lose its ability to serve as NIST's representative.
So I ask you now, Is OOXML worth squandering ISO's reputation and status over? Because that is indeed what is happening.
I would urge ECMA and Microsoft to sit down with OASIS and work to make the existing standard better for all, rather than continue on the foolish path.
W^L+: I'll have a look for those changes to see what their effect is.
The trouble with Marbux' claims (which seem pretty dubious to me) is that they don't seem fit the facts we have here. If we will now have five ISO standards that potentially fit office documents (HTML, PDF, DSSSL, ODF, OpenXML) he/she seems to be claiming that international law somehow forces governments to accept all of the formats. Instead, I expect they will increasingly adopt HTML and ODF by preference, with PDF and OpenXML allowed temporarily where HTML and ODF don't provide coverage. I think OpenXML's big market will be for companies who already have large Office investments, but who are moving more to a Glushko-esque Document Architecture basis.
But all of them will need to use profiles. And I think in the medium term ODF will add equivalents to any Ofice-specific formatting rules, and we will start to see some mix－and-match (ODF with OPC, OpenXML formulas, OpenXML document assembly, and so on).
What I would have liked to have seen from the ODF side is a stronger roadmap: we have ODF 1.0, with 1.1 bedded down and 1.2 on the way, but I would have liked to have seen a clearer indication of where things like OpenFormula, OASIS CAM, fit in to provide alternatives.
If I were king of ODF, I would take a different approach. i would show which kinds of documents ODF and implementations was currently useful for. I would make (by hand, not by application) a set of basic government documents which validate against the ODF schemas, and then make these available to all the ODF applications to get their basics right. A single page document with a business letter with a logo, sender, and addressee paragraphs ( testing graphics, left, right and full justification and footer) would be one. Then make a gallery of applications showing that they can handle them. A set of these kinds of test documents would clearly show the tidemark of interoperability; governments and regulators could then make internal rules such as "6 months after the tide covers a certain kind of document, we will require all documents of that kind to be generated using ODF" or (more practically) "When the tide covers the cases of all our common documents (it may already!) we will adopt ODF for all those documents but allow and exemption for specialist documents (which may use OpenXML accompanied by PDF)."
Then ODF products can advertise themselves "KDE: meets the requirements for ODF basic office word processing document interchange" or "Gnumeric: meets the requirements for ODF advanced spreadsheet document interchange" or "OpenOffice: meets the requirements for ODF intermediate presentation document interchange". This would allow regulators or procurement departments to express their constraints in something meaningful and practical: "Departments should adopt open source products that meet the appropriate ODF itermediate document interchange requirements" for example.
Back to international law. IANAL. Wouldn't any treaty constraints only apply to activities with an international aspect? I don't see how a trade treaty can force a local government agency to save its private internal memos to its local hard-drives in ODF, for example. It strikes me that Marbux draws too long a bow.
Furthermore there has been zero economic analysis that I have seen. The assumption that an ISO standard of OpenXML would provide a barrier to trade or be biased towards US is hard to reconcile with the existance of ODF (which legislators could adopt as the preferred format) and the long-running calls for MS to open its formats up (it is the secrets that provide the barriers.) I think many of the ISO voting countries see OpenXML as a way to shift the balance of power away from MS as far as the default save formats, with the ODF interoperability angle being to some extent a sideshow.
Everyone wins: MS gets to be in the game and several new games at the expense of opening up; organizations get access to their data and an increasingly viable alternative to MS' native formats; markets get genuine competitors; governments get a richer solution space; ODF gets documentation on Office, together with clear indications of what features MS considers legacy (which they don't need to implement); archivists get ways to rescue the information in binary formats; other commercial vendors with open source or non-open source products get increased information and interoperability with the prospect of ODF increasingly allowing the non-MS office; anti-trust people get the ability to validate documents from Office against the OpenXML schemas to prevent MS from arbitrarily changing their format to block out competitors; systems people get the robustness of document-based conversion rather than API-based fragility. (I guess the people who don't win are those who want to treat ODF as a euphemism for blocking out sales or use of MS Office. Standards are good for getting level-playing fields but not immediate or effective remedies for marketplaces that go into a monopoly position, because the monopolist will just adopt the standard and continue to have the benefit of their market position and installed base.)
>Everyone wins: MS gets to be in the game
>and several new games at the expense of opening up;
>organizations get access to their data and an
>increasingly viable alternative to MS' native formats;
>markets get genuine competitors;
It seems you forget that OOXML is *now* a Standard ( ECMA one ). So, all this good things you mention, can happen now !.
Let it be that way. Don't taint ISO with this ( keep the base level of quality of standards please ! ).
Microsoft needs ISO ratification for business reasons ( competition ). The name they choose reflect this: Office Open XML ... against Open Office XML formats. By the way, MS C++/CLI standard, proposed by MS to ISO fast-tracking was rejected for a similar issue with naming.
We owe Eric Kriss ( http://en.wikipedia.org/wiki/Eric_Kriss ) that MS is now so "open" with their formats. But by rushing the ISO approval, MS is showing no respect to standards. IMHO.
Your previous disclosure raised more questions than it answered.
Are you still working as a hired expert for Microsoft?
Is this blog post part of your paid work for them, directly or indirectly?
omz: What I find interesting is how often people will answer an article pointing out that some commonly stated position is not really correct with just a blanklet restatement of the position.
So when I point out that the standard will take 13 months minimum to get through ISO (and add that to the time taken through ECMA you get much more, and add that some of the languages and documentation have been in development for five years already) and yet there still is a talk of a "rush". I've mentioned before that the kind of development effort that an external spec requires of ISO working groups (i.e. none) means that what you are left with is just the review period. I think OpenXML DIS will get had a pretty thorough review; and there is always the option for national bodies to ask for the various markup languages of the DIS to be split up into parts for incremental review, if they feel that some parts are more acceptable or have been reviewed more than others. That would stagger the process by a few months, but in the tortoise world of standards that is not long.
The same is true of ODF, but it is a much less complete standard because it invokes so many non-ISO standard formats (W3C SVG and W3C MathML in particular.)
Finite: No, not directly or indirectly. The blog isn't for sale, nor would the MS people I deal with be crass enough to even ask. (See the disclosure notice on my bio page.) But I am happy to do consulting, seminars, technical reviews, pilot projects, schemas and organize custom software for anyone, including ODF and IBM.
As far as I know we haven't actually billed MS for anything yet; I have been delayed in tracking a bug in Xerces down because Eclipse on my Linux system just stopped working last week, and that is holding back wrapping up a couple of projects...sigh
My dealings with MS are: 1) They bought a copy of my company Topologi's Schema Evolution Utility for the XSD to RELAX NG conversion last year, 2) I have worked 3 out of 4 intended days on improving Wikipedia entries related to OpenXML technical, evaluation and procedural issues with no interference from MS, which I have done through the comments section not by direct editing so far, as competely allowed by Wikipedia's rules 3) I have spent under a week discussing and improving a seminar on office standards that Allette Systems (who I have presented training sessions for, for the last decade) looks like running in various parts of Asia and the Pacific for MS which I may present some of the more technical material for, and 4) IIRC Doug Mahugh bought me 1 beer, 1 lunch (fish and chips) and 1 dinner (noodles) when he visited to discuss the courses (Allette bought some beers and lunches for them, too).
During the last months I also have worked on our Java tools, Topologi's IBM WebSphere and Linux Apache hosted Schematron firewall tool Interceptor, an open source Ant task for Schematron (i.e. Java), the open source XSLT-based new version of ISO Schematron, an OpenXML pilot conversion project for a defense contractor, a Schematron pilot project for a national government, and several tail-end jobs which don't relate to MS at all. I've also been following SC34 committee issues and had quite a lot of email (including with moderate and constructive ODF people, who I think are in the majority.)
I've had to spend far too much time on OpenXML and ODF on this blog, because I found with the dumb comments after my Interesting Offer post, that there is a section of partisans and flamers who are only too happy to skew and slander, especially under cowardly anonymity. It is better to respond fast to comments, even from parrots and sock puppets, otherwise the mud sticks. ODF fans who would prefer me to spend less time discussing OpenXML should stop repeating bogus claims (I see the bitfield thing has been wheeled out again, for instance, gawd! If they just said "Bitfields are not best practise, in the absense of unusual size tradeoffs" it would be fine, but not to clim that bitfields cannot be used by XSLT or Schematron or are not extensible, which is just ignorant, in the case of markup languages, which can have extra attributes added any time.)
The thing is that, as is clear from the national body comments in the administrative review phase, there is a strong desire from national bodies that the people involved in SC34 (and WG1 is the working gorup involved) perform reviews of fast-track specification. Now the participants in WG1 are (despite being incredibly ancient), public spirited, stubborn, painstaking, opinionated, highly experienced, technically competent, and fiercely independent. None of us get pay, let alone bonuses, if a particular standard succeeds or fails.
That you might not agree with me that fairness of procedure overrides anti-trust concerns in the forum of ISO technical committees, is fine. You might think that the needs of indstrial publishers and archivists are minor compared to the needs of office suite manufacturers, that is fine. You might think that having multiple standards will confuse legislators who are too inarticulate and guileless to find a form of words that lets them achieve their ends, that is fine. But I would ask you to be open to the possibility that someone serving on a technical committee may have a different opinion to yourself not out of sinister motives and money, but just because they have a particular technical point of view and experience and values that honestly leads them to a position. This blog reflect my private opinions and daily interests and has frequently criticized MS (and Sun for that matter) views or technologies where I thought warranted and I have frequently endorsed ODF for certain uses.
However, in a sense it is a fair question. I now regularly try to quickly find the identity and affiliations of people who ask questions, because of all the sock puppetry. (That is one of my main criticisms of the Wikipedia pages, actually, that they have been written quoting people without giving any details of affiliation: people who actually work fulltime or longterm on projects get quoted and linked to as if they are experts with an independent track record.) A mate of mine actually suggested that I should not allow any anonymous or nicknamed posters, to reduce cowardly attacks and hidden PR campaigners.
[Rick]: ...contrast this with the panic from the "30 days is too short" crowd. I cannot believe the instigators didn't know better: even a fast-track procedure is lengthy, open and serious.
________________________________________________
Rick, I'll disagree. That "crowd" are actually nations, as in National Bodies. Reviewing over 6,000 pages of a technical spec in 30 days means comparing and contrasting over 200 a day. Can you do that? Can any human? If not, then it's Microsoft and ISO that is not taking the review process "seriously," wouldn't you say? As Andy Updegrove notes: "There are many national bodies that are (a) genuinely offended by how this is being pushed through, and (b) actually interested in open formats, and interested in seeing a good result." It is the Microsoft "crowd" that seems to have turned this into an adversarial process, where the national bodies must be defeated or overcome in order to make Ecma 376 an ISO Standard. That's a foolish stance to take -- ceclaring victory because you were able to ignore and bypass objections from NB's is the opposite of serious. Look no further than the way that the OASIS ODF TC handled the accessibility concerns raised in Massachusetts should be a model for how to handle criticism of a standard. OASIS didn't push the critics away. They brought them into the process, asked them to help solve the problem, and turned their recommendations into a revised standard.
That ISO failed to define what constitutes a contradiction in this case is both odd and negligent. If a contradiction cannot be defined, then by definition Ecma 376 has none! By the tone of your post, you seem to be saying: "What are all 'you' people bitchin' about? Step aside, shutup, and let Microsoft rule." On top of that, while defending the ultra fast-tracking of Ecma 376, you criticize the lengthy process that ODF submitted to (and which Microsoft refused to participate in). Did Microsoft take ODF "seriously?"
Further, I've yet to see a discussion on how two document standards serves the end user. I see Microsoft's interests being served, I see corporate interests being conveniently sustained, but we both know that a 6,000+ page spec is only realistically implementable by Microsoft alone. No other vendor could afford the man-hours needed to sort through all the contradictions, not to mention reliably account for all the proprietary interdependencies built into MS-OOXML. What is being standardized are things that have a direct impact what we as individuals can do with the digital lives we all now have. Standards related to DRM, encryption, spam, media formats, document formats, etc., should be of interest to consumer organizations, policy makers, politicians, and end users. Some standards organizations, like ISO, are slow to recognize the change. Standards are no longer for technology that is deep within the back server room, unseen, unheard. The technology is now on the desktop, in the living room, in the car, used by school kids and grandparents, in all languages, across the globe and instantly. These standards are decisively important; hurrying them through the process at the very least makes Microsoft look insecure at the least, and desperate at the worst.
Zane: The premise of your first argument, that the adminstrative review period involves a detailed technical review of a specification, being wrong, your conclusion, that there was not enough time, is not supported. And conformance doesn't require full implementation.
As for me saying "Step aside, shutup, and let Microsoft rule," learn to read honey.
By way of follow up, here is the relevant section on contradiction resolution from the ISO Directives (recently updated to clarify, actually) 13.4:
"If a contradiction is alleged, the JTC 1 Secretariat and ITTF shall
make a best effort to resolve the matter in no more than a three month
period, consulting with the proposer of the fast-track document, the
NB(s) raising the claim of contradiction and others, as they deem
necessary. A meeting of these parties, open to all NBs, may be
convened by the JTC 1 Secretariat, if required.
"If the resolution requires a change to the document submitted for
fast-track processing, the initial document submitted will be
considered withdrawn. The proposer may submit a revised document, to
be processed as a new proposal.
"If the resolution results in no change to the document or if a
resolution cannot be reached, the five month fast-track ballot
commences immediately after such a determination is made."
What is note-worthy, I think is that the focus of finding contradictions is on getting them resolved so that the DIS or its successor can be put on track again. It is mistake to think about ISO procedures in terms of guilty/innocent right/wrong good/bad judgements, and the triumphant detection of showstoppers, rather than in terms of mediating agreements.
i'm often feeling that is in vain to argue with rick... with my respect to you rick... but well, here we go again:
>[rick] omz: What I find interesting is how often people will
>answer an article pointing out that some commonly stated
>position is not really correct with just a blanklet
>restatement of the position. So when I point out that
>the standard will take 13 months minimum to get
>through ISO (and add that to the time taken
>through ECMA you get much more, and add that some
>of the languages and documentation have been in
>development for five years already) and yet there
>still is a talk of a "rush"
You said "13 months of review". How optimistic view of the reality! ;-)
This is the impression of a member of JTC1SC34 about the "behemoth" ( his words ) task they were given ( note: JTC1SC34 was assigned to manage the fast tracking of Microsoft's OOXML ):
"
The kind of fast-tracking procedures used for ODF and mooted for OOXML give very little time for a small part-time volunteer group to give such bulky documents adequate scrutiny ... With OOXML predicted to weigh in as a behemoth 7,000 page standard the danger that OOXML will be inadequately scrutinised is greater still.
" [1]
[1] http://adjb.net/index.php?entry=entry060602-055251
Omz: Why did you elide Alex's middle sentence? "There is no doubt ODF is, right now, less good than it would have been if subjected to the full rigours of ISO standardisation"
I think Alex (and I think this is a view shared by others in the UK delegation to SC34) is of the opinion that probably neither ODF nor OpenXML should have been standardized, with the benefit of hindsight, without greater technical involvement of SC34 to vet the standards in some way: he doesn't have confidence in the fast-track procedures. (I have proposed behind the scenes to some SC34 people that there should at least be an opt-in period for SC34 provide NBs with a review.)
I don't know whether Alex goes as far as thinking that ODF should be withdrawn as a standard however, or whether he thinks (as I do) that if ODF can make it through (erring as it does on the side of incompleteness for ODF 1.0) then I can see no reason why OpenXML cannot (erring as it does on the side of emetic garrulity.)
Now in the case of OpenXML, I think there is now no chance that it won't be one of the most reviewed specs ever to go through ISO standardization; it won't just be the small team of experts that work with SC34 and NBs, but a much greater range of people.
Alex is certainly right in being concerned about "the danger" of a large spec, but the solution is for people who are interested to actually review it. (Indeed, the same thing goes for ODF: if you are interested in that, get involved in the OASIS ODF 1.1 or 1.2 review. OASIS membership for individuals is cheap, but there is probably a mail list you can send comments to directly: it is probably best to lurk a little first and read old posting to avoid wasting editors time with already-resolved issues though, of course. But getting involved in making ODF better is the best way of making sure that it "beats" OpenXML, if you see matters in those terms. )
This will provide the challenge for assessing the review material that we can expect a lot of the reviewers to have a very narrow perspective, in particular to have no experience of document engineering or document systems or industrial publishing, the large SGML-style document systems, but to be view everything through office-suite-colored glasses or even through HTML-colored glasses. Its not that they don't have a legitimate point of view, it is just that it is only one point of view, and not one that has any right to be treated more favourably than other market requirements at ISO.