The IETF has promoted the next revision of XML standard to recommendation status. Among the improvements for Xml 2.0 are no more schemas, reduced processing frameworks, an expansion of namespaces, and automatic transformation of tags to other formats such as jsOff, CSV, Sql, and a compressed binary form based on two’s compliment.
Bill recently commented on another small flare up on the REXML front. It is too bad that Ruby doesn’t have a better set of libraries for XML. As Bill mentions, Python does a great job with XML. He mentions ElementTree, which is definitely better than something like pure DOM. Lxml is another option, which actually implements the ElementTree API and includes some pretty slick objectify functionality. Ian recently performed some rather unscientific, but still interesting, benchmarks on some Python libraries for parsing HTML. Ian found lxml to be quite the performer. There is also the 4Suite and Amara toolset that provides a very comprehensive suite of XML tools including an entire XML/RDF based document repository and full featured XSLT engine.
It makes me wonder why the Ruby community have not stepped up with some better options. The Python community is very similar in that XML has not been a hallmark of the community as compared to Java or .NET. One argument could simply be time, since Python has been around a bit longer. No matter the reason, I think it is time for the Ruby community to consider stepping up and producing a healthy alternative to REXML. My first steps would be to start with the libxml bindings and go from there. Lxml and Amara have both proven that utilizing a fast C library for the grunt work pays off in the end.
Lastly, I want to make it clear that REXML is still a pretty great tool. It meets the needs of many of its users, which is more than many software projects seem to accomplish. With that in mind, lets not stop there when we can do even better to make Ruby a great language for working with XML.
One valid answer to the question in the title would be: I’m both into linked-data and RDFa. Hey, but that’s not the answer you are interested in, right? We’ll have a look into both and find a better answer by the end of this post. Oh, right, by the way, let me introduce myself shortly. I’m new to xml.com and I try focusing on Semantic Web stuff.
In the beginning, there was the URI. Kingsley recently wrote about it, coming from the plain old untyped @href hyperlink. Then there was RDF, not so well known, and still often confused with one of its serialisations, namely RDF/XML. But there are other ways to deploy RDF as well. In a couple of weeks, presumably, RDFa will be finalised by W3C. RDFa is all about delivering structured metadata in HTML. Much as microformats, RDFa uses attributes to ‘hide’ - or, more technically: embed - metadata in HTML.
Coming back to URIs: The hyperlinks basically were the success factor of the Web as we know it. Typed or semantic links are expected to be the same for the Semantic Web. TimBL wrote up the so called linked-data principles a bit ago (URI for everything, HTTP URI, RDF properties). An example might help understanding both RDFa and linked-data; compare
this page is under <a href="http://creativecommons.org/licenses/by/2.5/">CC 2.5</a> license this page is under <a rel="cc:license" href="http://creativecommons.org/licenses/by/2.5/">CC 2.5</a> license
The key is the rel="cc:license" bit. This is actually a piece of valid RDFa (telling that this content is under a certain license) and equally is a typed link. It overloads the simple @href hyperlink and let’s an agent (be it a search bot or a syndication site) interpret and follow it properly. I think you get the point, right? To sum up: RDFa is the way doing linked-data. Coming back to the initial question, I guess the main point is that both are manifestations of the real-world Semantic Web emerging these days. While in the last couple of years most of the people involved in Semantic Web stuff maybe thought ontologies and reasoning are the most important issues to deal with, it’s a bit like building a marvellous roof and finding out one day that there are no walls, and not even a foundation to put it onto.
Patrick’s forward-looking post mortem is worth a read by everyone involved in standards over the last year.
Update: via a recent follow-up comment from Rick Jelliffe, we have ourselves our QOTD,
If DIS 29500 mark II has been accepted, then the narrowness of the victory needs to be something that Ecma and Microsoft take very seriously: standards maintenance needs to be a budgeted, normal cost of doing business. They should be aware that they are being thrown a lifeline, to some extent. If this becomes a one-off publicity stunt, as is the dire warning of MS’ competitors (and therefore, their own publicity stunts!), and timely, real maintenance is not performed, I would expect OOXML would be de-standardized at ISO.
[Original Post]
Open XML appears to clear ISO standard vote | Tech news blog - CNET News.com
Early reports Sunday indicate that Office Open XML (OOXML) appears to have enough votes to be certified an ISO standard. An official tally is not expected until Monday.
Some of you may have noticed that I decided a while back to ignore the whole OOXML/DIS 29500 debate here on XML.com. Two reasons: 1) Too much cost, not enough gain. 2) Rick Jelliffe had things covered from top to bottom, someone *MUCH* more qualified and capable than I to provide a proper perspective of what was going on and what it all meant.
I’m just getting back to Salt Lake City after spending the last 4 days in Seattle/Redmond at the Microsoft Technology Summit. Had a *GREAT* time, meeting, for the first time, a few folks that I’ve known through email and/or user groups/mailing lists and/or industry reputation (the good kind ;-) for quite some time. I hope to do a proper summary of the entire MTS08 event before the weekend comes to an end, but in the mean time…
[UPDATE] I thought I’d give some graphs for the results of the ballot-changes of DIS 29500 mark II. These are the results as at Wednesday, and I think they are the finals. (There is one non-P NB whose vote I am not sure of: I have shown it as abstain though it could be accept.)
Here is a graph of all the votes case, showing the change from the initial ballot until now (as far as it is publicly known). This is based on all the NBs who voted. (However, this is not the count that is used to determine success…)
At ISO/IEC JTC1, national standards bodies (called NBs) nominate what kind of participation they are interested, for each of the multiple subject-oriented Steering Committees (SCs). They can nominate in two classes: Participating Members (P-members) are supposed to maintain an active interest, attend meetings, and vote on all the standard drafts that come up. Observing Members (O-members) can vote, but they don’t have any obligations to show up to SC meetings.
Here is a graph of all the votes case, showing the change from the initial ballot until now (as far as it is publicly known). This is based on the NBs who are O or P members (However, this is not the count that is used to determine success…)
Here is the vote when you just look at the P members (as far as it is known.) Note that “abstain” votes have a very particular meaning in ISO: it does not mean “reject” or “protest”, it means that the voting body could not decide, or is happy let the consensus of other NBs determine. There is no shame or difficulty with an NB voting abstain. (At earlier stages of drafts, there are “No with comments” votes: these often are “conditional yes” votes, which can explain how a “reject” vote can become an “accept” vote. At the current stage, however, no means no.)
Finally, now we have seen the big picture, we come to the real numbers that count. There is a negative test and a positive test. First, no more than 1/4 of all NBs who vote can be negative (ignoring abstains). This has not been reached (i.e. not enough rejections: the all-nation acceptances are over 75% on the following graph.)
Then there is a positive test: at least 2/3 of all P members who vote should vote for acceptance (ignoring abstains again). This has been reached (i.e. enough acceptances: the P-nation acceptances are over 66.7% on this graph.)
So OOXML has been accepted, seemingly by 24 to 8, which is enough of a margin to avoid “hanging chad” clawback games.
It is clear that most NBs think DIS 29500 mark II makes a credible and acceptable or useful standard, but there is a substantial and active minority that does not.
The Java Community Process is the mechanism Sun set up to develop and evolve Java “in Internet time”. It brings together “a cross-section of both major stakeholders and other members of the Java community”. A group of experts make the initial draft, then “Consensus around the form and content of the draft is then built using an iterative review process that allows an ever-widening audience to review and comment on the document.”
The result is a specification, a reference (proof of concept) implementation, and a technology compatibility kit (tests).
One specification I have been interested in for a while is JCP 296 the Swing Application Framework. The JSR (Java Specification Request) was approved in May 2006. There is an implementation at Java.net.
However, I cannot figure how to find the spec. Looking at the JCP site, there is everything about the spec, but no actual link to it. Looking at the implementation site, again no actual link to the spec. This strikes me as an entirely odd way to do business. What are they trying to hide? :-) Whatever it is, they are doing an excellent job of making sure that no-one finds it.
Looking at the site, it seems JSR 298 is marked “in-progress”. That means, I suppose that it is still a committee draft, that has not been released. After 18 months? So much for Internet time!
It seems like in order to see the draft, I will have to sign up to be a JSP member. For an individual member, it is $0 which is nice, but I have to send a fax to the other side of the world and await them to fax back a password. Or I can fax and send hardcopy by courier.
But even then, I don’t actually know that the JSP 296 draft is available for community review. The status is given as “In progress” but there is no mention of this status on the JCP description page. Presumably for the last 22 months the draft is being written. I presume a draft exists, because it has software that claims to be an implementation.
What is interesting is that this is the opposite of the ISO process. At ISO using the normal rules, it is the early drafts (working drafts, committee drafts) that are given the most exposure and can be floated around openness, and only the very final draft standards that are supposed to be controlled (to reduce interoperability problems where people write systems according to different drafts rather than the final standard, and for the standards that are published commercially by ISO and standards organizations for cost recovery.)
While I am generally in favour of committee room secrecy, to prevent intimidation and silly marketing point-scoring and to disenfranchise armchair experts, and while I can understand that drafts can change substantially so you don’t want to have old drafts floating around, openness is better. But after 22 months, and after there is an implementation, to have no actual draft casually available is not “Internet time”, is it?
If there was any single issue with EC2 that was harder to overcome than any other — at least mentally if not physically and/or technically — is was that of not having access to a static IP that you could rely upon being there regardless of what machine it was mapped to.
That has now changed…
Amazon.com: Homepage: Amazon Web Services
We are excited to announce Elastic IP addresses and Availability Zones, two features that were among the top requests of Amazon EC2 developers. These new capabilities allow developers to achieve greater reliability and redundancy for their applications in the cloud, especially hosting websites. Unlike traditional static IP addresses, Elastic IP addresses can be dynamically remapped on the fly to point to any Amazon EC2 instance. Also available is the ability to launch instances in multiple Availability Zones, each with its own reliable, physically independent infrastructure, which allows developers to build fault resilient web applications through simple API calls.
Of course if there was any other single issue that was the source of significant pain and/or worry it was that of not having the ability to guarantee against single (hardware) server meltdown, or in other words, there was no ensure that if you have multiple instances running that these same instances were not running on the same physical piece of hardware. As per above, that has now changed as well.
*SWEET*! :D Thanks, AWS!
There has been so much disinformation put out about the limited review time for OpenXML, that it might be salutory for people to revisit a review of the Open XML draft I put on this blog dated Thursday May 25, 2006.
You read it: May 2006. That is 22 months ago! Not “5 months”, not even 9 months as the claptrappists say. June, July, August, September, October, November, December, January 2007, February, March, April, May, June, July, August, September, October, November, December, January 2008, February, March.
To the people who are saying they have not had enough time in 22 months, I have no sympathy: you should have been reading my blog! :-)
I want to go through all of what I said then. I think it holds up really well.
A new draft of Open XML came out on my birthday. 4081 pages of PDF, and very impressive for anyone who has worked on specification and standards. Two things stick out: first how horrible XML Schema fragments are when stuck inline to document structure; second, how the implementation-neutral tone of the introduction is at odds with the elements for various kinds of Active X embedded objects. I suspect people would be a lot more comfortable if the elements for Active X embedded objects were in a different namespace, and gathered into an appendix of some kind. Antiques and curios. It will be interesting to see what the extensibility strategy will be (it hasnt been released in this draft.)
By halfway through the Ecma period, the spec had doubled in size with extra material from its original submission of about 2000 pages from Microsoft. In the subsequent six months it increased by the same amount. So much for the idea that Ecma TC45 merely rubberstamped the original submission from Microsoft.
The comment about the horribleness of XML Schema fragments is still one I’d make. The BRM at least made them non-normative, but it did not agree to remove them entirely. I expect when people see the new generation of multi-format standards that some SC34 people are championing, where you can turn on and off normative sections, we can see the end of this clutter at reader request, which is perhaps the sweet spot.
The comment about Active X of course later became a mantra, with various demands that either DIS 29500 should have no normative reference to proprietary binaries or that it should more (to bring them under the OSP). But it was an important issue that was addressed during the BRM and can benefit from continuing vigilance. The idea of gathering legacy proprietary elements into some kind of appendix is exactly what happened, at least for the compatibility elements, at the BRM. (I don’t know that many of the participants at the BRM would have been comfortable with namespace-based notions of conformance, I didn’t get the impression that using namespaces or schemas as tools was on many delegate’s radars, no disrespect intended.)
The extensibility strategy came out as a separate part, with no significant trouble as a technology. Though some people have subsequently discovered that extensibility and “openness” (meaning guaranteed receipt) do conflict: this is something I have repeated talked about: the need for profiles. On the general subject of extensibility and interoperability, Joel Spolsky has another good article this week: Martian Headsets
On the technical merits, well actually I dont know if they matter much. I say potato. Exporting to HTML or XHTML gives people base-level interoperability for most documents, which neither ODF nor Open XML will challenge; at the high end the solution is exporting to XML using a domain-specific schema (e.g. S1000D for military & aerospace) and not ODF or Open XML at all; in the casual middle we will have ISO ODF available, perhaps as the interchange format of choice, as well as ISO Open XML (if it is accepted) for when you need to track MS Offices capabilities closely. I think there is substantial value in a standard XML format for MS Office documents even within organizations that will mandate ODF for interchange and archiving. The availability of the alternatives reduces the need for ODF or Open XML to be the one true interchange format.
I think I still agree with everything there. (By technical merits, my point is not to do with the state of the draft, but about doctrinaire views on optimal technology which are ultimately subjective, and the benefits of plurality.)
I still think ODF is the appropriate format of choice for level-playing-field document interchange, especially for governments, though it seem ODF 1.2 and 2009 are the more realistic time-frames for this. And Don’t forget about HTML!
Probably coming from the industrial publishing background biases me here: the need for dumbed down interchange formats is real sure enough, but the need for intricate close-to-the-metal feature-exposing typesetting feature access is also important for different contexts. Word’s binary formats and RTF’s weaknesses have long held Microsoft’s applications back from being happily usable in serious industrial publishing systems (or, at least, have often held back the people who adopted them.)
+1
Three programmers gathered at the next cubicle to mine yesterday, clucking and snorting as is their want. I looked over to ask what was going on. “A bug in Java” they said. The problem was with ZIP files, specifically some differences between ZIP files made by different methods.
They had some files with non-breaking spaces (U+00A0) in the file name. Not something that I would do myself, but the number of people who want to use non-ASCII characters in their filenames is surely now much greater than the number of people just content with ASCII-only names. Aha, so file this under internationalization (I18n)!
The problem was, it seems, that WinZIP stored the filenames using the system default encoding. But Java would read the filename using UTF-8. So sometimes ZIP files parts would have the non-breaking space, and other times the same file saved a different route would have 0xFF at that position. Now this is the kind of behaviour and problem that you would expect a decade ago, but I was surprised it still occurred.
Checking through Sun’s bug database, we find that this bug (or its clone) is actually the second most requested (2008-13-28). The engineer who evaluates the problem gives the excuse that Sun decided to use UTF-8 for JAR files (which use ZIP) and seems a little surprised to discover that ZIP may actually be created by other systems to.
Looking at the bug report, we also find it was first reported 07-JUN-1999. Almost nine years ago. The bug report says it is only reported up to Java 1.4.2, however I cannot see anything in Java 1.6 that addresses it.
So what has happened? Several things:
So for almost 10 years the Java version of ZIP has been broken for internationalization purposes, the fix seems to be caught in limbo (are they waiting for non-UTF-8 encodings to go away, perhaps?) , and so people are forced to go to other implementations. WORA undermined! Indeed, this seems another example where Java is simply too large for Sun to maintain adequately.
But what about this angle: the current ZIP spec has an appendix on file names and encoding it says
The ZIP format has historically supported only the original IBM PC character
encoding set, commonly referred to as IBM Code Page 437.
Which means that Sun’s policy of merely writing UTF-8 is now going against what the ZIP spec says.
Software maintenance and juggling issues on a budget are not easy. However I think it is more than plausible that had Sun gone ahead and submitted Java to ISO for standardization a decade ago, this issue would have been fixed long ago. Because ISO National Bodies give very high precedence to issues such as internationalization, accessibility, modularity, and conformance. So the lack of proper encoding support in the ZipEntry API would undoubtedly have come to the fore in the very first round: Japan never lets this kind of thing slip, for example.
By exactly the same token, if the ZIP format has been put through as a standard, proper encoding support would have undoubtedly been raised as part of the first review. Standardizing either would have been good enough to have a technical fix agreed on, published and pressure applied for a fix ahead of the demands of corporate featuritus. But standardizing both would still be best.
After Sun backed off last time, leaving so many people who had participated feeling burnt, it is hard to see that standards people won’t be deeply suspicious of them. And Sun people may not be keen to submit even to a “bullshit process” based on pragmatism and incrementalism. But Java would clearly, IMHO, be in a much better position today if it had been standardized. And so would ZIP.
What standardization of a living technology gives stakeholder companies is more than just bragging rights and ammunition to shoot their rivals with and to confuse procurement people with, tempting as those things may be, it also give an objective audit program dictated not from the corporate POV but from (to a greater or lesser extent, depending on interest) the market and relatively disinterested third parties. Any long-term software project gets encrusted in the personal politics and ideosyncrasies of the development team, and needs a circuit-breaker. This is a view of standardization as a kind of major technical audit, particularly of the documentation but also of areas that are becoming more market-critical: standards use and compliance, openness, responsiveness, accessibility, internationalization, integratability, testability, and so on.
These are all things that established technologies need. Now of course you can get audits in each of these areas by hiring experts. That is good, but you don’t get the breadth or provable transparency that National Body participation can bring. And expert opinions still have to get evaluating the context of the power relationships of the company, the very same relationships that allowed the problem to arise (these might be as simple as CJK requirements not having an adequate champion or I18n not being a profit center that can demand changes.) And you can get benefits from using boutique standards bodies in which vendors or their representatives can have voting rights: W3C, Ecma, OASIS, and so on. That is good too, but it does open to domination by one side or the other.
Which leaves the ISO family (e.g. ISO/IEC JTC1) as being effective forums for this kind of audit. People who think that ISO standardization is always a pushover should consider the current OOXML debate: you have MS and friends on one hand and IBM and friends on the other both pushing as hard as they can, and yet as I write neither can establish clear dominance. And these are the largest players in the world. Whether DIS 29500 mark II passes or fails it will be because national bodies decided on technical issues, not pack alliances, as far as I can tell. I am sure that neither MS nor IBM is feeling comfortable at the moment: and this is the strength of the ISO kind of procedure, regardless of the outcome.
We have all had enough experience of open source to be aware of its strengths and weaknesses now. Making something open source does not automatically mean that bugs and so on will be fixed. No silver bullet. As I wrote in this blog a couple of years ago in Sun should open source Swing
it is not enough to Open Source something: the mechanism for speedy response to bug fixes and releases is crucial too.
And neither will auditing a technology by making it a standard. Nothing is automatic. But Error-full systems emerge from single-strategy maintenance regimes and the dinosaur systems such as Java and Office are full of examples of this. The ISO standardization process has many qualities to commend itself for large companies as a tool for shaking things up and circuit-breaking. And we still need an ISO standard for ZIP too.
Placeholder for ongoing notes from the Microsoft Technology Summit…
—
I have been pretty disappointed in the new operating system distros I have been trying out recently. In the last three to six months there has been:
So instead I have installed a recent Solaris Developer, from the DVD of some Linux magazine in the newsagent.
This is one of the easiest installs I have had. (I could only install onto a partition on the main disk, install onto a partition on the secondary disk failed with a bogus message about user accounts. No biggie.)
The system boots up, SAMBA works fine and detects most things I want to detect. (It is interesting that it only detected one printer on our network, however Vista only detects the other one, so that is not so bad.) It has Firefox and Thunderbird, which are what I’d use anywhere, and StarOffice, which is good enough for now; I cannot really use it or OpenOffice for making presentations until Impress gets tables in v3.0. It comes with Java installed, and Netbeans, though I’ll be downloading Eclipse for compatibility with the workgroup here.
The desktop is a nice GNOME and really uncluttered and to the point.
Best of all, it feels like UNIX. Not a half-assed wannabee, or a messy child’s toyroom, the way some Linux distros seem to be. But lots of GNU goodness. I still have to see how it copes with some issues like updates (which was the only real flaw I found in Mandrake Linux, that I was happy with for a few years.)
So I really like Solaris. It seems to suit what I want and expect better than any other OS distro I have come across yet.
But it has one big problem: the screen graphics are super ugly. In fact, so repellent as to make it unusable. I have a 1440×900 LCD monitor and this is not one of the built-in types supported. No problem, I thought, I’ll just change the appropriate xorg.conf (or whatever is the equivalent) file. But I cannot see how to do it: it looks like it is hardcoded or something. So I have some other resolution, with a half inch dangling above the screen and unreachable. And the fonts are ugly and thick: even when I turn on anti-aliasing and play with the LCD settings it makes little real difference. Unless some kind reader can make a good suggestion, it just doesn’t compare to what I have been used to under Windows, Mac or even Linuxes.
I really hope I am doing something wrong, because apart from that Solaris really seems to fit the bill for me. Maybe I have to resurrect the old CRT monitor.
[Further Adventures] I tried to install a different card, only to have a hardware problem, so I switched back to the original new card. Oops, now the thing doesn’t boot. Checking though, for some reason the BIOS had switched around which of the two hard disks to boot from. I don’t understand how this could have happened. In fact, I don’t understand why it can happen either, because I thought both hard disks would be checked for booting in any case. But swapping the order of booting from disks fixes the boot problem, and I am online again.
I looked through the X windows logs, and sure enough the VESA driver only has a limited number of screen resolutions available, and 1440×900 is not one of them. Sigh… So to use Solaris I have to either go down in resolution to fit the monitors we have, or buy in a new monitor. I was finding the wider screen really useful for Eclipse, so I guess I will have to search for something else. All this is taking a frigging long time: I expect to live for three years on a single installation, so having to go through four or five large an problematic installs is wearing me down.
Readers may care to amuse themselves with this double think: Arnaud Le Hors Clarifiation about ASF and OOXML in which he says
In case anybody misunderstood my blog entry “Let’s be clear: The Apache Software Foundation does NOT support OOXML“, I did not mean to imply that the ASF has any official position one way or another regarding OOXML.
Err, except that the title of the blog was ASF does NOT support OOXML!
even stranger, le Hors is responding to a claim that he made up himself
OK, I’ll admit that nobody has claimed otherwise. Yet. But in these days and age you are never too prudent. It wouldn’t surprise me to see this or other similar fancy claim being published eventually.
So le Hors makes up a claim (that someone is saying ASF officially supports OOXML), then decries it (that ASF does not support it), then is forced to retract the decrial (that ASF has no position), then claims that that he never meant to imply what he had said in plain words in his own headline! O what a tangled web we weave! But quite a funny example of the mentality that seems to have possessed some people: truthy rather than facty.
I only read Arnaud’s blog because I got a mention. He repeats Groklaw’s decade old story about MS secret dirty tricks to maintain control of its proprietary binary standards as de facto standards, and somehow vaguely tieing me into it: daft given that I have been so open and my concern is with helping (force) MS out of its market-dominating proprietary standards. He also mentions Patrick Durusau’s change of heart I see, and repeats the IBM mantra handed down in IBM VP’s Sutor’s Critical questions that dissenters should expect their reputations to be at stake.
As part of IBM’s commitment to intimidation, le Hors reaches a new low, which kind of offsets the chuckles I mentioned at the top. Speaking in the context of me and other experts who dare dissent
the consequences when being caught to have failed to disclose any relevant affiliation could be far greater than they currently are. I’m not excluding judicial prosecution here.
Where are their heads at? People who wonder why I spend so much of time on OOXML issues in this blog, which is time spent at my own cost (and it is a real cost: I could be doing paid work instead of writing this) should recognize that it is largely in response to this kind of bullying and intimidation, that I saw glimpses of early on as the suits started to invade ISO in 2006.
Jeff Barr’s Blog � Save the Rovers!
Apparently NASA plans to cut one of the two Mars Rovers from it’s budget. But Jeff Barr has a plan to help save it…
The second plan is a bit different. What if we just raise the freaking 4 million bucks ourselves and simply give it to the team? What would happen if we showed up at NASA HQ with a big plywood check (backed up by real money of course)? Could they take it, and would this make a difference? Does anyone know?
I just registered savetherovers.com and would be happy to use it as part of an organized effort to do something remarkable. I would also be willing to pitch in $100 as my part of the $4 million. That means we need just $3,999,900 more.
I’m in Jeff! Anyone else care to join the effort? I’ll contact Jeff and see how he plans to facilitate the volunteer/donation effort and report back the result…
Stay tuned!
Patrick Durusau has a few more items on his website. Always worth a read for anyone interested in getting more than the party lines. Here is some of his latest TOC:
I was told recently that of the 250 or so fast-tracked standards that Ecma has successfully had accepted by National Bodies at ISO/IEC, only three of them have failed. I thought it would be interesting to read up a little more on them.
Ecma makes standards on a wide variety of subjects, and has particularly strong involvement with the European and Japanese computer hardware industry. In a response to a comment on another item, I posted this list, which is of the current groups and chairman’s affiliations, to give an idea of its scope:
Now I knew that the C++/CLI effort had failed (for what seems good reasons to me.) But I was not so sure of other efforts.
I found this article, from 10 years ago: Sun Uses ECMA as Path to ISO Java Standardization which I will look at in more detail in a moment. But there is an interesting passage halfway down the page:
In 1996 Microsoft Corp was able to shoot down another ECMA standard, the Public Windows Initiative, at this stage, thus preventing it from becoming an ISO standard. The PWI was a Sun effort to get Windows APIs put into the public domain. … Microsoft was able to mount a successful campaign against PWI at ISO on this issue.
What do we learn from that? That Ecma was happy to serve as a neutral forum. That Sun was happy to try to make use of the Fast-Track procedure when it suited them, for competitive reasons. That in fact IP buy-in from the critical stakeholder is necessary. And that MS has made a 179 degree turn on standards since a decade ago. (I am always amused at how often anti-OOXML material will, when it fails in a current objection, resort to decade-old material as if it were fresh and compelling. The company then was fleeing standardization; now they are participating and allowing significant changes. You do not have to trust or like them to acknowledge that.)
ISO standards are a very scary proposition for large companies. Many of them are not comfortable with any position other than dominance and stability. The control of the API is terribly important to them, and they regard loss of control of the API as a risk (whereas it can be a circuit-breaker and new-market enabler.) This is one reason why all the large companies try to favour the member-based boutique standards bodies: W3C, OASIS, Ecma, because there is more chance that they can establish a beachhead and make participation at those bodies unattractive or futile for their competitors. The need for stability is sometimes stronger than the need for dominance: when you see calls for “equilibrium” to be maintained in a market, you know that is a buzzword for maintaining the status quo. (And it is not always the market leader: it can be a smaller player in fear of losing their share just as much.)
It goes in cycles. The wheel turns and sooner or later the big companies are forced to deal with ISO and national bodies, and they find this lack of control very unpleasant. Sooner or later they find some reason to split back to more dominatable bodies, and they jump ship.
It is not all venal (or even venial) or negative though: for example, look at SGML: Sun’s Jon Bosak (and many others) were unhappy with the way and speed that SGML maintenance was proceeding and we went to W3C as a forum for making a simple profile and addressing a lot of peripheral issues, and XML in turn became the foundation for the update of SGML. There is always an interplay between what the boutique, specialist bodies are interested in, and what the national-body-based regimes such as ISO are interested in: industry activity is actually really important, because it clarifies what the ISO groups should be doing.
The downside is that when these large, usually-US-based multinationals hop over to their boutique bodies, they have to try to justify their jump by slagging off at ISO/IEC. This is a predictable behaviour: it has happened in the past, it is happening now, and it will happen in the future. Some parts of the complaints are often reasonable, some parts are often merely self-serving, but it is not a new behaviour.
Now back to Java. Originally Sun put up Java to become an ISO standard using the PAS process (the fast-track process that ODF used) using the Open Management Group (another boutique group) as the submitter. Then Sun changed its mind and decided to submit it to become an Ecma standard (and thence to ISO on Fast-Track) because
In examining our standardization options, our primary goal always has been to preserve the industry’s substantial investment in evolving and using the Java technology,” said Dr. Baratz. “By paring the collaborative Java Community Process with ECMA’s proven standards process, we can achieve international standardization while preserving rapid innovation and cross-platform compatibility.
According to this article Sun chose to go with Ecma, because it was flexible enough to allow maintenance to continue on through the Java community process as it stood then. Other articles suggest that one reason for Sun’s reluctance to be involved at ISO was their strong desire to keep effective control. One particularly interesting aspect of the article is that it mentions the potential danger from Sun’s point of view of HP, Microsoft and so on doing exactly what Sun had attempted to do with PWI: make up their own version of the standard and submit it to ISO!
Of course, what Sun was concerned about was Microsoft’s attempts to destroy Java’s Write-Once, Run Anywhere promise by grafting on their own graphics primitives into J++ and splitting the market. This is of course how IBM put a nail in Java’s coffin for the desktop, by doing exactly the same thing with their SWT graphics library, as used in Eclipse: it is not a part of standard Java and Java applications that use it are not WORA applications.
The fight between Sun, IBM and Microsoft over their effective graphics libraries shows a couple of things that are very instructive. For a start, it shows that they all try to use standards for their own competitive purposes. It is no news: the challenge it to try to use the standards process to channel them into behaviours that benefit society and the market.
It also shows the futility of non-layered standards. The WORA spiel is really compelling, and it is something that I bought into with my company Topologi, but all systems that have to grow need to support what I call Organic Plurality. Systems with modularity in the wrong spots die but can cause problems in their death throws: it seems that with Java, the graphics interface was exactly such a spot, unfortunately for the vision. (For another aspect of this, see The Software World of 2010: Its about the Suite.)
But thirdly it shows that the big players have been involved in these kinds of standards games for years. For a while, and under the noxious impact of the MPEG group, the large companies got excited by the idea that they could use standards bodies to become revenue-generators by standardizing on Royalty-bearing technologies.
In the middle part of this decade, there were attempts at OASIS for this, and many of us spoke out against the large companies trying to do this, and we were successful. For people with short memories, the background of this was the attempts to get non RAND-z technologies adopted for DRM proposals: the major pigs with their snouts in the trough at that time were ContentGuard (ex Xerox), Microsoft and IBM, all the usual suspects. (Readers may also be interested to note that Patrick Durusau got involved in the OASIS DRM effort, on the side of the angels: he has a very hard-headed attitude to all the large companies, and not one that endeared him to Microsoft or IBM.) By 2004, the OASIS DRM group wound up without getting this endorsement for the non-RAND-z technologies. RAND-z won!
David Berlind has quite a good article on why a non-RAND-z standards organization is a “patent shelter” and not open: it is great that OASIS has straightened up here, and I hope SC34 continues its long-standing RAND-z policy. But it is especially great that companies like Microsoft, IBM and Sun, which a few short years ago were all excessively concerned with trying to keep control and use standards as patent-shelters are behaving well now. However, just because Microsoft, IBM and Sun have little credibility in the world of standards for altruism’s sake, it does not mean that they should be blocked from participating legitimately in standards. To the contrary, we need to have institutions to allow these behemoths to act as good citizens: RAND-z standardization is a great vehicle for a behemoth!
Back to our Java story. In late 1997, SC32’s Java study group had recommended that Sun should submit Java through the “more traditional” processes. Sun eventually did shift to use the Ecma route, but apparantly out of fears it would lose control. Then
.In another effort to block other companies and interests from developing Java platforms that do not meet its strict guidelines, Sun Microsystems on March 1, 2000, declined an offer from ECMA to standardise Java. ECMA, which is a standards organisation in Geneva, Switzerland, denounced Sun because the company refused the standardisation proposal. TechRepublic
Industry gossip was that Sun wanted to make their source code a normative part of the standard and they withdrew when they found it would not be possible through Ecma (or ISO or anywhere!): nice try fellows! I’d love to get some confirmation or another angle on this. But clearly the issue is one of control: integrity, interoperability are all nice side-effects. The trains always ran on time under Mussolini: we should not pretend that centralized control and monocultures do not have some benefits.
However, when we look at the way large companies act with respect to standards bodies, one very large question should arise: it is a variant on Adam Smith’s aphorism (or was it G.B. Shaw) that every profession turns into a conspiracy against the public interest. If monopolistic, cartels and collusive behaviour are undesirable (I don’t use “wrong” here because it carries a moral implication which distracts people from the point and lets them drink from the waters of Lethe from the sweet cup of self-righteousness) because they result in sub-optimal market operation.
So why are standards allowed: surely they are collusive, and interfere with the market?
The traditional answer is that public policy encourages standards because and as far as they create markets. When the Torx screwdriver company got its hexagonal screwdriver heads adopted as a standard, they may have been wanting to encourage a market in screws not competitors in screwdrivers, but they were creating a market none-the-less. OASIS lawyer Andy Updegrove, who I criticize a lot for his flakey reporting and bias, has really good legal material at his website which quotes the (U.S.) Fifth Circuit Court of Appeals decision in Consolidated Metal Products v. American Petroleum Institute in 1988:
A trade association by its nature involves collective action by competitors. Nonetheless, a trade association is not by its nature a “walking conspiracy”, its every denial of some benefit amounting to an unreasonable restraint of trade. In particular, it has long been recognized that the establishment and monitoring of trade standards is a legitimate and beneficial function of trade associations.
One key aspect of the setting of standards is that they cannot be needlessly exclusionary: this is why there is always the need for multiple boutique bodies, because when a company is unable to get satisfactory inclusion of its technologies or requirements because existing members have “stacked” the process against it (and it should be noted that this is a negative stacking aimed at blocking: there seems to be no such thing as stacking a standards body in favour of a legitimate technology, quite the reverse: a standards body is there to foster agreements) then that company can go elsewhere. The need for a market in standard technologies requires a slew of supporting markets, including a competitive market for member-based standards organizations. (It’s turtles all the way down, as the joke says!)
When we get to ISO/IEC JTC1 we run out of competitive standards bodies. At the international level, there is quite a clear difference between the kinds of work that, for example, IEEE takes on and the work that ISO takes on. So if allowing plurality rather than blocking is at the very core for justifying standards (I mean voluntary technical standards used by industry, not regulations or which side of the road to drive on) as market-creators and preventing standards from being feet-in-the-door for cartels, what happens at the apex, at ISO/IEC JTC1 for example, when there are no competitor bodies?
The answer is simple: plurality. ISO/IEC cannot be in the business of allowing cartelization, since the only justification for standards is because they actually prevent cartelization by creating markets.
From this light, I hope my support for OOXML getting standardized even though I recommend ODF for public government documents, becomes clearer. The need to support plurality goes to the very heart of the mission of international standards bodies. It is one thing to speak of technical issues, it is another to blanket state “We already have a standard that is good enough for us, therefore you don’t need the standard that you think would meet your needs”. Because that is just code for “We want to prevent your technology for operating in its market by limiting the market to our favoured technology”. That kind of blocking behaviour needs to be exposed and rejected.
The large US multinationals have always been trying to use standards bodies to compete, and they have always shopped around, and none of them like giving up control. The recent defection of some of the leading lights of the Open Document Foundation away from ODF springs out of exactly this issue: the charge that Sun has tried to keep too much control. They all try to play this game, it is not new.
So what can we do? We have to be like bear trappers. The bear is bigger than us, has an off-putting odour, and a taste for honey. But when the bear wanders into a cage, you don’t say “Oh, Mr Bear, you are too big” or “Oh, Mr Bear, you stink” or “Oh, Mr Bear, all you want is to raid the honeypot, such a naughty and greedy animal does not deserve to be trapped!” You close the trapdoor and jubilate. The history of these large companies is that they all try to find the route where they can maintain the maximum control, and very often they will get skittlish at the amount of control they have to give up. Even Ecma, which is polloried at the moment for being some kind of a rubber-stamp, would have required giving up too much control for Sun with their Java effort: and you would not want to think that Ecma were necessarily the most accomodating here.
A lot of the anti-OOXML material over the last year has been along the lines “Don’t you know how bad MS is” spouted by companies who have been playing exactly the same kinds of games. Think SWT, think DRM, and so on. But standardization can be a real game changer: one of the few game-changers on the horizon. The chance to capture a large mass technology into the review and influence of the international standards organizations comes very rarely and IMHO is not a chance that should be squandered on petty ideological or competitive points. Open Source millionaires and closed source search engine companies, all of them are in the same boat as the rival office suite developers: competitors with vested interests to block the development of multiple markets.
The thing is that competition between these kind of standards is not just good, it is essential. I have just been looking at the new feature list for OpenOffice 3.0, due mid year, and it finally includes tables in Spreadsheets. Now it has been incredible to me that this has not been there before: I don’t know how you can make a presentation without tables. But tables in spreadsheets was not something encouraged by ODF before OOXML came on the scene. (It is not a feature suggested for spreadsheet applications in the informative feature table in ISO ODF, in particular.) And the recent changes in OOXML have surely occured in part to catch up with ODF: it is not one sided. The competition is forcing each technology to be improved in places that their original champions did not consider important.
Given the utterly toxic relations between the various players at the moment, which makes any talk of sitting down at the same standards body ludicrous, what we need is frog race. Rival technologies whose stakeholders are attempting to leapfrog each other, but with each jump taking them closer to the goals we have set: open standards, with better QA, harmonized and mappable where possible, supporting plurality, extension and adequate profiles, with decent validation and test suites. The anti-OOXML side tries to claim that the best way to openness it through enforcing a monoculture, but the experience of the last two years, and the substantial improvements in the ODF and OOXML technologies that have occurred and are pending are clear indications that standards need to harness the competitive energies of the stakeholders rather than dissipate them in prolonged committee-room chicanery aimed at maintaining the current “equilibrium”.
if you tweet and nobody is following you, did you tweet?
I have recently accepted the position as Site Editor for the XML.com site, becoming responsible for the content appearing throughout the site as well as helping to guide functionality and look and feel for this particular portion (and to a certain extent the other sites in the O’Reilly Network). Having contributed to xml.com for several years, I feel honored to get a chance now to steer the editorial direction of the site, but I also need help doing it.
What I’m looking for right now, more than anything, are bloggers interested and passionate about XML and who would like the forum of XML.com to share these ideas. Given the breadth of the XML field at this point, what I’m looking for in terms of skills or expertise is equally broad; specialists (and generalists) in:
These are currently unpaid positions, though we’re working on plans to change that, but the site is widely recognized as being one of the pre-eminent authorities on XML technologies on the web, and we hope to provide as much editorial freedom as possible to all of our bloggers.
So if you are interested in writing a regular blog on the hottest trends in XML, give me a shout at kurt@oreilly.com with what you’d like to do and, if you have any, some samples of writings on the web.
IBM Vice President Bob Sutor is continuing the campaign against DIS29500 mark II on his blog entry Critical questions for national bodies considering OOXML/DIS 29500. I have tried to make this blog about his questions, and not in any way about him, though it is difficult to write in a way that keeps this distinction as sharp on the page as it is in my mind; nevertheless towards the end I found myself struggling not to spit the dummy.
Dr Sutor’s questions revolve around two premises that he wants us to buy into. The primary premise can be characterized as this: it is better to have no standard than an imperfect standard; unless DIS 29500 has no problems at all you should vote NO or ABSTAIN That this is rubbish should be obvious to everyone: there are always trade-offs and room for improvement. His secondary premise is along the lines of If you support it, you should prove why your support is not suspect. To say this is nasty crap is to denigrate nasty crap.
Lets make it straightforward. Here are each of Dr Sutor’s questions, with my comments.
* Was this specification appropriate for the Fast Track process? If not, it should not be approved in such a process and you should ABSTAIN or vote NO.
No Bob. The question is “Is this revised draft good enough and useful enough to be accepted as an International standard, in the reasonable expectation that there will be a good, agile and aggressive maintenance program and in the knowledge that there is work currently being performed on harmonization and testing?”
The issue of the procedure used is irrelevant to the desirability and usefulness of the final draft standard. You can take up the issue of Fast-Track with JTC1. You can get your National Body to lobby JTC1 to not allow standards of more than 100 page to be fast-tracked, or some other concrete proposal.
Think about it for a second! There has been an enormous amount of international scrutiny of the OOXML draft, far more than you would expect in a slow-tracked standard. You can look through an OOXML file and see by even the most trivial inspection that the standard does provide the lion’s share of documentation on the XML and notations in the OOXML file; and you can go through the Editor’s comments as accepted at the BRM and see that the lion’s share of National Body comments have been taken seriously. Objectively, this standard has been really well reviewed, and the process has not prevented this review. NASA sent a man to the moon in about 8 years, for goodness sake.
Saying that there is a lot more work to be done is saying nothing: there is always more work to be done, and the sooner OOXML is a standard and under maintenance, the sooner these things can be addressed. The BRM addressed the big picture issues that need to be right before a technology is on the books: the organization, the conformance classes, the normative status of schemas, the conformance language like should/shall, and the need to have a clear slots for handling legacy/compatibility/deprecated kinds of issues going forward. The big picture is completely good enough for DIS 29500 mark II to be an ISO standard. The BRM dealt with small-picture issues, such as typos and wrong examples and a lot of word-smithing issues that are also needed for a draft to be good enough quality as a standard. And the BRM also dealt with some key internationalization, accessibility and modularization issues that are also a reasonable bottom line for a standard.
Which leaves a myriad of middle-level issues. A number (hundreds!) have been identified and resolved through the BRM. I am sure that many of these issues require more work, which is why moving on to maintenance rather than farting around in draft-standard limbo is the best way to go.
* At each stage of this process, was sufficient time allowed to develop contradictions, completely review the specification in its entirety, generate all appropriate comments, review all proposed resolutions completely and explicitly, and fully review the updated document? If not, you should ABSTAIN or vote NO.
Yesterday I quoted Jim Melton, the editor of SQL, on the XML-DEV mailing list, and I will put the same quote, because unless people realize that the vote to become a standard is the start of getting a standard perfect, not the end they will be sitting targets for FUD: in particular the kind of disinformation that says “This is our only chance, it has to be perfect.” Here is what Jim Melton said:
Or perhaps most people were somewhat intimidated by the prospect of (thoroughly) reviewing a 6,000 page document. To put this in perspective for those who know SQL’s size and complexity, the sum of all nine parts of SQL is about 3950 pages. A ballot on SQL frequently receives several thousand comments, and we’ve been balloting versions of SQL for 20 years!
In fact, virtually every large spec I’ve ever had the “pleasure” to review leads to “thread-pulling”, in which every page yields at least “one more” bug, and following up on that one leads to more, and following up on those leads to still more, etc. I would personally be stunned if 30 dedicated, knowledgeable reviewers of a 6,000 page spec on its first public review were unable to find at least 3,000 unique significant problems and at least 40,000 minor and editorial problems. But that’s just me…
No Bob. The question is “At each stage of the process, was this draft treated fairly and any differently from other ISO standards?
And another question is “How can I encourage my National Body to participate in the continued improvement of this standard (and ODF!) at ISO/IEC JTC1 SC34? It is very important and worthwhile, and it is so great that this we can proceed steadily on with no artificial deadlines using maintenance.”
* Have all your comments been fully and correctly addressed? Are the changes reflected correctly everywhere necessary in the specification? Have you verified this? If not, you should ABSTAIN or vote NO.
No Bob. The question that should be asked is “Have enough of your showstopping issues been approved now to the necessary extent that the advantages of having a standard outweigh the disadvantages.?” It is a balance, a tradeoff. And “Have the championing stakeholders (ECMA and Microsoft) demonstrated a preparedness to engage and make improvements?” for which the answer is objectively yes, based on the BRM at least.
* Is this high quality technology? If not, you should ABSTAIN or vote NO.
No Bob. I’ll visit the quality issue later. But the question is “Does this standard fairly attempt to meet its scope statements?” The purpose of DIS29500 is different to the purpose of IS26300 for example: the underlying technology is a fact at loose in the world: hundreds of millions of people use Excel and Word and Powerpoint and the scope of the standard is not to say “This is the best approach in the world” in some abstract way, or “This is the best thing for data interchange” but “This is the information that these applications produce and consume.” Bob wants the data formats that millions of people use to be kept proprietary, to be caught up in some limbo of committee work and red tape: by his actions, he wants to opposite of openness.
* Can you can say that you completely understand the specification that emerged from the Ballot Resolution Meeting (BRM), with all its changes, and that it is now a very high quality specification? If not, you should ABSTAIN or vote NO.
No Bob. The question is “Does your national body believe that it would be better to have a standard using DIS29500 mark II than to not have a standard?”
Now I certainly agree that National Bodies should give a reasonable amount of technical scrutiny to any standard they vote on. But if you don’t have the technical ability to do a reasonable amount of scrutiny, then you ABSTAIN. And voting NO is completely the wrong vote: if you are ignorant, you don’t impose your view on other nations, you keep quiet. And you never, ever, ever try to block something merely because it does not interest you: you respect other National Bodies enough that if they want something but you don’t, they can have it. That is the way that consensus works at ISO.
I’ll deal with quality below.
* Are you fully confident that no additional problems were introduced at the BRM that your national body would insist must be addressed? If not, you should ABSTAIN or vote NO.
No Bob. The question is “Are you aware that your National Body can notify ITTF and SC34 of any perceived editing SNAFUs and that they will be discussed and, if true, fixed?”
The FUD over all these additional errors that will be introduced is premised on their being no procedures or maintenance to fix them. But it falls down, because in almost all cases the texts that are being introduced in the new draft are completely spelled out. For most of the Editor’s dispositions, people have had months to think about the text; indeed in many cases the Editor’s comments are just direct applications of the wording that the National Bodies originally suggested.
One of the big unknowns is the move to ISO-ese “should/shall” conformance language: however, since this move in no case changes the normative impact of a sentence (though in a few cases it might make the normative impact of a requirement more stark, it is to be hoped) it should not be a cause for any freak out. But it should be pointed out that ECMA did in fact make available at the BRM an unofficial version of the draft with all the Editor’s response changes already there: so any national body can certainly avail themselves of that.
(Now I do agree that the Fast Track procedure is sub-optimal after the BRM: a longer time limit for editors to get his work done would be reasonable so that NBs can vote on the consolidated text, and to relieve ITTF of their overseeing role. But it is not in itself a reason to vote NO or ABSTAIN or YES.)
* As an international standard, does this specification inappropriately favor a single vendor and its products? If so, you should ABSTAIN or vote NO.
No Bob. The question is “Does this specification address more than the petty competitive rivalries of large US multinational corporations and give for the first time a voice to national bodies to provide the kind of documentation and maintenance openness that a market-dominating technology should have? Does this help the users in the world who have to, for whatever institutional or client reasons, use or integrate with Office and its files?”
* Are you 100% confident that there are no intellectual property problems that would prevent anyone from fully and completely implementing everything in the OOXML specification? Do you have this assurance from experts who are not from Microsoft or in their financial ecosystem? If not, you should ABSTAIN or vote NO.
No Bob. That would require that every person who voted had a law degree: no standard is voted for on that level. The question is “Are you reasonably certain that the intellectual property issues of OOXML are no different from those of other similar standards with similar licenses for which there has been no problem”
Lets get realistic here. The people who get sued for IPR infringements on software are the people who have made big enough money to be good targets. And they hire fancy lawyers and duke it out.
The mention of “fully and completely” implement is the giveaway here. Bob’s comment is not interested in end users, sitting there creating Excel formulas: these number in their 10s of millions and have no voice and absolutely no chance of suffering IP attacks: they are consumers. And Bob is not interested in system integrators, who are faced with integrating into existing and future Office sites just as a fact of life. No, Bob’s comment is interested in making IBM’s niche requirements into something that should dominate other considerations. If Bob were remotely serious about this, why hasn’t he raised exactly the same issues about ODF? And IBM’s license is patently similar to Microsoft’s: it is a bravado performance to criticize something as inadequate that is almost identical to your own company’s license!
* Has the process as you have seen it been without undue and inappropriate influence by the supporters of OOXML? If not, you should ABSTAIN or vote NO.
No Bob. The question is “Has the process resulted in a standard that is good enough to start and get improved by maintenance at ISO? Has the normal ISO processes resulted in the various sides making their cases, problems being looked at, and serious attempts been made to resolve issues? ”
I don’t think people are fools Bob. They can see that the readiness of the NOOOXML people (who Bob is happy to share a stage with and happy for his staff to link to) to label any setback for their cause, no matter how trivial, as a de facto sign of undue influence, process iregularity, corruption and bribery, makes these claims completely without credibility. Where is the smoking gun Bob? When you look at these claims, what do you find: ooh some business who would benefit by having the standard goes to a standards meeting and says “This would be useful for us” and that is a sign of corruption?
And Bob, how on earth can you talk about undue influence when you later write Are you willing to stake your professional reputation on that action? Are you really saying that people who dare to disagree with you can expect to have th
