I have finally found an example of “contradiction” in two ISO standards that meets what I think ISO means by contradiiction: I have been looking for an example (and how it was dealt with) since writing my blog entry earlier in the year, What is a contradiction of an ISO standard. The example is the conflict between ISO POSIX and ISO Linux.
ISO POSIX and ISO Linux came in to ISO from external sources (IEEE/Open Group and Linux/GNU) and have considerable (or more) overlap. The Linux spec is a profile of the POSIX spec in general (and is interesting because it is one of the first signs of Open Source documentations infliltrating the ISO system, which is generally riddled with industrial, government, niche developer and academic interests.) But there are some cases where POSIX says one thing and Linux says another.
Now my definition of a contradiction was:
- One standard attempts to redefine another, or is a rival standard for exactly the same named thing but is different in some aspect.
- One standard disrupts another.
- One standard pretends to be another.
- One standard incorrectly uses another.
and I gave the Linux/POSIX difficulty as an example of a contradiction. Well, now it is six months later, lets see how ISO/IEC has dealt with the matter.
They have allowed the ISO standard for Linux as well as POSIX but they have also clearly stated where there is contradiction and gathered the information together into a technical report Technical Report on the Conflicts between the ISO/IEC 9945 (POSIX) and the Linux Standard Base (ISO/IEC 23360). So these contradictions, instead of being showstoppers, are exposed and clearly publicized as “conflicts”.
The POSIX standard remains clear. The people who want a standard for the external technology Linux get what they want. The differences become clear for software developers to workaround. And the differences can be passed on as information to the standards maintenance effort. Everybody wins.
I think this shows several things. First, that ISO is (now) geared to getting win/win solutions, and not one geared to allowing one group to stymie another (oh not this hobby horse again!) Next that the mere presence of a minor contradiction is not a showstopper, especially where there is continuing dialog between the parties: there is some aspect of proportionality. And finally that when a standard documents an external, existing technology, the standard needs to be “warts and all” and not a faked-up sugar-coated version.
Now some national bodies are apparently working assuming a much stricter definition of “contradiction”, where one standard disrupts another. And other people have a much slacker version, good luck to them. But it is interesting that no matter whose definition is adopted, the ISO Linux/ISO POSIX conflicts seems to be “contradictions”. And the ISO response? Constructive engagement to allow the development of consensus voluntary standards: this is a point I made before that ISO standards are like conversations not laws
Rick,
I agree of course one shouldn't be puritanical to the nth degree on standards, but where would we all be if two standards promoted the colours of power cables so 'Live' in one, was 'Ground' in another? Or if two railway standards existed for the track widths at the railway station in Sidney? And the point about the ECMA 376 'standard'is there are key areas on the Microsoft side where a conversation can't be had, because no-one outside Microsoft understands what's meant by the 'behave like word95...' type statements in it.
Of course though, standards need to move with the times, and that's where open conversations around open and workably documented standards will operate.
Chris
Chris: I think you need to understand how markup works. All markup does is provide enough information for software to make decisions and pull in values. Which decisions to make is the developers' decision. Open XML is a standard that describes a schema, not an application.
Ecma 376 does not even explain the *normal* typesetting that it uses, let alone how the optional hints about formatting like Word95 or WordPerfect do their typesetting. The names are enough to indicate the intended semantic. And neither does ODF. You would need an application standard to do that, not a file format standard. An application standard would provide details of algorithms and a test suite; a file format provides symbols and abstract semantics and would use schemas for validation.
Almost everything do do with typesetting is subterranean from typical XML's POV (certainly, for Open XML, ODF, DOCBOOK, HTML, etc.) An implementation is nor required to do anything with the information, and a developer is free to figure out how or if they want to handle that hint. I don't think it is a very convincing "key point".
In fact, for declarative markup, the name "FormatLikeWord95" or whatever it is shouldn't be taken as a processing instruction but as documentation: "This paragraph was originally formatted like Word95" rather than "You must format this paragraph like Word95", if you see the difference.
On the issue of track widths, of course you would be aware that there are many dual gauge and even triple gauge railways in my country. The share a common track and allow different size trains to operate without interfering with each other.
In fact, for declarative markup, the name "FormatLikeWord95" or whatever it is shouldn't be taken as a processing instruction but as documentation: "This paragraph was originally formatted like Word95" rather than "You must format this paragraph like Word95", if you see the difference.
I quote from the specification, with italic emphasis mine:
2.15.3.6 autoSpaceLikeWord95 (Emulate Word 95 Full-Width Character Spacing)
This element specifies that applications shall emulate the behavior of a previously existing word processing application (Microsoft Word 95) when determining the spacing between full-width East Asian characters in a document's content.
[Guidance: To faithfully replicate this behavior, applications must imitate the behavior of that application, which involves many possible behaviors and cannot be faithfully placed into narrative for this Office Open XML Standard. If applications wish to match this behavior, they must utilize and duplicate the output of those applications. It is recommended that applications not intentionally replicate this behavior as it was deprecated due to issues with its output, and is maintained only for compatibility with existing documents from that application. end guidance]
The words "shall" and "must" make it sound a lot like a processing instruction to me, or at the very least, a requirement. The last part of "don't intentionally replicate this behavior" seems to, ummm, contradict the "shall" and "must" (but then again, it's only guidance).
David: The words "shall" and "must" do indeed sound like a requirement if you stop reading before "It is recommended that applications not intentionally replicate this behavior".
You cannot read standards on a sentence-by-sentence level. They are written to be interpreted systematically, even badly phrased standards. So the first thing to look at is the Introduction of Part 1: this clearly say that the standard defines a schema. *Not* an application (ODF says the same).
The second thing is to look at the conformance section: this clearly says that document and application conformance are "purely syntactic", which is the corollary of merely defining a schema, and that issues of human information are not required for conformance. This severely limits how we interpret "shalls" and "musts" in the rest of the text, when we are discussing what the standard requires of us.
So I believe the problem here is in the Conformance section at 2.0 where it says "Use of the word "shall" indicates required behaviour." which is confusing. If conformance is syntactic, how can there be required behaviours?
This is inconsistent drafting, and is the weakest flaw in Ecma because it looks like a bald statement but actually it is severely limited by the rest of the conformance section. It needs to say something like "Use of the word "shall" indicates required document syntax. It also is used to indicate the typical semantics, which may be described in terms of general application behaviour."
The third thing in the conformance section is that it is entirely application dependent to choose any semantics at all.
Fourth we get to Part 4, and the introductory text of s2.15 where is says "Compatibility Settings - settings which tell applications to perform behaviors which are designed to maintain visual output of previous word processing applications. These settings are for backward compatibility and are all ignorable."
So finally we get to the text you quote itself. The "shall" means "shall" in the context that if it ain't syntactic it ain't required for conformance, and that an application doesn't need to support it anyway. (I.e., it should be "should".) And then it clearly says it is deprecated and why. And the "must" clearly indicates that there are no references to help the developer if they go on this route.
And, always, when looking at an issues, we then have to look at its significance. Japanese have half-width and full-width versions of several of their syllabaries: zenkaku katakana and so on. The Japanese are also pretty organized, in that they have long had a very nice set of general typesetting rules, the kinsoku rules, which nowadays everyone implements for CJK products: there are differences in the details of course in the Chinese and Japanese , but many of the issues are held in common. So anyone attempting to implement CJK typesetting needs to get a hold of the kinsoku rules. (And Ken Lunde's OReilly book CKVJ Information Processing if they can find it.)
Now in this particular case, I don't see why Ecma 376 couldn't have given a better "typically" comment, unless it is really complex or subtle or minor or a rare edge case, or if it requires an enormous amount of backgrounding in kinsoku and metrics etc which seems unlikely. But a small hint like as "Word95 aligns on full-width character grid" or whatever it is, would at least let implementers zero-in better. But I don't think it is remotely a problem that they have not documented it, because they give a clear warning.
From my perspective, this is probably a "would be quite useful to have" issue and certainly not a "kill the standard" issue. However my perspective is not relavant here because I am not the market: it is the people who need a standard who should get the voice, not the people who don't need it!
I will be very interested to see what the Japanese and Korean and Chinese national bodies say, though, because we can presume they have experts available who know how Word95 acted and whether it is information that should be in this standard now or not. Sometime full-width character spacing can be significant, because of the prevalence of fixed format (think ASCII art) forms in CJK bureaucratic work in the 1990s, so it may be something that their archivists do need more information on.
With any standard, you simply cannot just look at a clause and say "this is required" unless you first check the Introduction, Scope and the Conformance section. ISO standards are laid out requiring these to be put right at the start of a standard to force the issue to the top, because it is a cornerstone issues for interpreting the rest of the text. DIS 29500 follows the ISO organization here, but at least some of the parts have not been written with a strict adherence to the ISO (or common-sense) usages of "shall" and "should".
That is certainly an editorial issue, but it is also an issue that a standard is a unique kind of literature that is not a casual reference work in the sense that any clause can be understood in isolation from its context.
Rick,
The interesting article you refer to about dual-gauge railways wisely states : "The complications and difficulties outlined show how important it is to ensure that railway gauges are standardised in the first place"...
Luc: Yes. If we had a year zero for computers, and could start from scratch, it would be feasible to start with a single document format.
However, that case is like saying "if we only had 2 dimensions"...
Even in such a world, the format would have be designed very differently from either ODF or Open XML, because applications will always have different and evolving feature sets.