Regularly as clockwork, every five years another group attempts to make a new standard language for typesetting. FOSI, DSSSL, XSL-FO, and ODF (plus the less grandiose scopes of CSS (styling) and OOXML (legacy).) I predict that in a decade we will see the same thing. In the past, these efforts came from the user side rather than the vendor side, and were driven by user requirements rather than vendor requirements. But requirements for standards now predominately come from questions about “Our product X supports feature Y and therefore the standard should support it” rather than “Our document A uses typesetting feature B therefore the standard should support it”: the cart is driving the horse. There is more vendor buy-in because the new standards demand and achieve so little.
In part it is understandable, the catch-up mentality does not necessarily encourage imagination.
One very common tool for organized standards groups is a feature matrix: rather than just ad hoc consideration of this feature or that feature as proposed by vendors, the idea is to make a list of the general features required by the users document sets, or by the technologies being evaluated, or the products chosen to get first-class support. Traditionally, standards groups for typesetting and publishing have included actual typesetters (at ISO, Martin Bryan actually worked in type for example.)
A really good example of this can be seen in a document from a decade ago Final DSSSL Survey and
Assessment Report for the DOD CALS IDE Project (Kidwell, Richman). This is a good introduction both to the Output Specification (FOSI) formatting language used by US military typesetting in the 1990s, and the ISO Document Style and Semantics Specification Language (DSSSL) which has been available standard on many Linux systems using James Clark’s open source JADE program.
The feature matrix can be found in Comparison Matrix which shows how well the standards support the document requirements: we learn that the US military requires both cartoons and running feet. This is the kind of table that I think should be driving requirements for ODF (and OOXML); preferable to the approach of feature- (or vendor- or product-) centric comparison matrix and much preferable to ad hoc feature requests.
FOSI and DSSSL
The US military adopted FOSI because it was under consideration by (what is now) ISO/IEC JTC1 SC34, however SC34 ultimately went with an extended version of Scheme under the (terrible) name of DSSSL; FOSI was deeply unlovable and never floundered outside its early adopters who were locked in; DSSSL was like the other power-user oriented standards from SC34 of the time and never found much commercial adoption by had uptake in the publishing industry that SC34 catered to. James Clark, the DSSSL editor, later merged it with CSS ideas and split it into XSLT and XSL-FO at W3C using an XML+XPath syntax rather than the S-expression syntax.
Where DSSSL and FOSI (MILPRF28001 Output Specification) differ in particular was that DSSSL adopted a strict transformation approach: this is of course a UNIX-ism since the days of nroff, and the idea was that you could output to particular page description languages (RTF, MIF, etc.) Consequently there was no way for the DSSSL processor to make decisions based on typesetting metrics on the fly; instead the race was on for a set of abstract properties that could describe common cases. This fits in well with the checkbox mentality of desktop publishing tools, but was entirely counter to the typesetting-as-programming approach of the 1970s and 1980s generation of systems (systems such as troff and TeX used macro facilities so that creating a typesetting system for a document could involve all sorts of custom smarts to capture the design and fit in with the data; very high-end systems such as Interleaf even had full-blown LISP available for processing: some of these systems are still around with their niches: XYwrite and 3B2 for example, however they face a rising tide where quality and power is increasingly mysterious to the market.)
FOSI did allow or require some kind of interrogation of the pages while they were being typeset: while this can certainly allow much more expert typesetting and decision-making, it also must be tightly coupled to the formatting engine, which effectively prevents any network effects.
What do I mean by expert typesetting?
To give an idea of what I mean by expert (also known as “quality” or “industrial”) typesetting and decision-making, consider the case of typesetting a Yellow Pages (phone directory for businesses categorized by type of business.) Imagine you have to produce a Yellow Pages document using your favorite tool. The page designer and sales force come up with a design and timetable. The layout will be five columns. Entries may not span pages. Some entries take up part of a column and should be put as near to alphabetical order as possible, but rather than break they can be placed before or after their alphabetical position with previous or subsequent entries swapped before them. And there may be two, three, four or five column display adds, which also have this arrangement. And there can even be adds that take a half page but span over two pages.
And it is important that ads should not be orphaned or widowed, with one ad on a previous page by itself or on a subsequent page. And there are 6,000 pages of this. And you get the final data 24 hours before you have to deliver it.
Declarative abstractions are worthy replacements for programs and scripts but have different coverage
Now the history of (SGML and) XML is the effort to key presentation cues from structural information: the benefit of marking up “invisible” containers is that they are often not invisible. The current approach of both ODF and OOXML of allowing foreign container elements (in different syntaxes) but not providing facilities to format based on them, is the worst of all words: for QUASIWYG systems users will be loathe to do anything (well) which does not have a resulting visual/stylistic result in the on-screen draft. And (as was pioneered in pre-Adobe FrameMaker and taken up in CSS) the abstraction of frames (floating or relative, linked boundaries into which text can be flowed) also provides many hooks for making declarative properties that otherwise might require programming.
The way that standards for public declarative publishing formats (whether HTML or ODF) should go, in my opinion, is by progressively asking the question, how can we make it easier for users to do what they want to do? In the old days, this was easy: you had physical paper (from mechanical typesetting, for example) or device-independent page designs (the Yellow Pages for example) and you then programmed it by inserting commands in with the text. SGML and generalized markup came along and said describe the data in markup, then move the processing out of to a presentation system, except for Processing Instructions where you need specific overrides inline still. After this re-factoring came libraries where common code or functions were provided with the base system, and then consolidation where the code for the libraries was hidden from the user, and then exposure where only programming capabilities were removed and only the declarative portions left. RTF and MIF are examples, but so are OOXML and ODF.
At this point, users of transformation systems (such as XML with XSLT) have a lot of capabilities, even for overcoming the differences between the underlying typesetting engines of systems (see Different classes of typesetting engines and Markup’s Dirty Little Secret, but they have none for the kind of page-based calculations required by the Yellow Pages.
Now you could continue to make abstractions: nested keeps with partial float for re-ordering, for example. And in the past, there was a hope we might progress there, because the driving factor for markup languages and style languages was to cope with the kinds of designs which simple word processors failed at. But as I said, the cart seems to be driving the horse: I have no objection to document formats for existing and legacy applications (nor obviously to have them as voluntary standards, readers will no be surprised to read).
Universal pretensions without an assertively inclusive process merely disenfranchizes the weak and the foreign
However, and this was something that I saw as a flaw in the XML Schemas process, the more that you claim your format as a universal format, the more that you need to cope with cases that may be “niche” to vendors (i.e. that didn’t fit in with their development or profit model) but which are significant in their own right. When a technology, standard or not, mandated or not, does not provide a capability needed for a job, it will not (because it cannot) be used.
Lets take a concrete example. In about 1999 I spent a year looking at the various requirements for Chinese and XML, at Academia Sinica in Taiwan. As part of this, I looked at how Chinese actually did typesetting before the advent of computerization. I first made a (example below) of some interesting, but not at all atypical tables, some of which have visual structures that Japanese will recognize. (In effect, when you have the equivalent to very small word size, there are other graphical possibilities that don’t go well in Western text.)
Then I made a suggested a possible structure that could be used to reconcile them. I was surprised at the reaction: Westerners universally made comments like “Oh, but those are *bad* tables and bad practise” and “They show confusion and unstructuredness”. Microsoft did add diagonal headers to Word 2000, but the SGML (and pre-SGML) idea that you should look at the artifacts and let design lead, rather than merely let vendor’s developers lead, had by the start of this decade died a rather sad death, it seemed to me.
Since then, the Chinese have gone their own way with a fork of ODF called UOF which features, as far as I can make out, Chinese element names (yah!) and extra markup for Chinese-specific requirements that other systems didn’t support. In April 2007, a request came in for
Putting page design back at the centre
The only way to put the horse back in front of the cart is to put page design (in all its detailed aspects) at the centre of the process. Get stakeholders involved who are prepared to contribute (many will have them already) the kind of checklists that the Comparison Matrix has.
However, I fear that this may only push the issue back without changing it; if the external stakeholders themselves have their opinions formed by what commercial pre-fabbed system such as Office provides. In Different classes of typesetting engines I mention how the different implementation approaches lend themselves to different declarative properties. People realize that Office has pretty minimal keep-together control, but instead merely substitute some other products capabilities. We are quite lucky that countries like China are now getting fed up with the lack of imagination and responsiveness by Western developers and standards makers: it provides one of the few chinks in the protective armour by vendors that they only want change when driven by them.
So this has been a rather codgery item, are there any good signs? Well, I have praised Office’s Smart Art before and it is exactly an example of what goes on the page driving the technology: it is not the making format into as the driver but inventing a new class of page object (just as a table is a page object). Now Smart Art actually has crappy arbitrary structure, but the direction it can take is clear, and ODF could leapfrog it, if they could be bothered. There are hundreds of thousand of pages with simple diagrams, and once you decide to support what people actually do (and look at where people find things tedious) , that is putting the documents first.
So the only drivers I see for this, again, is for large user-side organizations to participate and dominate all the standards bodies, to work out their checklists, and force through the changes to ODF/OOXML/CSS/HTML that are required to conform to how people make documents when their focus is on good or natural page design (usability) rather than on incrementalism and conservativism.