You may read my posts on XML.Com and believed that because I blog here I may know a thing or two about XML. Or you may think I am just a lucky idiot. The truth is a little of both!

One of the really convenient features introduced in XSLT 2.0 is Grouping. It is a typical second-generation change in a programming language: Not essential for the language itself (grouping can be done by hand using techniques such as the Muenchian Method), but required by many users and thus a useful addition to the language.

XQuery so far lacked support for grouping, with the same results as in XSLT 1.0: Grouping is cumbersome to implement and potentially slow when not implemented right. The recently published first XQuery 1.1 draft makes only few changes to XQuery 1.0, among them a Group By clause for FLWOR expressions. This should make the language quite a bit more useful, because grouping is such a frequently needed feature.

The other notable addition to XQuery 1.1 is the Window clause for FLWOR expressions. Windows come in two flavors, either tumbling or sliding windows. Windows clauses are somewhat similar to grouping, because they allow specific iterations over the binding sequence (not by grouping the items into new items, though, but by grouping them into windows, which always contain consecutive items). Windows are not that hard to understand when you look at the tumbling window examples and sliding window examples provided in the draft.

So far, XQuery 1.1 looks like a really small and useful set of changes to XQuery 1.0, which is good. Let’s just hope it does not share the fate of XSLT 1.1, which was abandoned and never really made it into implementations. Are there any other changes which should be part of XQuery 1.1? The XQuery 1.1 Requirements list quite a few, including functions for date formatting, numeric formatting, and error processing and recovery. It will be interesting to see how these requirements will be addressed in future versions of the XQuery 1.1 draft.

I wrote a recent article the other day on how Google has decided NOT to use XML for a recent project they open sourced. I received a LOT of very opinionated responses to that post. Unfortunately every one was from a complete MORON.

I just came across an article that announced Google open sourced their ‘Protocol buffers’ but decided NOT to use XML.

It is great that Google is contributing to the community and is showing the world how their system works, but I wonder if the inmates have taken over the asylum. I am a software engineer, so I enjoy the technical information, but I also ran my own company, and evolved to view problems from the business end. If you let the ‘engineers run the show’, then you get a very narrow viewpoint.

I think we have a severe ‘Not invented here’ syndrome inside Google.

My fellow blogger, M. David Peterson posted a very informative blog in relation to Jeff Atwood’s rant on XML. It is a great article, and quite a treat from Mark’s usual concise style. I have deep respect for Mark’s technical skills and he has some very valid points about the true nature of XML. If anyone has seen the ups and downs of semantic tags, it is M. There is no specific piece that I can pick apart. Unfortunately, it is also wrong. Dead wrong.

Effective participation in standards bodies involves quite specific commitment and development of expertise, it is not a generic capability that can be instantly redeployed, Rumsfield-style, to trouble spots.

For example, while knowledge of OASIS procedures may help you understand some standards admin issues, it does not give any specific knowledge that can help understanding or predicting ISO procedures. Similarly, if you are an expert in implementing a word processor, that does not give you any specific knowledge of schema languages. And being an expert in a problem domain does not give you specific knowledge on different implementation/architectural strategies for schema language standards.

So what you end up with is say 10,000 standards committees each of which can benefit from 3 or 4 different kinds of expertise. If MS has 45 staff permanently working on particular standards, they may well be well-suited for a few dozen committees each, and may be able to skill up over a year to serve on a few hundred others. But that does not mean that MS, IBM or even governments necessarily have the ability to jump into SC34 WG 1 (or whatever) and instantly be effective.

The strange attack of modesty

Several bloggers have picked up the strange comment of Microsoft’s UK National Technology Officer McKee that it had little or no experience or expertise around software standards until the company was mid-way through the process of getting Office Open XML approved by the International Organization for Standardization. Andy Updegrove has a post (which is notable for actually agreeing with a point I made) and MS’ Matusow made a response too.

Can they all be true? Can MS be inexperienced (McKee), yet also be so big and active for such a long time (Updegrove)? And in how does Matusow’s comments that, no, MS is actually big and active for a long time actually counter Updegrove’s comments?

I think the problem is of merely talking about “standards participation” as if it were a generic commodity is the problem: Please buy 100 units of standards experience now! In fact, while certainly experience in one standards body gives a better grounding in how other standards bodies might work (compared to no experience), the devil may be in the details.

Mission-critical and multitudinous

Even within an organization, participation in one technical group (e.g. an JTC1 SC) might help in participation in another group from a procedural point-of-view, because typically standards work involves diving deep into the technical issues to the extent of being able to judge and suggest alternatives and new possibilities, it is very likely that a participant in one group is not a domain expert for the new group.

Furthermore, this is all compounded when you have international standards bodies: if the largest multinationals only have enough resources to commit one or two people globally to any area in the long-term, they certainly will not have the skills on tap. This is not only a problem for MS, but for all large companies; standards participation has not been seen as a revenue generator, nor has standards skills been seen as a mission-critical asset for high-tech corporations (or governments, for that matter.) (For example, I remember asking a Google honcho a couple of years ago what their standards commitment was, and being surprised to hear they didn’t have any at that stage; individuals could participate though.)

But standards are now mission-critical.

And they are vital to competition policy: for helping form markets from inchoate messes, for helping allowing broader stakeholder participation in oligopolistic markets, and for reducing barriers in dominated markets (see my blog All interface technologies by market dominators should be QA-ed, RAND-z (voluntary) standards for more on this.)

Expertise: contribution not domination

So the real story is something like there are many of the big high-tech firms have multiple people working on multiple standards at multiple standards bodies, but none of them have the expertise on tap that they can instantly have someone up-to-speed and participating effectively on any arbitrary standards group. And this is worst for national body participation and international standards participation.

But I don’t see why this is a bad thing.

While I certainly think all stakeholders should get their act together and participate, and develop experience, the last thing that standards groups need is invasion by non-technical suits making decisions on non-technical grounds, and the second-last thing needed is participation by technical people who do not have proper commitment from their corporate masters to commit to implementing the standards, however they come out.

After a decade and a half in standards work, I know and accept that the big companies blow hot and cold on standards bodies. (This is my “trap the bear when it is in the cage” argument.) They jump ship to the bodies that will give them the best result, they try to convince people that they jumped ship because the ship was rotten and not because they were rats, and they implement a standard only as far as it fits in with their development/marketing cycle, which often means minor version changes are not implemented, or that there can be periods of stagnation. Even the largest companies find it hard to stay on standards committees when they know they have no intention to implement that standards or its updates.

Increase in idealism then pragmatism

But we are seeing a sea-change in the attitude of regulators towards standards. Aware
that so much of the technical change of the last decade has ridden on the standardized
technologies, governments and regulators are increasingly likely to want to mandate standards; however, they then immediately discover that the standards processes of
all standards bodies in the IT area are not geared towards this being justifiable against
public policy: this has been an unintended consequence of the concerns expressed by many other stakeholders that OOXML was dominated by a single vendor (it turns out that most standards were contributed by single vendors or come from committees dominated by a handful of commercial players or from organizations with direct representation by vendors.)

Sensible stakeholders are taking the pragmatic view: encourage standards adoption but not prematurely or naively, penalize non-adoption of standards, and above all, participate.

The problem I saw in the OOXML (and the ODF) standardization at JTC1 was not so much that the large corporate stakeholders (MS, IBM, Google, etc) did not have resources on tap who knew about office formats (the domain areas) *and* SC34 technologies *and* JTC1 procedure *and* ISO editing *and* national body experience, but that there were so few other stakeholders involved who did.

Certainly I think all the corporate stakeholders should be involved more. And they should have assertive representation at OASIS, W3C, Unicode consortium, SC34 and the national bodies. But all stakeholders need to identify strategic standards and make sure they have the skill sets in place to be able to participate in the boring ongoing process of maintenance, not just the headline grabbing period. One of the good results of the OOXML standardization process is that more stakeholders have put their money where their mouths were, and started to participate at OASIS on ODF, as well as on OOXML.

Participation: babies are not freeloaders, non-participants are!

One of the great disappointments of the open source movement has been the way that lazy users don’t feed changes and improvements back, but are passive recipients. And often we see open source programs reflecting the priorities of its sponsors not its users. However, the standards process (when running correctly) have procedures in place to make sure that stakeholder comments will get looked at; but just like with open source there is an enormous intertia and laziness among stakeholders to participate.

The value proposition of open source and open standards, for many organizations, is that they get something for free, but that attitude ultimately means they get something sub-optimal for free. Organizations, and governments need to consider this very strongly, who have mission critical deployments or procurement programs based on open standards or open source need to assertively, pro-actively participate in the development and maintenance efforts of those programs.

There is a great quote (I’d have to track down who from: Dan Savage?) about gay couples holding hands in public: that to some extent in order to live in the world you want to you have act as if it were there rather than waiting for it to happen outside your actions. The same is true for standards: participation is essential.

[NOTE: this was previously posted to the OReilly DEV NEWS site by mistake.]

SwiXML is Wolf Paulus’ XML User Interface languge (XUI or XUL) which uses the regularity of the Java Swing GUI libraries to allow very lightweight implementation: XML elements are used for JComponents, XML attributes are used for properties (e.g. <frame size="5"/> would be JFrame.setSize(5)), and there is provision for layout managers, ids, custom JComponents and so on. Substance is Kiril Grouchnikov’s Look and Feel library for Swing components, which allows various subtle and lurid modern effects: the latest release, version 5, has just come out. Kiril has also written a good series of articles, available at his blog, on what he has learned about running a successful responsive open source project.

So, how can I use SwiXML with Substance 5?

Seelcting the look and feel with SwiXML is easy: you just add the appropriate plaf attribute to the top frame element. You can find a list of the various looks and feels available in Substance in the documentation or under the skin directory of the source tree. So your top element is, for example,

<frame  plaf="org.jvm.substance.skin.SubstanceOfficeBlue2007LookAndFeel" ...>
...
</frame>

In many cases, that is all that is needed. However, I found a gotcha with an easy fix.

In Swing, you are supposed to do everything concerned with the GUI in the Event Dispatching Thread (EDT). This prevents some kinds of thread-related problems, for example where one thread tries to access an object already destroyed by another. Some of the Substance code has checks, which generate errors if the function was not invoked in the EDT.

However, all the example code from SwiXML is run straight from the main thread. This is OK in theory because you are just initiating the object so there is no chance of mishap.

The simple answer is to run the SwiXML initialization in the EDT, and invokeLater() is our friend here. Instead of

   Component myGUI = new SwingEngine(this).render("xml/myGui.xml");

you use

  Component myGUI; 

  SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        try {
           myGUI = new SwingEngine(this).render("xml/myGui.xml");
       } catch (Exception e) {
           e.printStackTrack():
      }
     }
   }
  Thread.yield();
  Thread.yield();

Unfortunately, even though Substance is highly configurable, it mainly uses various kinds of system or UIManager or rootNode properties rather than setters. It is possible to make a home-made property element in SwiXML , however because Swing elements are initialized before being attached to their parents, there is no way to access the client properties of the intended target object, so it is not well suited. (It seems you may be better of just specify the look and feel mix you want in Java rather than SwiXML at the current time.) But the simple selection of Substance LAF seems to work fine.

SwiXML already has support for the JGoodies Forms library built-in, which some of the Substance demos also use. I hope to get around to using the Flamingo library with SwiXML soon.

What are those Thread.yield() calls? The first is to make sure that the EDT thread runs immediately, just because we want to be in a good known state. The second is because it is possible, though not likely (though in Java who can say?), that Substance itself may use invokeLater() calls on the EDT where the runnable will be cued on the EDT but scheduled after the return from the first Thread.yield(). The second call is at least harmless, and perhaps best practice: some people think that just because in Java you often don’t have to think much about scheduling (or memory management) it means you never have to: quite wrong for desktop apps.

Firstly, I, like many of you, am glad to see that Dare Obasanjo’s indefinite hiatus from the blogosphere was short lived.

Secondly, while I most certainly agree with the premise of his recent “In Defense of XML” post — which was written in response to Jeff Atwood’s most recent attempt to get everyone in the XML communities of the world to hate him ;-) — the fact of the matter is:

Jeff Atwood is right!

Now before any of the members of the XML communities (of which I am a proud member of several!) pull out their pitchforks, and before any of the XML haters of the world declare some sort of moral victory, let me be perfectly clear:

So is Dare!

“WTF?” you say? Well, let’s first take a look at a portion of Dare’s argument:

CherryPy 3.1 is out and there are some exciting new features. The first exciting piece is the Web Site Process Bus. Robert Brewer had come up with an idea to create a generic server management API to help make management tools and libraries for Python servers standardized. Essentially, this is like WSGI for managing Python web servers. The next big feature is cherryd, which allows you easily run a CherryPy server as a daemon. Paste had a similar feature and it made managing Python web applications feel more like managing a tradtional web server. Also, the set of changes for CherryPy 3.1 make it possible to run CherryPy on Google’s App Engine. If you’ve never checked out CherryPy, take it for a spin. Congrats to Robert and the rest of the CherryPy team!

My saga on problems with GMail continue. Despite of the -ve feedback (”GMail is working fine“, “GMail is awesome‘, “Not sure why you are complaining GMail?” etc) to my posts, I continue to see the problems with GMail. I am not alone on the planet, lot of people are in the same boat(You can read the problems with GMail here, here, here, here, here, here, here, here and here). The problems are frequent and particularly when they release new features. Some times I feel that Gmail is rushing to release the features without proper testing. May be they think that it is OK to roll out the features with bugs as it is in beta. Until now it was my guess only, but it turned out to be a fact. Sergey Solyanik who worked on GMail revealed some interesting facts on Google procuts and culture after leaving Google.

In the last year, and slick as it is, there’s just too much of it that is regularly broken. It seems like every week 10% of all the features are broken in one or the other browser. And it’s a different 10% every week - the old bugs are getting fixed, the new ones introduced. This across Blogger, Gmail, Google Docs, Maps, and more.

It seems Google culture is focused on introducing the cool features, not focusing on quality. Does Google think that since it is free for the user to use, quality does not matter? Well, it may be free to use, but Google is making money off of it by placing ads.

The culture part is very important here - you can spend more time fixing bugs, you can introduce processes to improve things, but it is very, very hard to change the culture. And the culture at Google values “coolness” tremendously, and the quality of service not as much. At least in the places where I worked.

Incidentally his journey from Microsoft to Google was not as good as he thought and took U turn back to Microsoft. Also he explained why Microsoft is better than Google to progress in the career.

The Google Manager is a very interesting phenomenon. On one hand, they usually have a LOT of people from different businesses reporting to them, and are perennially very busy.
On the other hand, in my year at Google, I could not figure out what was it they were doing. The better manager that I had collected feedback from my peers and gave it to me. There was no other (observable by me) impact on Google. The worse manager that I had did not do even that, so for me as a manager he was a complete no-op. I asked quite a few other engineers from senior to senior staff levels that had spent far more time at Google than I, and they didn’t know either. I am not making this up!
At Microsoft, the role of a manager is far more obvious. A dev lead is responsible for the success of the feature and the health of the feature team. A dev manager is responsible for the success of the product and the culture of the dev team. A PUM is responsible for the success of the business, and interoperation of the three teams that work on the product.

Isn’t it bad for a company like Google not focusing on the quality?

Update: Slashdot is also discussing this from a different prospective “Some Developers Leaving Google For Microsoft” and on:

Everything is pretty much run by [engineering] — PMs and testers are conspicuously absent from the process. Google as an organization is not geared — culturally — to delivering enterprise class reliability to its user applications.

During the recent discussion of the OAI-ORE drafts (which use RDF), the claim was made that RDF is serialized in RDF/XML and thus could be considered an XML representation of the underlying data model. My response to that was that the RDF model is different from XML, and that it thus is pretty hard to process RDF/XML using XML tools, in particular when considering all constructs allowed by RDF/XML, and maybe even the possibility how to update RDF/XML data using XML tools alone.

I tried for some time to find a general-purpose RDF/XML parser written in XSLT, but so far could not find one. But Google is imperfect and i might not know the best places where to look. So here is my question: Is there a general-purpose RDF/XML parser written in XSLT? It has to support all the fun stuff allowed by XML and RDF/XML, such as weird uses of namespace declarations, XML Base, rdf:ID and RDF/XML syntactic sugar. It must accept anything that is valid RDF/XML. As a result, it should produce some form of normalized RDF/XML, but I really don’t care that much about the exact format (ideally, it should be XPath-friendly). The parser must be robust enough to produce the exact same normalized result for inputs that look radically different because of XML and RDF/XML syntax variations.

I am really interested to see whether such a beast exists, and if so, how big it is. My guess is that it’s not trivial to write such a parser, but it definitely is possible. After finding out whether such a beast exists, my follow-up question will be whether there is an associated function library that can then work on the parsed RDF model, so that the data can be traversed, queried, updated, and serialized.

It is interesting to see the progression of free software along side the proliferation of the web. When I first started programming, I got involved with a web CMS I used in my contract work. I would write a new plugin or feature along with migrating a design to the software and afterwords, I would try to contribute it back. One time, the designer I was working with asked me to remove some of the project branding as well as a GPL notice on the login page. Some of the community found out and a rather long dialog started regarding whether or not I had violated the licensing. In the end, I came to compromise with the original author and we all moved on.

I still think I was right. I contacted the FSF about the issue and they confirmed my evaluation. My argument was that users of the site were not the same as users of the software. My clients had access to the source code and they were free to change it however they wished. I considered the client the actual users of the software. My removing of GPL licensing information from being visible in the login HTML did not limit or restrict the users freedom in any way and I continued to include the same copyright notices in the source of the page. Needless to say, while I disagreed with the author’s perspective, I had no problem coming to a compromise. The whole situation was far from heated and I personally think it was a healthy dialog for the community.

My situation made it clear there were still questions to be answered in terms of free software on the web. It seems the AGPL is one answer for this kind of problem and there are some folks actively promoting it as a means of distributing web based free software. While, I think the terms (as I understand them) of the AGPL would not work for something like a CMS, the concept of providing software over a network and considering the output as licensed has its value.

The best thing about the AGPL is that it provides a licensing option to help provide free software on the web in a similar way the GPL effected desktop software. I don’t think many web developers have thought about web applications in the same light as desktop applications. Installation challenges, database dependencies and server requirements have all made running web applications something left for developers and advanced users. Fortunately, server software is becoming common place and programming languages like Ruby and Python are helping smooth over the issues with distribution and deployment. It is good to know that as web applications continue to evolve, free software will evolve with it.

The W3C just published a new TAG Finding called Associating Resources with Namespaces. Here’s the abstract:

This Finding addresses the question of how ancillary information (schemas, stylesheets, documentation, etc.) can be associated with a namespace.

I don’t quite understand why the TAG findings are hidden on some badly named Web page. Some of them are pretty interesting documents, and yet they are not published on the W3C Technical Reports page, and the W3C Home Page does not link to them or publish news snippets about new findings. I think these documents should be easier to find.

Technically speaking, the finding talks about how to create namespace description documents, so that namespace names can point to helpful resources, rather than being abstract identifiers. The TAG finding breifly describes possible languages for namespace description documents (RDDL 1.0 and 2.0 and GRDDL), and describes a vocabulary of terms for describing the nature of resources being linked to in a namespace description, and what the purposes of these resources are. The definitions of these terms, though, are one-liners with little guidance to what that concept is supposed to represent.

What I am missing most (and what we were concentrating on when we were defining our own format for namespace descriptions in an e-government scenario) is the ability to associate namespace descriptions themselves, and make assertions such namespace x depends on namespace y. Or rather simple but really helpful pieces of information (in particular for developers) such as namespace x is usually associated with one of these two namespace prefixes, here is where you can find test data, or here is where you can find some example data.

At the Semantic Technologies conference in San Jose I attended an interesting presentation entitled “persistent identifiers for the real web”. XML often uses URLs for identifying schema namespaces, and I suppose could be credited for influencing RDF’s practice of using URLs for identifying resources. In using RDF to describe and annotate things a problem arises…are you describing the web page, or the thing the web page is talking about. For example, if I assert that:

<http://tcowan.myopenid.com> :likes <http://www.myspace.com/lettucefunk>

Does that mean I like the web page or the band the page is about? As you’re traversing the semantic web it’s going to be advantageous to distinguish between content assets and the real world entities they may represent. Their proposed solution involves PURLs (http://purl.org for example). Normally a permanent URL redirects you to the best representation of the resource via a 302 response. They propose that when the PURL represents a real world entity that the response be given as a 303 (see also). The computer agent can then understand that the “thing” is a real world entity, and that the redirect is not to the real thing, but to another web resource about the thing.

I’m very much in favor of permanent URLs. Otherwise all our assertions will become disjointed as links break, or we’ll have to keep our own “archives” of dead links and sites. I also appreciate the simplicity of Dave and Eric’s proposal, however, I’m not so sure this is really the best way to solve identifiers for real world things. Consider books for example…what would be the best way to represent a book, it’s URL on Amazon or it’s ISBN number as a URN? If we use the Amazon URL we can’t be sure it’s a book, it might be binoculars or a coffee table. The URN however makes it clear:

URN:ISBN:0-395-36341-1

The urn namespace indicates that it’s a book, without a doubt. If PURL were to host a “see also” permanent URL scheme for each declared URN namespace we’d be able to visit that URL to find out more…

http://purl.org/urn/isbn/0-395-36341-1

But on the practical web, we don’t use PURLs or URNs for books, we use the Amazon.com url. I think in practical terms things are going to be represented on the web by the domain that has the best collection with the best open content. Perhaps the best approach in the end is to take advantage of blank nodes.

<http://tcowan.myopenid.com> :likes _:a
<http://www.myspace.com/lettucefunk> :describes _:a
_:a a :funkBand

In English, http://tcowan.myopenid.com likes the funk bank described by http://www.myspace.com/lettucefunk. Now we’ve made it clear, and without the use of PURLs or some new PURL redirection strategy.

First some jargon (from the Glossary of Typesetting Terms or Harrod’s Librarians’ Glossary full props to Google.) Castoff: The calculation the number of typeset pages a manuscript will make, based on a character count. Proof: An impression made from type before being finally prepared by printing. Proofs are made on long sheets of normal page width… Galley proof: Proof of text before it is made up into pages…just as long as can be conveniently photocopied – usually 13 inches. Compose: To set type-matter ready for printing.

Deciding on breaks

I am ancient enough to have used galley proofs, the long pages of text of books before it had finally been made up into the final pages and runoff on a printer (or rather, by a printery.) It still exists in the draft modes on some modern word processors, I suppose. There has always been a chicken and egg problem in documents which contain dynamic forward references that expand to section or page numbers (e.g. See page 99: how do you know how much space to reserve for the page number? A reference on a tightly-set line or full page may cause different page breaks if it is a two or three digit number, for example. A traditional way to deal with this was to allow a lot of space around page references (to reduce the impact) and to take two passes of the document, the first to estimate the pages and the space required for each reference, and the second to actually compose the document using the calculated space as fixed and squeezing the generated text if necessary.

The idea that you could divide the same text into different length pages is obvious, and quite early on even the electronic typesetting programs alllowed draft modes (or provided alternative macros) for producing proofs. The requirement of some publishers for double spaced manuscripts made the idea of separating structure and presentation, ideas ascribed to Charles Goldfarb and (independently) Brian Reid, does not seem a big leap to us nowadays. Multi-publishing and retargetting became commonplace in the SGML arena, with the advent of declarative stylesheets looming for a long while, but the next really big step was with the advent of the WWW and the impact of resizable windows on formatting.

One of the most important ideas following from the separation of presentation (into stylesheets) and content has been the formalization of the page-flow model (frames), which was championed by Frame Corporation’s FrameMaker though the simpler concept of regions was of course older. The idea is you “pour” the text into the frames and they flow, break and cause new pages where they will.

Loose

In my blog yesterday, I mentioned that the transformational approach of stylesheets in XML (the DSSSL, XSL-FO streams) is only loosely-coupled with the typesetting engine (or formatting engine…some people think that word processors don’t do typesetting, I don’t want to get hung up on terminology) so there are some kinds of page design rules that are impossible even if because the developers cannot be aware of every design rule anyone might want to make.

The separation also impacts another area: the area of document interoperability. I have written several blogs referring to Markup’s Dirty Little Secret, which is that because everyone’s system and each system’s algorithms and resources and capabilities are different, you cannot expect perfect fidelity to the extent of the same line and page breaks when exchanging XML+stylesheet documents (such as OOXML, ODF, DOCBOOK, you name them). This goes quite against the expectations of some users (though I think people are much more realistic about this now than two years ago) and quite against the hard requirements of others (for example, people who need fixed page numbering for legal requirements.)

In yesterday’s blog, Standardization as a collective loss of imagination? I suggested that users may need to assert themselves to prevent the standardization of the current round of office application formats from a particular pitfall of losing sight of the centrality of page (and document and information) design: how to help people communicate rather than how to add the latest pet feature from some vendor. Not that pets are not fun and valuable.

Hinting at our priorities

The tie in that suggestion and the page-fidelity problem (which is really an interoperability issue) is that I think we need some more imagination about whether our current re-pour-each-time model of formatting is actually good enough if we genuinely want substitutability of office applications. People don’t want to be sold a turkey.

Now SGML did provide processing instructions, a kind of markup that still exists in XML, for applications to add extra information that belonged to formatting for example. The ArborText Publisher program used them very successfully, with processing instructions that let you force page and line breaks in certain places, for example. That is one way Iof integrating page markup, but it is not what I am suggesting (for various reasons.)

At the moment, I think that a much better approach would be to add a kind of cast off hint as an attribute to each block-level object (paragraph, list item, table cell, etc). This would be added to the XML markup by the formatting engine as a hint, to enable a subsequent formatter to try to get the same results.

The first time data came into a document, the normal composition mechanisms would apply. But the document’s block structures would also be decorated by these hints at save time. And subsequent opens of the document would use these hints as well when composing the pages. For example the castoff hint might be as simple as
giving the bounding box of the block on the page. The composing system would used differences in these bounding boxes with the bounding boxes it wanted to use as penalties to adjust line feathering (or even margins, padding, breakpoints, spacing, text size.)

Auto-sizing is not completely unknown: WordPerfect had a patent on automated adjusting various page parameters to make sure some range of text fitted on a single page. And many people are aware of the behaviour of some page-oriented systems such as presentation programs to automatically resize text (including nested text lists) to fit into the available space.)

It could be user selectable whether to freeze the page according to the block hints or just use them as hints, or ignore them. As a hint, it wouldn’t interfere with minimal implementations.