I’ve always been fascinated by dictionaries. We create dictionaries one word at a time, attempting to nail down within a printable tome all of the words the form a language, and for many people such dictionaries represent something immutable, a stamp of authenticity that states that this is the proper way to represent legitimate forms of spelling, legitimate statements of meaning, and in many cases some form of an audit trail that attempts to provide the history of the words.

For all that, dictionaries are fundamentally arbitrary. Someone has to make an editorial decision as to whether a given word is in fact a legitimate word with the language. Is sombrero an English word, or a Spanish one. Is degauss, to remove a magnetic field, a term of technical jargon or a formal member of the language. Is it theater or theatre? My teenage daughter talks about her and her friends being “random”, by which she means that she’s not easily categorizable to modern marketing efforts. Random, to me, is a stochastic definition that means that there is no readily discernable pattern dictating the results of a given function. While there is some commonality to both terms, they are not the same. Is my daughter’s definition of random valid? Is mine?

As I become more heavily involved with ontology and computational semantics, I find this distinction between the notion of a standard as being authoritative vs. the arbitrary nature of developing such standards both troubling and insightful. A standard, if it is well designed, is not something etched in stone; rather, it represents (in the best case) the ideal balance between internal integrity and the necessity of capturing complex (and often intangible) ideas in a world where the basis of authority is often remarkably flimsy.

Several years ago, when I was staying with my uncle in Washington, DC after graduating from college, I went by the Library of Congress (something that I did quite often that year) to check out some of the secondary libraries that are part of the library complex. Sitting at the computer catalog in the music and film library, close enough to the front desk to hear the conversations, I saw a very genteel lady come up to the librarian at the front desk.

“Excuse me,” she said, in a clipped British accent.

“Yes?” the librarian responded. “Can I help you?”

“Please. I’m trying to find the origin of the term ‘chorale’.”

“I can help you with that. Let me check the Oxford English Dictionary.”

“Oh, dear. You see that’s the problem, isn’t it?”

“Huh?”

“Well, I’m from the Oxford English Dictionary …”

Who do the authorities go to when seeking meaning?

Ontologies are tricky things. The first part, defining the terminology inherent within an ontology, places a significant burden upon the professional ontologist (or even the well meaning amateur), because such ontologies must perforce describe conventions, rather than just terms. You are creating the terms of discussion, the set of primitives that make up the relevant language, whether that language is describing a workflow or, well, a chorale.

Yet beyond this initial stage comes the more complex process of defining the relationships between the various tokens or terms of the ontology. Alfred North Whitehead and Bertrand Russell attempted to do this with the intrinsic predicates of logic, in the years before Kurt Goedel dashed their hopes of ever completely defining mathematics irrevocably. No ontology can ever be complete, for there will always be predicates that exist which cannot in fact be stated within that ontology conclusively. This is the curse of the ontologist, the realization that there are in fact intrinsic limits to the ability of defining any language.

We, programmers and computational linguists, are the intellectual descendants of these three prodigies, tasked with the process of using language to drive computers and build models. Yet all too often I wonder whether we have, individually or collectively, lost track of the ultimate futility of what we are attempting to do, losing track of the Heisenbergian uncertainty associated with the process of “standardizing” the languages, human or computational, that all of our efforts are subject to.

Lately, one of the more interesting, and heretical, notions that I have been playing with is the idea that schemas not only can’t be considered to be canonical, but that thinking of schemas as being canonical can prove to be extraordinarily crippling to building complex applications. One of the things that XML does, more than any other language, is to provide a clearcut vehicle for separating the schema from the schema instance. In most computer languages, schema (also known as type, in this context) is inextricably woven together with the instance - the type acts as a formal template, and changing that type (through inheritance or other similar mechanisms) will also change the instance in a one to one relationship.

Yet with XML, the schema is not in fact intrinsically bound to the instance. For most people, this likely doesn’t make a big difference, but I think in fact that this decoupling of schema and instance is perhaps one of the most profound and important aspects of XML, because it recognizes that even definitions can be malleable.

Consider, for instance, one of the more troubling aspects of form design - the concept of dynamic enumerations. For instance, suppose that you have a particular field in your XML based invoice system that is meant to contain the access code for one branch of a company (say, for instance, Starbucks). Now, in any given city there are likely dozens of Starbucks, and the number seems to be increasingly daily. This is information that you cannot in fact encode within a normal XSD schema. You can describe the fact that a given store access code may have a particular pattern or you can create a pointer to a web service, but that pointer still requires that there be some explicit “defining” done beyond that which can be encoded in schema.

This is one of the difficulties inherent in model driven architecture (MDAs). It is rather fiendishly difficult to describe that model completely, because part of the nature of that model is that it may be both dynamic (dependent upon information that may be transitory in nature) and malleable (even static parts of the model may have relevancies that are difficult to encode within any one given schematic language). Up until now, this really hasn’t been that big of an issue - relational database models were considerably more static and self-constrained in terms of their schematic descriptions, and as such the type of MDAs possible from a relational database system were fairly limited.

However, with XSLT you introduce the notion of transformability - you can dynamically create schemas that in turn describe not only models but also possess the capability of displaying those models within a given context (such as XForms, XHTML, or SVG) that allow for the modification of the appropriate instances. This raises some tantalizing prospects - if you can generate your interfaces automatically from schemas, you can eliminate a lot of very expensive programming … and you can also handle the depiction of remarkably complex forms that can handle a great deal of interdependency.

I recently had a comment on a different blog post that the forms generated in this manner are visually uninteresting. Actually, though I find that while there is some validity in the criticism, this is perhaps not a reflection so much of the process (or concept) but rather derives from the fact that our understanding of what is meant by schema tends to be limited to that of static schemas. In point of fact, you can create extraordinarily interesting (and powerful) forms by recognizing that three additional points must apply:
  1. An XML schema (in the most generic sense) is an attempt to model or describe an XML structure. So long as the schema does not in fact violate the validity of the schematic instance, it is a legitimate mechanism for providing one facet of a definition. I call this the Principle of Instance Dominance, because in this case what is most important in the XML world is not the schema, but the schema instance, in direct contravention to the way programming is usually done.
  2. By dint of #1, any schema that provides a consistent definition of a given instance is as valid as any other schema, to the extent that a given instance may in fact have (potentially an infinite number of) multiple different schema types. This rather extraordinary statement means that schemas are a lot like bosons - you can stack multiple bosons in the same space without violating the laws of physics. I call this the Principle of Schematic Transience.
  3. Finally, any given schema system implies that there is a mechanism for processing that particular schematic language that may similarly exist in parallel with other such mechanisms, and so long as the schematic languages used to validate the instances do not violate the validity of the instance within the context of other schemas on that instance, such processing is valid and legitimate. I call this the Principle of Validation Independence.
What does this mean in practice? It means that in order for model driven architecture to work, you may (indeed almost certainly will) need to extend the set of schemas beyond simply an XSD. For instance, the use of Schematron has for the most part been confined to its role as a validator, but in point of fact its not hard to see where the phase design and assertion/report mechanisms of Schematron can in fact become the basis for the integration of business logic within a forms based solution.

Similarly, I find it useful in creating MDA solutions where potential ambiguous situations (such as the identification of a containing element holding a collection of disparate containers and elements as a “tabset”) can radically simplify the process of letting the content itself drive the presentation, and the views so created are considerably more interesting (and can be done in far less time) than the equivalent architectures for hand-derived forms.

Of course this also raises the rather worrisome prospect that so much more work will be needed to specify the additional cloud of quasi-schemas (and their processors) that the resulting effort will far exceed designing the same piece manually. There is a certain degree of legitimacy in such concerns - the amount of effort involved in building such “generators” is not insignificant, and requires some fairly sophisticated XSLT skills. However, in most cases, what becomes critical in building such systems is to recognize that you have both semantic-bound and semantic-less processors at work, and the more you can move toward the latter, the greater the degree of reuse of your code.

A semantic-bound processor is one that needs to know something about the semantics inherent in the instance in order to perform some actions. A schematron “processor” is semantically bound - you are generally looking for specific named nodes in order to perform either an assertion or a report on the given object. However, the generator for a schematron is semantic-less (relative to the initial instance); it does not in fact need to know anything about the semantics of the instance, only the semantics of schematron itself. Similarly, a language that describes such things as the fact that certain elements should be treated as bags while others are either records or collections of records is semantically-bound, but the generator for processing these entities should be semantic-less - the generator should not know anything about the semantics of the instance.

The beauty of such semantic-less processing is that the tools for building the resultant generators will perforce be applicable to a large number of potential instances, regardless of their underlying semantics. This is the real power of such model driven processing … by working with information at a level where the semantics is irrelevant (by placing as much of the semantics into manipulatable schemas as you can) you are able to create very sophisticated applications regardless of the domain involved without having to become enmeshed in the minutae of scripting - not that the scripting isn’t there, but the scripting acts in an encapsulated fashion as a binding to provide behavior to elements in the declarative module, so doesn’t need to be exposed to the integrating developer.

Thus, by seeing a schema as a dynamic document, one constrained by the instances it describes but that is nonetheless quite malleable even given that, model driven architecture becomes viable. There is one last aspect of this that’s worth exploring - visual presentation. A CSS document is a form of schema, though this point is typically lost because people tend to view CSS as ultimately static and typically applied at the very end of the presentation pipeline.

However, CSS can also be generated from XSLT - and in many ways should be generated from XSLT in model driven applications. You can create reasonably sophisticated applications just knowing the general shape of the various elements in the schema (bag, collection, record, rich text, external list, and so forth). Yet if you assume the two stage transformation where you process the schema at the semantic-less level in order to generate an intermediate transformation that in turn is able to handle customization at the semantic-bound level, possibly with XForms or other bound-node technology, you can handle the relevant CSS generation where it differs from stock by treating it as a CSS schema at the second level of transformation, likely from some XML “style-page” source.

While this may seem like a lot of work to build this customization (such as placing form elements in a very tightly constrained grid that looks like a specific paper form) the important thing to remember here is that you would have done this work anyway. A well designed generative system should provide the relevant hooks for you to provide your own customizations, but the customizations, since they require human judgment and human aesthetics, still need to be done by a human being. By creating XML-based CSS “schemas”, you also gain the advantage of being able to automatically create visual interfaces for automating the task of creating the customization, as well as creating a mechanism for documenting the semantics so that a CSS programmer won’t have to try to guess what exactly “.thisDiv” meant in a five thousand line CSS document.

I’m hoping in the next couple of months to publish as an open source project that encapsulates these ideas. There are others who are playing in this arena as well (most notably Dan McCreary, a friend of mine and XML expert who has been doing some very nice things with schema driven development), and I find on my XForms.org site that others who are working with XForms are beginning to see the benefits of this approach as well. My hope (and intent) is to move us all past the point of being independent developers striking out in the darkness and pull us together into a cohesive movement, built around the principle that model driven (or more properly, schema driven) design is both viable and necessary, especially for large scale systems. If you find that the words I’ve written here resonate, please contact me.

Kurt Cagle is a writer and software architect specializing in XML based technologies. He’s also the webmaster of <a href=”http://www.xforms.org”>XForms.org</a>, a community foum focused at XForms and Model Driven Architecture.