This has been one of the rougher weeks I’ve faced technologically speaking. A technical glitch forced the server that housed most of the Understandingxml.com content for the last year to reformat itself, wiping out much of what’s been there for the last several months. Fortunately, much of this content was also echoed on this site, so not a huge amount has been lost, but it will take a while before all of my essays have been migrated back. For those of you who’ve wondered where Understandingxml.com has gone, well - it will be back up in time. I’ll be using XForms.org as my primary blogging point for a while, until we can get things sorted out with UXML.

It happens periodically that I get hit with what I’ve come to term a “chaos storm”, where technology in general seems to experience extreme entropy around me. When I was first learning computers in high school, I’d actually held off even though programming appealed to me because I had absolutely no aptitude whatsoever with electronics, and even several years later when I was working on a degree in physics I chose fairly esoteric theoretical areas because the experimental physics professors quietly discouraged me from getting anywhere close to a laboratory.

One of the most seminal novels of my late teen years was a book by called Master of the Five Magics by Lyndon Hardy. The novel itself was reasonably well written, engaging enough in its own ways, but I think that I found most fascinating about it was the underlying premise (*spoiler alert*) - a young man showed evidence of strong magical potential, but he was completely and totally inept in any of the five disciplines of magic that existed. It was only after much exploration and frequent encounters did the truth emerge - that the man was actually a meta-magician: he could not readily use the magic in a given system, but he could change the rules by which the magic in that system worked.

I’ve always felt a certain degree of sympathy with the protagonist of that novel - societies at a whole are systems, ones that work according to mostly unstated rules. For most people, those rules simple exist, their reality is composed weft and weave of those rules. They understand implicitly what is needed to not only survive but prosper in this world, and they do it without thinking about why it is that they do those actions.

However, there are some people who can actually see the rules, understand the arbitrariness of them, and can step outside those rules to examine them “from the outside”. I think its a phenomenon that most immigrants discover, because the rules that they followed before are frequently different (sometimes dramatically so) from what they had known, and so they are forced by their circumstances to be outsiders. Yet even there, this is a transient state for most - looking at any society from the outside is typically mentally painful, making life harder than being contained within that society, so they learn their new culture quickly.

Curiously, however, once they have adapted, their transition back to the old culture when they return is often hampered because they now carry in their heads two sets of rules for behavior, and so such behaviors require thinking about your actions rather than just doing them. Anyone who’s had to think about riding a bicycle understands how hazardous this can be.

Programmers can similarly be thought of as being either magicians or meta-magicians. The magicians are people who are capable of getting computers to do anything, to sit up and beg, to build complex games or tweak the last bit of performance out of the processor, usually by applying their particular programming language and understanding that language intimately. The magicians are responsible for a significant amount of the code that’s out there.

The meta-magicians are fewer and farther between. They are often not all that good as programmers in the traditional sense. On the other hand, they are frequently much better at observing patterns, at recognizing utility, and at building new languages. Some meta-magicians end up becoming the authors of the languages that the programmers use, yet they’re not necessarily all that good at making the jump from creating languages to becoming proficient in using the languages to create applications. While I don’t think there’s as clear cut a distinction in the real world between magicians and meta-magicians as there is in Hardy’s book, the reality is that you have a fairly limited amount of time in the world, and you can concentrate either upon learning how to create languages or how to use created languages, but it’s extraordinarily rare to find someone who is proficient with both.

XML to me is certainly an embodiment of this principle, and the dichotomy expressed above can certainly be seen with XML, perhaps more so than normal. XML is all about the creation of languages - it is, in fact, the fundamental characteristics of the this particular meta-language. Sometimes those languages are little more than simple data structures - think Atom, for instance - but often you can create XML based-languages that provide the same level of expressiveness as any procedural language, albeit at a different level of abstraction.

I’ve actually been thinking on this concept for a while. Something I’ve come to realize is that one consequence of the model/view/controller model is that there is a very clear separation between the data model and the view, via the presumably “black box” controller. The problem with most imperative solutions that I’ve seen architecturally is that such a black box is usually more likely a dark brown or purple … in other words, the imperative language often forces the creation of interdependencies between the data model and the presentation layer, and moreover tends to encourage similar bad habits in programmers to remove some discipline for programmatic ease of development. The “meta-” becomes too entwined with the “language”, and one effect of this is that you end up with creating one-offs that are both fragile in the face of changing data models and limited in their code reuse.

It’s a big part of the reason that I find such things as XBL and XForms so intriguing. XBL makes it possible to create custom “extensions” to existing presentation layers such as XHTML - in essence, to extend the language by giving behaviors to elements that were not originally designed to have such within the original specifications. This means, in theory, that you could effectively create an entire schematic match between an XML language and an associated view layer just by building a comprehensive binding system. This approach was actually one taken by the SVG when they were working sXBL and is what makes XUL possible as a working language. Language creators can thus associated bindings with things not only with those things that have “document-like” characteristics, such as web page divs or graphical entities, but also with those things that are more object-like in nature - an invoice, a part description, a process workflow declaration.

In a related vein, languages such as XForms present the notion of a binding with one or more data models in a clear and unambiguous fashion. The XForms application acts as a facade to the data itself, and while that facade can be something as simple as one to one associations between text fields and content, it can also handle considerably more sophisticated interfaces that approach (and sometimes exceed) the notion of site navigation. The mechanism - the controller layer between the data and the presentation - is remarkably solid as a black box entity, largely because of its reliance upon standardization … to distinctly different XForms implementations can still use the same XForm (with at best very minor variations) and as such XForms becomes a remarkably sophisticated tool for the conversion of XML languages into demonstrable applications.

Thus, I’m beginning to realize that my own proclivities for meta-programming rather than programming are manifesting themselves most effectively in XForms and related language technologies. As a language it’s “safe” from people like me, because in essence, such XForms applications can be developed as very lightweight faces on these XML-expressed linguistic structures - regardless of the vocabulary.

The next logical extension to this is a way to generate such XForms from the data model schemas in the first place. I’m working on this now, and hope to have some very interesting articles on such data object publishing systems in the very near future.