I’ve been giving a lot of thought lately to JSON and JavaScript in particular. For those in the XML community not familiar with it, JSON was largely the invention of Douglas Crockford, a JavaScript expert who currently works as an architect for Yahoo! and who I had the privilege of meeting at the recent AJAXWorld conference.

The idea behind JSON is surprisingly simple, and like many simple ideas, is also remarkably powerful. Dissatisfied with the complexity involved with using XML as a data format for seemingly lightweight tasks, Crockford asked one of those questions that causes all kinds of interesting repercussions: “Why couldn’t we use the “native” data format of the JavaScript, the associative array, as a vehicle for transferring content between client and server, instead of XML?”

A typical object in JavaScript actually has a number of traits that make it particularly suitable to this task. It is late-bound, which means that any JavaScript entity consists of one of a handful of primitive types - number, string, arrays, objects, dates and e4X entities (the latter only with Mozilla and Flash at this stage). Moreover, the combination of a handful of primitive types, associative arrays and linear arrays can effectively represent most data structures used in programming - something which XML developers realized some time ago.

JavaScript also has its foot in both the Algol based languages such as Java, C++ and Smalltalk, and the Lisp like declarative languages including Scheme, Prolog and Haskell among others. Not surprisingly, this means that parsing out JavaScript objects in other languages is in general simpler than just about any other representation. This idea lies at the heart of the JavaScript Object Notation, or JSON: Provide a basic syntax for serializing JavaScript objects that can in turn be used transmitted elsewhere and parsed back into JavaScript objects on another platform. Given the ubiquity of JavaScript within web clients (and the rise of XMLHttpRequest pipes) this strategy is actually quite effective … and provides an alternative to XML in a number of cases (more on that in a bit).

Consider a reasonably complex JavaScript object (in this case using my favorite example, the “standard” RPG play character), described in hash format:

var char = {
    name:"Aleria Delamare",
    gender:"female",
    player:"Jen Watson",
    vocation:"mage",
    level:5,
    health:18,
    shields:5,
    species:"human",
    attributes:{
       strength:12,
       intelligence:17,
       wisdom:15,
       dexterity:14,
       constitution:9,
       charisma:17
       },
    spells:[3,2,2,1],
    toString:function(){
        with (this){
           return name+" is a "+gender+" "+species+" "+vocation+" played by "+player+".";
           }
        }
    }

Once defined in this fashion, accessing any field in the object involves using simple dot notation, array bracket notation, or via invocation:

print(char.attributes.charisma)
=> 17
print(char.spells[3])
=> 1
print(char.toString())
"Aleria is a female human mage played by Jen Watson.";

(Note that since toString() is the default method used when a function is invoked in a string context, you could also have said just print(char) and it would have done the same thing.)

Serializing this information once the object has been created is not completely trivial, but nor is it that hard. It helps (considerably) to define a helper function called getType, which can be used to determine the type or more complex objects and provide a wrapper around the fairly ineffective typeof() operator provided in JavaScript 1.5:

var getType = function(obj){
     switch (typeof(obj)){
         case "number":{
             return "number";
             }
         case "string":{
             return "string";
             }
         case "function":{
             return "function";
             }
         case "object":{
             if (obj == null){
                 return "null";
                 }
             else {
                 var constr = obj.constructor.toString();
                 var reType = /^functions(w+?)(/;
                 var resultArr = reType.exec(constr);
                 return resultArr[1];
                 }
             }
         }
     }

The fairly complicated block:

                 var constr = obj.constructor.toString();
                 var reType = /^functions(w+?)(/;
                 var resultArr = reType.exec(constr);
                 return resultArr[1];

serializes the constructor of an object, which is typically of the form:

function ConstructorName(){/* code goes here */}

and then extracts the name using a regular expression object.

print(getType(obj));
=> "ConstructorName"

This is primarily used to differentiate between an array and an object, which cannot in general be disambiguated with the typeof keyword (which reports that both are of type “object”).

A second helper function (and one which is assigned as a global to the string method because it can come in handy in any number of scenarios) is the repeat() method:

String.prototype.repeat=function(count){
    var expr = this;
      if (count == null){count = 0;}
      var buf = "";
      for (var index=0;index!=count;index++){
          buf +=expr;
          }
      return buf;
      }

This repeats the string it’s attached to by the count() number (with a count of 0 being no repeat). Thus, to repeat the letter “a”, you’d simply write this as:

"a".repeat(5);
=>"aaaaa"

The actual serializer (here called toJSON()) acts on an object (or array) and renders it as a serialized version:

Object.prototype.toJSON = function(){
     var recurseObject= function(arg,level){
         if (level == null){level =0;}
         switch(getType(arg)){
             case "string":{
                 return "\""+arg+"\"";
                 break;
                 }
             case "number":{
                 return arg;
                 break;
                 }
             case "Array":{
                 var buf = [];
                 for (var index=0;index!=arg.length;index++){
                     buf.push(recurseObject(arg[index],level+1));
                     }
                 return "["+"n"+'t'.repeat(level)+buf.join(',n'+'t'.repeat(level))+"]";
                 break;
                 }
             case "Object":{
                 if (arg == null){
                     return "null";
                     }
                 else {
                     var buf = [];
                     for (key in arg){
                         if (key != "toJSON"){
                             buf.push('"'+key+'":'+recurseObject(arg[key],level+1));
                             }
                         }
                     return "{"+"n"+'t'.repeat(level)+buf.join(',n'+'t'.repeat(level))+"}";
                     }
                 break;
                 }
             case "function":{
                 return arg.toString();
                 break;
                 }
             }
         }
     return recurseObject(this);
     }

This function actually creates a private named function called recurseObject which does most of the heavy lifting - It matches keys with their contents, and if their contents are themselves objects or arrays, if performs the same function on the child nodes. For instance, with the above char object,

var charJS = char.toJSON();
print(charJS);
=>
{
    "name":"Aleria",
    "gender":"female",
    "player":"Jen Watson",
    "vocation":"mage",
    "level":5,
    "health":18,
    "shields":5,
    "species":"human",
    "attributes":{
        "strength":12,
        "intelligence":17,
        "wisdom":15,
        "dexterity":14,
        "constitution":9,
        "charisma":17},
        "spells":[
            3,
            2,
            2,
            1],
    "toString":function(){
        with (this){
           return name+" is a "+gender+" "+species+" "+vocation+" played by "+player+".";
           }
        }
     }

The formatting here is deliberate - it makes it far easier to read a JSON object with the added spacing, and such spacing can be removed readily with a regular expression.

However, note that the purpose of such a JSON object is ultimately to be transmitted and reparsed at some later time back into an object. In the case of JavaScript, such parsing is remarkably simple to achieve, requiring only the use of the JavaScript eval() statement:

String.prototype.parse = function(){
    return eval("(function(){return "+this+";})()");
    }

Thus to reparse the serialized object, you’d simply invoke the parse method on it:

var char2 = charJS.parse();
print(char2.name);
=> "Aleria"

By the way, It is worth making a critical observation here: JSON objects passed in this way make it possible to inject code into the environment in a big way, since functions can be serialized and then re-enabled on a different machine. If you are in a trusted environment, with the JSON objects coming through relatively secure channels, this isn’t generally a problem, but if you are in a situation where script injection is possible, it is often better to use “safe” parsing methods that can be used to strip such functions from the source, such that the JSON objects being sent are exclusively data objects with no executable content.

There are a number of functions similar to those above at http://www.json.org for doing such safe parsing, as well as information about using JSON in a wide host of other environments, from C++ to Haskell to Ruby.

Douglas Crockford has been spearheading the JSON movement and is the driving force behind getting JSON adopted by the various browser manufacturers and standards groups. He has written extensively on JSON, envisioning it as a replacement for XML in the web services arena.

I have a few comments on that. Web services XML has always been somewhat problematic, as there has traditionally needed to be a balance between providing enveloping metadata while at the same time keeping the notation relatively terse for performance reasons. You are beginning to see JSON “feeds” that essentially mimic many of the same capabilities that are found within XML-based feeds, and to a certain extent they perform quite effectively in this role.

Which is better ultimately comes down to size and secondary use. I’m moving much of my XML binding architecture into more of a JavaScript layer because doing DOM parsing of content is remarkably extensive. E4X can help with this considerably - it is, in fact a fairly viable alternative to JSON while still retaining the ability to render as XML, but E4X has only minimal adoption to date. JSON is less effective in any situation requiring some form of transformation (such as via XSLT), and it is in fact here that an XML approach can prove superior. Moreover, the parsing and memory costs for JSON can become significant once the content reaches a certain size, as the JSON objects are generally loosely rather than tightly bound, so there is definitely a tradeoff that tends to favor XML as the content becomes more document-like.

Ultimately, in evaluating these two technologies you should understand that both XML structures and JSON entities are declarative structures (at least to the point where you start introducting serialized functions) - in essence, JSON can be seen as a variant of XML that replaces the angle brackets with curly braces and that doesn’t recognize (explicitly) the notion of attributes, although you can certainly use conventions to mimic this. I suspect that as ECMAScript for XML becomes more pervasive in the desktop environment, you will see some interesting struggles back and forth as to which of the two formats will become dominant, but that struggle is still a few years in the future.

Kurt Cagle is an author and programmer currently wrapping up Firefox AJAX Programming for Apress Books. He lives and works in Victoria, British Columbia.