Nick Carr (the Australian XML éminence grise not the US journalist) asked me whether the software world was fragmenting. Here’s my answer with some diagrams: software seems to be organizing itself into three layers of support/runner/plug-ins where each of those layers in turn is organized as support/runner/plug-ins.
The early model of PCs and simple computers had a model something like this diagram:three layers: device drivers, operating system and applications.
Exactly which features go where has always been subject to experimentation: microkernels, hardware extaction layers, and so on showed that the kernel conceptually belonged in a low layer. Windows seemed to try hard to unlayer matters, to make everything part of the operating system from device drivers to Internet Explorer. But almost no-one was fooled; just because you eat the whole cake doesn’t mean it has no layers.
The three layer model is re-establishing itself: this time with the layers of operating system, platform and suite.
Each of those layers in turn organized as similar threes, with libraries down the bottom, a runner in the middle and plug-ins providing alternative faces at the top.
In the Operating System layer, operating systems are being squeezed at both ends: the device driver layer is becoming more important, both because of smarter devices and the increasing power of BIOS. The rise of virtualization technologies is splitting the operating system into a run-time executive able to run multiple virtual operating systems. VMWare’s ESX is an example of that kind of executive for servers, while Apple’s virtual OS 9 is an example of a virtual OS for desktop systems.
Next comes the Platform layer: Java and .NET notably, but also Perl, Python and other platforms. They have a set of libraries, a virtual machine or runtime, and different languages.
Rather than applications, I think it is better to consider the top as the Suite layer. Suites are where all the action is and have subsumed libraries and component architectures. For example, you could take Eclipse code as a library and build your own application, or you could use Eclipse as a Rich Client Platform and use its platform capability, but Eclipse’s big gain is the network effect of using it optioned-up in conjunction with other plug-ins. Other suites: Apache, Mozilla, Open Office, Microsoft Office, Visual Studio, NetBeans. (Ruby on) Rails is quite suite-like with its component architecture: certainly some of Oracle’s offerings are.
In the world of suites, software development becomes a matter of value-adding: of not needing to provide functionality that other plug-ins provide but also of needing to enhance them as well as being enhanced by them. If you cannot use the suite, you use the rich client platform; if you cannot use the client platform; you just use the library and framework. If you cannot use the framework, then you go to the non-suite world of SourceForge libraries.
Just as the Suite has its value because it provides a framework, client platform and one or more plug-in applications, so the Platform has value because it provides the libraries, the runtime and the language, and the Operating System has value because it provides the drivers, executive and a virtual operating system. Indeed, these built-in features are probably not well-layered at all, but integrated and required: but it seems a matter of time to me—over time code that starts of integrated will be refactored into this kind of layered pattern natrually, it seems to me.
In a way, the Marc Andreessen comment (actually Bob Metcalf’s?) about Windows reducing to a poorly (or partially) debugged set of device drivers for Netscape isn’t wrong: just that Windows is becoming a set of device drivers for virtual Windows (virtual Linux, etc). I think this is one reason why Microsoft doesn’t need to put out Vista to a deadline or in a hurry: the monolithic model is already superceded by events. This is not to say that Vista is dead in the water by any means: it is a cohesive bundle and a brand independent of any conceptual layering aspects. I was astounded to see that Microsoft itself distributes Red Hat Linux (as a virtual OS for its virtualization system)! Microsoft are repositioning themselves, to some extent, from being a pure software company to being a media company. And the need for cheaper, unbundled operating systems for poorer countries coupled with regulatory pressure should help make these layers more explicit over time.
The web fits into this model too. (It is tempting to see WS-* as the early attempts to make a fourth layer to bind together different three-layer stacks.) I haven’t thought them through completely, but I think Web Applications and Web Operating Systems fit (or will ultimately fit) into this pattern too.
The same 3-layer pattern can be discerned even inside plug-ins: application skinning and schemas are kinds of plug-in, I guess.
Anyway, after all that, the bottom line answer to Nick’s question is that the cracks we are seeing (i.e. all these different technologies that seem to challenge the old Windows/Win32 tight coupling) are not fragmentation but reflect natural layers.