I have recently read a series of articles on the Manageability Blog titled Why REST is BETTER. This is a short summary of the articles along with my perspective. The articles are definitely worth a read.

The first article makes a case for a uniform, domain independent object interface comprising of CRUD style functionality. I don’t really understand the comparison of SOAP vs REST as the former denotes a transport protocol and the later is an architectural style. Maybe the comparison should be REST and WSDL/SOAP web services. The article also mentions a number of useful principles, which facilitate extensibility, maintainability and coupling, namely coarse-grained method invocation, argument sequencing and interface uniformity.

The second article doesn’t really add much to the why REST argument. It does iterate that interoperability is a key selection criteria for using either and even goes on to say that the interface and it’s uniformity is key to achieving interoperability. No arguments from me on this point although I am not convinced that we need to adopt such a grand unifying theory on interfaces and interoperability.

The third article discusses asynchronous messaging in its two styles, push and pull. In a push based system it is the producer that is pushing messages to the consumer whereas in a pull based system it is the consumer that is pulling messages from the producer. The article argues that the former is more tolerant to interoperability across change and compares RSS with SMTP, the former being pull-based and that later push-based. At the surface the argument seems compelling enough but a deeper level I am not convinced that the benefits are all that clear. The article also touches on pi-calculus and Robin Miner in order to support the significance of URI’s as first class citizens in architectural and design descriptions. The arguments may not be conclusive but they do provoke further discussion.

The fourth article in the series touches on the properties of software architecture (as described by Roy Fielding), the proliferation of WS-* specifications and virtues of aspect-oriented programming. but for me it didn’t add much to the Why Rest is Better argument.

I am currently involved in a mid-sized project, which uses both REST and WSDL/SOAP style architectures. We have recently completed the first iteration and from a practical sense I find the simplicity of REST a very compelling argument to support its use….but the proof will be in the deployment, scalability and maintenance of the service.

Since finishing the first version of the Work Manager web service I have been looking at load test tools. During my search I revisited a load test framework called Grinder.

I first came across Grinder a year or so ago but never got beyond installing it on my machine. The latest version, which is still in beta, has a lot of new features, including support for the Jython scripting engine.
The latest version also has a console, which collects and displays timing information generated by the load tests.

First impressions have been very positive indeed. I was able to install and run a simple Jython-scripted example in less that 1 hour. After one day with Jython and Grinder I was able to leverage some existing JUnit test cases to do some ad hoc load testing on my web service.

Initial contact with Jython has also been very positive and I intend on learning more about this scripting language. I found the Introduction to Jython Part 1 and Part 2 tutorials, from IBM devWorks, very useful.

Below is a simple Jython test script, which can be executed by Grinder. (Check the Grinder Tutorial for more details)

from net.grinder.script import Test;
from net.grinder.script.Grinder import grinder;
from com.comware.wm.test.rest.load import RestWorkManagerSession;

log = grinder.logger.output;
test = Test(1, "Create, Accept and Complete");
class TestRunner:
    def __call__(self):
        session = test.wrap(RestWorkManagerSession());
        session.getMyWorkList();

This test basically leverages an existing Java class called RestWorkManagerSession (partially shown below) and makes a call to the Work Manager web service to retrieve my work list.

public class RestWorkManagerSession extends AbstractRestTestCase {
	       ...
    public WorkItems getMyWorkList() {
	String myWorkListUrl = getLocationUri() + _sessionId +
                               "/workLists/myWorkList";
        try {
	    return getWorkList(myWorkListUrl);

	} catch (Exception exception) {
	    // convert to a runtime exception

	    throw new RuntimeException("getMyWorkList",
		  		       exception);
	}
    }
}

Grinder uses a property file to parametize particular aspects of a load test. In the property file you can configure the number of processes, threads and iterations for a instance of grinder. You can also configure sleep time between each iteration, which is useful for simulating web service transaction rates.

#
# Example grinder.properties
#
grinder.processes=1
grinder.threads=2
grinder.runs=10

grinder.jvm.classpath=build/classes;build/test
grinder.jvm.arguments=-Dpython.home=/applications/jython-2.1

grinder.useConsole=false
grinder.consolePort=6372

grinder.logDirectory=logs
grinder.numberOfOldLogs=2
grinder.logProcessStreams=false

grinder.initialSleepTime=500
grinder.sleepTimeFactor=0.01
grinder.sleepTimeVariation=0.005

grinder.script=src/test/load/grinder/CreateAcceptCompleteScenario.py

To run grinder you simply pass it the property file

  grinder.bat grinder.properties

The next step is to determine whether I can configure grinder to execute the following load test

• 10 users concurrently retrieving work lists every 5 seconds and

• 5 users concurrently creating work items every 10 seconds and

• 3 users accepting and completing work items every 15 seconds

Four years ago a collegue and I created openexec , which was supposed to do just that. This is what the configuration file looked like. (There is even some documentation).

<?xml version="1.0"?>
<OpenExecConfiguration>
  <Scheduler />
  <Logger />

  <Users>
    <User userName="chris" />
    <User userName="jima" maxConcurrentThreads="1" />
    <User userName="jimm" maxConcurrentThreads="2" />
  </Users>

  <Scenarios>
    <Scenario className="SleepExecutable" scenarioName="sleep100">
      <Properties>
        <Property name="sleep.time" value="100" />
      </Properties>
    </Scenario>
    <Scenario className="SleepExecutable" scenarioName="sleep1000">
      <Properties>
        <Property name="sleep.time" value="1000" />
      </Properties>
    </Scenario>
    <Scenario className="SleepExecutable" scenarioName="sleep10000">
      <Properties>
        <Property name="sleep.time" value="10000" />
      </Properties>
    </Scenario>
  </Scenarios>

  <UserScenarios>
    <UserScenario interval="500" delay="0" serialSceduling="false"
      executionCount="20" userName="chris" scenarioName="sleep1000"/>
    <UserScenario interval="500" delay="15000" serialSceduling="false"
      executionCount="20" userName="jima" scenarioName="sleep1000"/>
    <UserScenario interval="500" delay="40000" serialSceduling="false"
      executionCount="20" userName="jimm" scenarioName="sleep10000"/>
  </UserScenarios>
</OpenExecConfiguration>

…the CVS logs show that it hasn’t been touched for 4 years :-(

Related link: http://www.novellbrainshare.com/

Not every Novell employee gets to go to BrainShare, and I appreciate the opportunity as a new employee to come and learn. Herewith, my impressions of the first day.

The overall production is top-notch. It’s obvious that Novell continues to pour marketing dollars into the conference, now in its 21st year. They keynote had the requisite amounts of flashing lights and some technological whiz..

Linux is everywhere. From Novell Linux Desktop to SLES 9 to Open Enterprise Server, Linux is on everyone’s minds.

The hall is saturated with wireless. I’ve gotten a rock-solid connection everywhere I’ve looked for one. Other conferences could certainly benefit from this kind of wifi saturation.

The food is great. Abundant choices in copious amounts, and it can’t be easy to feed 6,000 hungry geeks twice a day.

The attendees are top-notch. At my Birds of a Feather session last night on choosing between J2EE, .NET, and Mono in service-oriented architectures, the small crowd really got into the discussion. It made our jobs easier as moderaters, and I think everyone came away from the session with a better understanding of the issues involved.

It’s easy to see why BrainShare has been the premiere event for Novell customers for over twenty years. If this year’s version is any guide, it will continue to be a great event for everyone involved for years to come.

What have you seen at BrainShare?

Related link: http://wiki.brainattic.info/Whitepapers

Since starting a company last fall, I simply haven’t had time to write whitepapers. Except I did…my weblog.

Another sign of the shifting digital sands. For a small company, whitepapers seem stuffy, overly-formal, and marketroid-speak-laden. I won’t rule out ever writing one, but for now, weblogs are doing the trick. All you need is a collection point to tie together your “greatest hits” from the blog.

Read on: Whitepapers

Do you see weblogs being used increasingly for technical “marketing” tasks?

Related link: http://www-128.ibm.com/developerworks/library/x-xhtmlnk.html

Related link: http://www.opera.com/pressreleases/en/2005/03/16/

Opera, the desktop and mobile browser maker, has just announced it is adding SVG Tiny support to their desktop browser. This is a very significant announcement as this makes Opera the first desktop browser to announce default support for a standard profile of SVG. This announcement also builds on last month’s announcement that Opera was adding support for Ikivo’s SVG Tiny viewer for their mobile browser.

Opera is actually not the first browser vendor to look at SVG. Mozilla have an SVG project that is well advanced already, providing SVG-enabled builds for both the Mozilla suite and Firefox. Also, the KSVG have been working hard on brining SVG support to Konqueror and the whole KDE 3.2 platform. But what makes the Opera announcement a breakthrough is that they’re the first one to add SVG natively, default and with announced compliance to a W3C specification, in this case SVG Tiny. So far both the Mozilla and KSVG offerings are made available as optional packages or alternate builds as development continues.

However, I think it is important to note that this doesn’t necessarily mean a lot of changes in the purely technical front. So far it doesn’t look like Opera’s SVG support is adding anything to what you could already have had running for years with most desktop browsers with the Adobe SVG Viewer. Indeed, Opera supports a (very popular) subset of SVG, SVG Tiny, and apparently only supports SVG as embedded within (X)HTML using the <object> or a similar mechanism. But I can only applaud Opera’s decision and recognition of SVG as a strong standard for the Web.

But these technical considerations cannot cloud this amazing announcement. In my opinion it is still very significant as it ties in with a general renewal of innovation in the browser. 2004 was a big year with the release of Firefox, and recently Apple also brought us Safari and more exposure to KHTML’s rendering engine. I’m waiting to see what impact Opera’s decision to add SVG as a part of their Web rendering platform will have on other browsers out there (although I’m not holding my breath for SVG in IE7).

Well done Opera!

I had a thought about using meta data technology, such as RDF, to promote loose coupling between data and processing element (i.e. the logic that operates on the data). This seems particularly relevant for business processes where the structure of the data (i.e messages) directly influences the coupling between the process partners.
Ideally, you want to be loosely coupled with all your business partners.

When a business process receives an XML message it will more often than not manipulate the message using XPath or XQuery expressions. This sort of manipulation closely binds the processing element to the structure of the data. For instance the XPath expression, /students/student[1]/name, will reference the name of the first student in the collection of students. If we change the structure of the data then we need to go back to the processing element and modify all the XPath expressions referencing that piece of data.

Suppose that instead of directly referencing the data structure we used the associated meta data to reference the information. This level of indirection would firstly bind the processing element to the meta data and secondly remove the dependency between the processing element and the data structure. The consequence of adding this level of indirection is that the underlying data structure is free to change without impacting the processing element.

I am reasoning that binding the processing element to the meta data will yield superior durability and flexibility than directly binding the processing element to the data

I recently got my hands on some photos that were taken during the Tokyo SOA Architect Forum, and Japanese ESB book launch event, which I wrote about here. Here are a few of them –

The crowd listens to simultaneous translators using headphones.

Osaka-San (General Manager of Sonic Japan) and Dave Peter (DP-San, VP Sales for Sonic A-Pac region) do the ceremonial thank you and closing of the event.

The ensuing cocktail reception / launch party for the Japanese translation of the ESB book.

Doing the business card thing while one of the translators stands by to provide assistance.

Osaka-San and DP-San work the crowd.

Group photo with the Sonic Japan team.

Dave

The DigiPen application
In all three cities during last week’s European leg of the SOA Architect Forum series we did a walkthrough and discussion of a very innovative use of a SOA using bleeding edge technologies in a business application–the DigiPen and the ESB.

A Sonic customer, Bendit Innovative Interfaces, did a live demonstration of their application that was built for performing real time toll collection for 1.2 million registered truck drivers in Germany. This is a SOA based application, built using an ESB, that connects, mediates, and controls the end-to-end processes between a simple device that every truck driver has, and the backend toll collection systems that are located in a datacenter somewhere in Germany. At a high level, here’s how it works:

Each truck driver is equipped with a kit which includes a DigiPen, a pad containing special forms, and a cell phone.

Now there are some special things I should mention about the things in this kit. Digipen has a builtin optical reader, which knows how to recognize and record pen strokes on the special paper form. The paper form has a special dot pattern that is barely visible to the naked eye. This dot pattern acts kind of like a bar code in that it is unique for every position within every field on the form, and also unique for every form in the stack. In fact I’m told the dot pattern is completely unique for an area the size of Europe. Using the DigiPen, the truck drivers simply fill in the form with information about the routes that they take.

The DigiPen also has a Bluetooth interface, which is then used to send the form data to the Bluetooth enabled cell phone that is also part of the kit. The cell phone sends an SMS message to an SMS gateway that is the “onramp” onto the ESB. The following figure shows some of the deployment characteristics of the ESB -

Part of the reason that it is depicted using the messaging layer components is to highlite the importance of continuous availability. The circles labeled “P” and “S” represent the primary and secondary message brokers that are used in the Continuous Availability Architecture (CAA).

The following illustration shows the high level steps involved in the processing of the toll collection form, once it arrives “on the bus”.

Here is the “SOA view” of that same high level process. This represents that actual services implemented in the ESB, and the ESB process that coordinates their interaction. What’s more interesting about this SOA view is that some of these steps represent composites of other sub-processes, which can be further broken down into their individual services.

In fact, the whole processing tree can get pretty complex, as hinted at by the following illustration. I think I’ll save that explanation for a more full-feature article when I have some more time to get back to writing.

ROI of the DigiPen application

Alternative approaches to this were considered. An internet login was considered, but deemed to be too costly and impractical for the truck drivers to become proficient at using. Then they considered using dedicated kiosks for toll collection. They installed 1200 kiosks around Germany to support a population of 1.2 million truck drivers. You can imagine how well that worked out. Additionally, a solution based on a dedicated on-board unit consisting of a GPS system and a GSM device. The following figure shows a relative comparison of the costs per truck driver for each approach.

In summary, these are the benefits of the toll collection system that is based on DigiPen,SOA, and ESB –
- No wait time at terminals
- No installation and maintenance cost
- Spontanuous, flexible route planning
- Independent from GPS
- Registration number authorizes trip
- Optional copy of form via eMail
- Real-time monitoring of toll costs

In the end, the DigiPen solution was the most cost effective and easiest to learn and use….Just write!

Dave

Related link: http://www.sdexpo.com/

After a couple of days back at HQ in Boston, I’m back on the road again. This time its off to Software Developers Conference (SD West). I feel like this guy lately . Actually, I’m going farther than him albeit taking a bit longer to do it.

Wednesday afternoon the “other” David Chappell is doing a presentation. Every once in a while we cross paths where we are both presenting at the same conference. I plan on being there during his presentation so that we can both introduce ourselves to the audience. In the past he has done something humorous about how he is the Microsoft guy and I am the Java guy, although lately those lines are getting a little more blurred in that he is the Biztalk/Indigo guy and I am the cross-platform SOA/ESB guy. If only we could get this other guy to show up , then that might be entertaining :). We’ll see how it turns out.

BTW, when I was presenting in Milan last week, one of the audience members told me that David was also going to be presenting in Milan as part of a Microsoft roadshow that is happening in a few weeks. I think he’s following me.

Anyway, my two presentations are Thursday and Friday. If you’re in the Bay Area, stop by the Santa Clara Convention Center and come say hello. Here are the details of the two talks –

http://www.cmpevents.com/SDw5/a.asp?option=C&V=11&SessID=4330

SOA: From Pattern to Production
Speaker: Dave Chappell (VP & CTE, Sonic Software)
Time/Date: Thursday (March 17, 2005) 8:30am — 10:00am
Track: Web Services
Format: Class
Experience Level: Intermediate - Basic familiarity or some experience.
Description: Service-oriented architecture (SOA) represents the opportunity to achieve broad-scale interoperability, while providing the flexibility required to continually adapt technology to business requirements. No small feat, particularly when one considers the extent and complexity of today’s IT environments. As both a technology concept and IT discipline, the challenge inherent in SOAs is maintaining the right architectural approach. If all services in a SOA are treated as interdependent point-to-point interfaces, then the complexity of implementing and maintaining them in this spaghetti-like architecture becomes enormous. The enterprise service bus (ESB) has emerged as one of the first true SOA product offerings, bringing SOA from pattern to production. ESBs provide a framework for building and deploying an event-driven, enterprise SOA, and accommodates the configuration, hosting and management of integration components as services across the business. This class explores how ESBs provide framework for SOAs.

http://www.cmpevents.com/SDw5/a.asp?option=C&V=11&SessID=4255

Delivering on the Promise of Distributed Integration: JBI and the ESB
Speaker: Dave Chappell (VP & CTE, Sonic Software)
Time/Date: Friday (March 18, 2005) 1:45pm — 3:15pm
Track: Java Programming
Format: Class
Experience Level: Intermediate - Basic familiarity or some experience. Description: Application integration is a top priority for companies today. A number of existing and emerging Java and Web services standards are making it more cost-effective for organizations to link applications and services more pervasively than ever before. Previous approaches to systematic integration of enterprise applications—customer coding, integration brokers or application servers—brought with them a high degree of complexity and cost, especially when a large number of applications are introduced across a distributed environment. A new category of technology known as the enterprise service bus (ESB) has emerged to address the need for distributed integration, while readying customer for the shift to service-oriented architecture (SOA). In this class, we will examine how the emerging Java Business Integration (JSR 208) specification will enable a loosely coupled integration model for distributed services, and standardize a key architectural underpinning of the ESB.

I hope to see you there….
Dave

Related link: http://www.sonicsoftware.com/news_events/seminars/soaforum_sonic/index.ssp

a.k.a. If this is Thursday, it must be Milan.

Last week continued with the world tour of the SOA Architect Forum series. Rather than return to the US, I continued on from Tokyo and headed west around the world to Europe. There I met up with Tim Dempsey, Sonic’s VP of Marketing from the US, and our colleagues from the Sonic European team, and conducted a series of SOA Architect Forums in Frankfurt, London, and Milan. The European seminars were very well attended, drawing an audience of enterprise architects in numbers ranging from 80 – 100 in each city.

As per usual throughout this seminar series, we had a local customer present a case study of their SOA project based on an ESB. In Frankfurt, the presentation was done mostly in German (with the exception of my portion). The local German audience seemed to appreciate this, with the exception of the 4 guys who flew in from the Czech Republic. I think they may have been as lost as I was through some of the proceedings :)

Thankfully in Milan we had 2 simultaneous translators which were translating in both directions. When the local Italian presenters where talking, I had the headset on and I could hear the translations in English….Pretty cool).

Anyway…back to Frankfurt. We had a presentation from a customer, Telekommunikation BIT, who provided an overview of their service oriented application that automates all of the tracking and processing of all customs declarations for goods passing through the Swiss Customs office. The name of the project is called e-Dec (e-declarations) Import. The diagram below shows a high level overview of the service components that make up the SOA.

This architecture shows the typical use of an ESB in conjunction with an EJB application server. The ESB deployment is depicted in the upper layer of the diagram, and the application server deployment is in the lower layer. This epitomizes the lines of separation of roles and responsibilities between an ESB and an appserver, each being used for what it is best intended for. The ESB is being used for the routing, transformation, auditing and logging, and queuing of requests, which allows the application server hosted pieces to focus on the business logic that process the import declarations (I would like to say more about what goes on in here, but it was presented in German).

The tall cylinder at the upper left hand of the diagram of ESB layer represents a message queue that is used to queue up requests that can arrive onto the bus in a variety of protocols (HTTPS, SOAP, SMTP) using various formats for carrying the payload (XML, EDIFACT). The requests get routed to an authentication and authorization service (authenisierung/ autorisierung). Depending on the format of the incoming message, it is selectively routed to a data transformation service (konvertieren), then in either case it is sent along the processing pipeline to a validation service (validierung). The message is then sent to an auditing service (archivierung). The message is then delivered to the e-dec Import application via JMS using MessageDrivenBeans into the Weblogic based application. (NOTE: the use of the word IMPORT here is referring to the processing of imported goods into Switzerland, NOT import into the appserver. The connection between the ESB and the appserver is a very natural thing using MDB). Once the processing of the business logic is completed in the appserver, a response message is placed back on the bus, routed to a response generation service (antwort gen), then audited again using another instance of an auditing service. Lastly, the “response” message is placed in an output queue, where it is sent back to the sender. This whole process is controlled by message itineraries.

In all three cities (Frankfurt, London, Milan) of the tour that week, we also discussed an interesting use of SOA, ESB, and the DigiPen to highlight a system built for automating the toll collection for Germany’s 1.2 million truck drivers.

On another note…In each city I usually seek out a local Starbucks because I really like their espresso. Milan has no Starbucks, but it doesn’t need one. That would be blasphemous to the locals. In Milan I had some of the best espresso I have ever experienced anywhere. If you’re into espresso, this stuff has a head of foam on it that can support a sprinkle of sugar for up to a whole minute before it sinks in to form a blend of pure coffee heaven. And its consistently good everywhere, which is usually about every 100 meters that you’ll see some sort of espresso bar.

Fashion Alert: Milan is the fashion capitol of the world. One thing I noticed there is everybody wearing big giant sunglasses. I’m not talking about the kind you see on J-Lo, Bono, Brad, or Jen, but really really big bug-eyed cover-half-your-face goggle-looking things! They’re headed this way, I’m sure. Be on the lookout. If I start wearing these things, somebody shoot me.

On the return flight from Milan to Boston, I was buzzing pretty good from multiple shots of espresso, so they made me sit out on the wing :)

Actually this is a nice picture of the Alps taken with my camera phone through the airplane window. As you fly out of Milan en-route to London you just clear the tops of these things on your ascent. It’s a breathtaking view up close.

Dave

I don’t really subscribe to the distinction between orchestration and choreography. For me the distinction is purely academic and provides little value in the practical world. Stefan Tikov’s blog entry on Choreography vs. Orchestration is a good entry point into the subject.

The two terms have concise and distinct meaning in the English language but they are used very loosely in computer science (particularly in respect to web services and BPMS). In fact there is so much overlap in our definition of orchestration and choreography that it only serves to confuse the market.

There are four specifications, namely BPEL, BPML,. WS-CDL and WSCI, which potentially fall into the orchestration and choreography bucket. From this group of specification I have noted the following, not so interesting point. The Web Services Choreography Description Language specification makes no mention of orchestration but makes extensive use of choreography. The Business Process Execution Language specification mentions neither orchestration or choreography. Finally, the Business Process Modeling Language specification makes a single reference to choreography (via a link to the WSCI specification).

There is so much overlap between the specification that we could merge them all into a single specification, dispense with one of the terms or come up with a complete new term like choreostration, without disrupting the market. In the vendor space no one is currently supporting WS-CDL or WSCI and there are only a handful supporting either BPEL or BPML (actually only one that I know off supports BPML).

In the customer space who is going to purchase an orchestration engine without so called choreography capabilities or visa versa. Maybe those that will buy a workflow system without BPMS capabilities.

I’ve spent a lot of time this week rethinking Microsoft Office 2003’s XML capabilities and figuring out how I feel about them a year after writing a book on them.

When I first saw the demos for Microsoft Office 2003, I was blown away. Word would let me edit XML documents, Excel would give me easy access to both reading and creating tabular XML data, Access would let me get data in and out, and XDocs (now InfoPath) would make it easier for people to create XML documents using forms. Everything looked great.

Since then, I’ve become sadder but wiser. The glossy demos hid an underlying reality which was far less delightful. The marketing story breaks down under any close inspection. While there are useful pieces here, no question, Office 2003’s XML support has to be viewed as a Microsoft 1.0 release, awaiting maturation to a Microsoft 3.0 release, when the parts finally come together.

What’s left me so bitter? A list is probably the best way to outline this briefly, without driving too deeply into a conversation that requires deep understanding of all of these parts. First, the technical issues:

WordprocessingML is a horrible mess.

On the bright side, it exposes everything in a Word document - you don’t lose anything by saving to XML. On the dark side, the structure of what you get is amazingly twisted, and it’s painfully obvious that WordprocessingML (formerly the catchier WordML) is a serialization of internal structures in Word, not an XML vocabulary designed by people who actually care about working with XML.

Word’s XML functionality isn’t integrated with its existing style functionality.

Users, at least some users, understand the style drop-down and applying styles. The XML task pane is new and additional. Creating XML documents in Word that look like you want them to look can require using both styles and XML. My advice to people who want to use Word to create XML: forget about the XML tools, unless you need lots of precise nesting and attributes. Use styles instead, and then extract the information from WordprocessingML. (Yes, I know that’s painful.)

Word’s XML functionality has awful interactivity.

Word’s custom schema XML functionality lets you add XML to documents, but the process for making Word respond to that XML is presently incredibly clunky. Right now, to make Word itself make your styles match your markup, you get to save and close your document, then reopen it. Chapter 5 of Office 2003 XML walks through the current process for making this more automatic, which requires a lot of painstaking coding. There’s lots of potential for creating a better experience with various panes and interfaces, hopefully soon in Visual Studio 2005 Tools for Office, which promises to be a “drag-and-drop experience rather than a code-writing experience,” but that’s still a task for .NET programmers, not the traditional Office power-user approach of using VBA within the application.

SpreadsheetML is incomplete.

Excel’s SpreadsheetML vocabulary is infinitely easier to read and process than WordprocessingML, but it’s also missing lots and lots of things, notably charts and VBA. Save an Excel spreadsheet as XML, and suddenly all the VBA vanishes. Big problem.

Excel lacks an interface for modifying XML schemas that are part of a spreadsheet.

Excel’s facilities for importing and exporting XML through maps and lists are generally magnificent. So long as my XML is roughly tabular, with a fair amount of tolerance for other structures, I can show and edit XML as parts of my spreadsheet. This means that I can write reusable spreadsheets, which, for instance, take quarterly sales data and show it as charts, break it down twenty different ways, and offer projections. At the end of every quarter, I can load a new XML document and everything will instantly update. There’s one big oops, though - if the schema of the XML data structure changes, there’s no interface for changing the maps inside of Excel. Right now, you have to export the spreadsheet to SpreadsheetML, edit the schema in that XML, and then reload. That works, unless you had charts and VBA in your spreadsheet… then it’s time for a lot of reconstructive surgery.

Access supports functionality in the GUI that’s missing in VBA.

The XML functionality is Access is one of my favorite parts of Office. The designers wisely recognized that database columns and XML elements both have names, and that XML structures can easily represent relational tables. They didn’t try to do anything especially funky, but let you get XML in and out. Despite that, there are a few big problems. Access lets you apply XSLT transformations on import and export through the GUI, but not through VBA! I’ve worked out how to apply transformations, but it means serializing a file to disk, reopening and parsing it, and reserializing it. This might not matter so much, except that Access only sees elements, not attributes, and lots of data formats use attributes….

The Web Services Toolkit is, well, lousy.

Word, Excel, and Access all rely on the Web Services Toolkit if you want to create web services-based apps from VBA. It generates lots of code, but isn’t very sophisticated, even about SOAP. As Jeff Webb has pointed out, and as we demonstrated in the book, doing REST (basically really simple XML over HTTP) from Office is a lot easier, even though you have to write your own code. The documentation doesn’t tell you that, of course.

InfoPath is its own universe.

InfoPath is built on an ingenious XSLT hack, and reuses lots of existing Web development technologies, which is a good thing. Unfortunately that also makes it very different from the rest of the Office suite, which generally use VBA. It’s also pretty different from the .NET tools Microsoft is developing in Visual Studio Tools for Office.

Outlook? PowerPoint?

There isn’t much to say about Outlook and PowerPoint in XML.

Visio is a bold SVG loner.

Visio 2003 came with amazing SVG support, but it’s pretty much the only Microsoft product that’s noticed SVG. Cool, but alone.

These technical problems are compounded by a number of marketing problems:

Office Standard vs. Office Professional vs. Enterprise vs. standalone apps.

For whatever reason, Microsoft decided to split the XML functionality depending on which version of Office people bought. Office Standard includes a version of Word which saves and opens WordprocessingML but doesn’t let you work with the custom schema features, and a version of Excel that supports SpreadsheetML but lacks the easy XML import and export. The extra features are there in the Professional, Enterprise, and standalone box versions. This makes it really really hard to talk about these things as Office features, since lots of people will try them and find they just don’t work. I was startled when my new O’Reilly laptop arrived with a copy of Office Standard, and suddenly some pieces I’d written with Professional didn’t work.

InfoPath is rare.

InfoPath doesn’t suffer from the Standard/Professional distinction, but instead it only comes with the Enterprise edition or as a standalone product. Because Microsoft chose to combine the forms designer and the data entry side into one application, you get to buy a license of InfoPath if you ever want to fill out InfoPath forms. I’m kind of happy Microsoft hasn’t offered a free form-filling application, as this gives Mozilla more time to work on XForms, but it does make it hard to publish on InfoPath. The only market for InfoPath is inside the enterprise.

.NET and VBA are two different cultures.

I could, if I wanted, use .NET tools and wrappers for Office to use .NET’s more powerful web services and thereby avoid the Web Services Toolkit for VBA, but that’s a huge leap from the VBA that I’ve been using in Office for years. I also don’t know how many .NET developers are particularly familiar with the Office object model.

It’s harder to use than the demos suggest.

Microsoft’s put a lot of effort into marketing this functionality, but hasn’t spent much time dwelling on how hard it is to use. It’s hard to sustain the excitement generated by watching a demo when you find yourself plunked into WordprocessingML and the badly-named Smart Documents API.

Who needs this again?

In general, I have a hard time seeing who the specific market is for these things. The XML import in Excel - sadly turned off in Office standard - seems like something that could be helpful to spreadsheet developers at every level. I have a hard time seeing people spending the time to create Word ’solutions’ using XML, but if that was an order of magnitude or so easier it might be something that could reach from the enterprise to small businesses. I once had hopes we could use this stuff in our own (O’Reilly publishing process), but the complication level quickly dashed that. I guess it’s fair to say that the Word XML features are all aimed at a market of people who have tremendous patience.

Schema ownership questions remain.

Reading things like this leave me wondering how safe it would be to build applications around these schemas. My guess overall is (and has been) that it’s pretty safe, but it’s still a higher barrier than I’d like to see, knowing how paranoid corporate lawyers can be. A simple statement that “we reserve the right to change these schemas but will never ever sue anyone for using them however they like” would make these things easier.

WordprocessingML is a mouthful.

It was a lot easier to say WordML.

All of these are certainly fixable, and there’s reason to hope for improvement. I probably should have know better than to put so much time into what is effectively a 1.0 release. Office is an especially tough piece of software to change deeply, after a few decades of development, and it’s not that surprising on reflection that this first generation of XML support is a thin candy shell around an existing product.

Can you do interesting things with Office XML functionality? Sure! Is Office’s XML functionality a powerfully useful feature? Only occasionally.

I’ll keep reporting on this as things improve. Hopefully that’ll make for some brighter blog entries.

Have you managed to surpass these problems and do interesting work with Office XML?

Related link: http://business2.blogs.com/business2blog/2005/03/is_firefox_doom.html

B2Day asks Is Firefox Doomed? noting some published thinking on the matter by a group of Wharton School of Business professors that conclude Microsoft will win out because of inertia.

I certainly see the point the professors make and think it is a real possibly of what will happen. Don't get me wrong. I love Firefox and hope to see it succeed for many reasons. That is not the point of this post though.

What is the point is how inertia is a key factor of any major software shift in the mainstream market, one side that this assessment overlooks.

The fact is inertia can be as much a hindrance for Microsoft as it is a help. Microsoft has painted themselves into a corner defending in the legal courts and the court of public opinion, stating that the browser is part of the operating system and they were not using their monopolistic power to destroy Netscape, but to innovate. So Microsoft dispatched Internet Explorer as a stand-alone product.

In keeping with this position, the Internet Explorer Microsoft has promised to release in beta form this summer is for Windows XP customers only. Those running Windows 2000 or other versions of Windows are (currently) out of luck. So the adoption of IE7 will be dependent upon operating system upgrades and existing Windows XP users to download and install it.

Our primary machines at home and work are Apple Powerbooks, however I know many people running exclusively Windows and the fact is they dread, kick and cuss at the prospect of upgrading. Not that they dislike or have a problem with Windows XP – well actually they do. The cost in terms of time and money. Much like a trip to the dentist or the department of motor vehicles, most will wait till they have to do it or the pain is too great before doing a Windows upgrade.

For example, since we mentioned dentists, mine was recently telling me that he has to switch their machines to Windows XP soon because the company whose practice management software they use is dropping support for all versions of Windows except XP by the end of year. Upgrading to the latest OS will cost him $200 a machine that he figures it is cheaper to replace the aging though still productive PCs around the office.

Since he runs a thriving business he can absorb the cost of new PCs and Windows upgrades. For the occasional home user this is a big deal and one they don't see the value in until, in their mind, they have to do it.

And this is where I see the inertia potentially going either way or being a wash out in the matter. Unless Microsoft eats their own words and declares IE as a standalone product again, their new browser's impact will be limited and FireFox has its opening to continue to outpace IE and grow its user base despite Microsoft's latest efforts.

What part will inertia play in Microsoft’s renew browser efforts?

In a general sense, a piece of software is made of algorithm + data and executes on a processor. Additionally, algorithm is a type of data, with a further constraint that it is read only. Both data and algorithm share the characteristic of mobility, whereas processor is fixed in space. By mobility I mean that in a networked environment we can ship the data to where the algorithm resides and execute (i.e. Seti@Home), or we can ship the algorithm to where the data resides and execute (i.e. Mobile agents), or we can ship both the data and algorithm to the processor and execute.

In my networked grid of 100 processors I charge per CPU cycle for work. As master of this domain there is nothing that would bring me more pleasure than looking at the management console and seeing 100 processors CPU bound 100% of the time. So, if there is work to be done and there are spare CPU cycles I would want to employ any of the above techniques to utilize the spare capacity.

SEDA (Stages Event Distributed Architecture) is a SOA, which promotes some of the packaging and distribution principles discussed above. The Enterprise Service Bus (ESB) also exhibits much of the required functionality. The architecture and design of the microprocessor with pipelines, caches, load/store unit, arithmetic logic unit, also provides inspirations by abstracting the micro architecture to the level of algorithm, data and processor.

The way we construct software, the abstractions we use and the architectural constraints we embrace (i.e. stateless interactions, context-free services, mobility) will ultimately determine the performance, scalability, usability and maintainability of the piece of software.

Related link: http://www.sonicsoftware.com

I recently read in an airplane magazine that its becoming common practice for technology geeks to use their blogs to talk about new product releases for their respective companies :), so here goes mine –

Sonic Software has released version 6.1 of its ESB-based SOA Suite product line. Sonic SOA Suite 6.1 adds a new Collaboration Server, Database Service, improved service invocation model, a new deployment tool, and a new SOA Workbench to bring it all together.

Here is an excerpt from the press release that sums it up pretty well -

“Sonic has extended its ESB-based SOA infrastructure line with two new products: the Sonic Collaboration Server™ for managing partner interactions; and the Sonic Database Service, which simplifies access and re-use of relational data sources. A new deployment tool simplifies the entire SOA lifecycle, making it easier to deploy and continually upgrade services and processes across large-scale deployments. Enhancements to Sonic ESB® and the Sonic Workbench™ in the 6.1 release increase SOA development productivity with a new uniform service invocation model that allows services of any type, from Web services to databases, to be easily incorporated into the ESB. “

The new Sonic Collaboration Server is a B2B server which extends the capabilities of the ESB to integrate external business partners using B2B protocols and Web services standards. It allows an administrator to fully manage the partner interactions, including using partner-specific protocols and security checks. It lets you add or change partner configurations without bringing down the server. The Collaboration Server is itself configured as a service on the bus, which allows it to enjoy full deployment flexibility of complete location transparency, scalability, and remote configuration and deployment through the ESB management infrastructure. Collaboration Server has a flexible architecture which separates out the collaboration management from the protocol management, which themselves are divided into separately deployable services that talk to each other through the ESB. Collaboration Server comes with built in support for B2B protocols such as ebXML and RosettaNet RNIF 2.0. Collaboration Server shares some of the capabilities of the Sonic Orchestration Server, which enables stateful collaborations and document exchanges that can persist beyond a single request/response operation. Collaboration Server also provides a graphical environment for modeling the process which fulfills your business’ role in conversations with partners using UML modeling notation (swim lanes).

The primary use case for Collaboration Server is to sit at the “edge” of the enterprise and handle partner collaborations, but it could also just well be used for message exchange and collaborations between departments and business units. The need for such flexibility in those diverse use cases has been a topic of discussion that has come up quite frequently at the SOA forums recently.

The new Relational Database Service provides the easiest and fastest way to integrate relational data sources into an SOA. We were able to leverage the the industry-leading drivers from our sister-company, DataDirect (http://www.datadirect.com) to provide out-of-the-box connectivity with Oracle Database, Microsoft SQL Server, Sybase Adaptive Server, Informix Dynamic Server and IBM DB2 UDB, in addition to any JDBC-enabled data source.

The Sonic Database Service lets you encapsulate and re-use relational data sources as services in a SOA. The database service eliminates custom coding, and makes it easy to configure and execute queries, updates and stored procedures as broadly-available services on the bus. The database service automatically populates SQL query parameters based on incoming XML message parameters, and transforms result sets back to XML. And because it’s a fully fledged ESB service it can enjoy remote administration of database connections and configuration through the Sonic Management Console. There is also a query editor support as part of the latest Sonic SOA Workbench that lets you create, test, and debug SQL queries, map XML messages to SQL query parameters and map result sets to XML messages.

Service versioning and migration - There is also a new graphical deployment tool which simplifies process and version control during migration of ESB services and processes from development to test and deployment. This means you can group together a process definition with the individual versions of associated services that it calls, and treat that as a project that can be version controlled, and migrated from a development sandbox, to a testing sandbox, to a deployment environment. Cool stuff.
This has been another popular subject of discussion in the SOA Architect Forums, identified as being a leading issue in what’s required to make a distributed SOA viable.

Improved service invocation model. We have improved the overall experience in invoking external web service calls from the ESB, and from the Sonic Workbench. The Sonic Workbench and ESB provide a uniform model for natively invoking services of various types, such as Web services, databases or intermediary services, allowing all types of services to participate as “first-class citizens” in the enterprise SOA.

The new Sonic SOA Suite, which combines all these things, is an ESB-based distributed services platform, providing business process management and operational visibility across the enterprise. Sonic SOA Suite extends Sonic ESB with service orchestration, XML-based operational data management, and fully integrated access to relational data sources. A cohesive solution to the challenges of broad-scale business event and process management, Sonic SOA Suite lets you dynamically manage, mediate and orchestrate services in a unified SOA framework.

The Sonic SOA Suite is a bundling that includes these products
• Sonic ESB
• Sonic Orchestration Server
• Sonic XML Server
• Sonic Database Service
http://www.sonicsoftware.com

Dave

I had a couple of free hours on my final day in Tokyo. I ventured out to visit a Buddhist temple called Sensoji (a.k.a. Kannon Temple) in the Asakusa part of the city.

Outside of the temple there is a marketplace area,

where you can buy lots of low-cost souvenirs.

Just outside of the main gate to the temple/marketplace area, there is a real Starbucks on the main street about 1 block to the left of the gate.

Inside the main gate area, there is a genuine take-your-shoes-off and sit-on-the-floor restaurant just to the left of the gate, where they served a really good assortment of sashimi and tempura.

This was accompanied by some of the best tasting miso soup I have ever had. The place looked fairly rustic inside yet the waitress, who was wearing a traditional Japanese Kimono, took the order by pressing buttons on a handheld device that she carried with her :) Pretty high tech for an ancient Buddhist temple.

Right next to the temple there is what is called the “5 story pagoda”.

The pagoda is a shrine built to hold some of the actual ashes of Buddha. The story goes that some 1500 years ago, when Buddha died, his body was cremated and his ashes were divided up into 5 containers which were distributed to 5 places across Asia. This is one of the 5 places. The five stories of the pagoda signify earth, water, fire, wind, and sky.

Shabu-Shabu

On my final night in Tokyo, I went out for a farewell dinner with a few people from Sonic Japan. We went to a place that served Shabu-Shabu. Shabu-shabu is kind of like a fondue setting, where there is a heated pan of boiling water at the center of the table. The servers bring out a large plate of thinly sliced beef, and an assortment of raw vegetables that resembled bok choy, along with some mushrooms and sliced sweet potato.

Shabu-shabu in Japanese means “swish-swish,” referring to the swishing action when you cook a very thin slice of beef in hot water. The servers place the raw food in the boiling water a few items at a time, and then deposit freshly cooked food in one of 3 bowls of different sauces that you have in front of you. They just keep doing this until you can’t eat anymore. At the end of the meal, the water they are cooking everything in has become a tasty broth, to which they add some noodles and other items and spices of untold delight and serve it to you as a soup.

I have had this type of food once before where you cook it yourself, but never like this where they did it all for you. The food was very delicious, and the food servers were very prompt and courteous.

Dave

While in Japan I did manage to do a little sightseeing before the start of the workday. Dave Peter, our VP of Biz Dev who was traveling with me from the US, and I were both still on US time and therefore wide awake at 5:00 am Tokyo time. If you’re ever in Tokyo and you are up at that time, you should visit a place called Tsukiji (Skee-jee). Tsukiji is a HUGE fish auction/marketplace down at the waterfront. A large proportion of the worlds fish supply goes through this marketplace everyday.

If is literally acres and acres of fish that is fresh off of the boats. Some is fresh, some frozen. Some is still being kept alive in tanks of water. Most of it is packed up in stackable styrofoam crates to keep it fresh and wet. There are huge tunas all over the place. Some are being carved up into steaks right there. There are also all kinds of other fish – clams, mussels, scallops, shrimp, eel, octopus, squid – you name it. Right outside of the main marketplace area, there are a number of little sushi bars where you can get some of the freshest sushi you could ever have.

It’s a rather busy place, and messy too. There are local workers hosing off bloody fish and there is melting ice all over the place. I wouldn’t recommend going down there in your tux or coctail dress after a night of dining and dancing. There is a chance you might get a little dirty.

On the way back to the hotel, Dave P (a.k.a. DP-san) decided we should be a little adventurous and try the subway. We took a cab from the waterfront to Shinjuku station. That was quite an experience due to the enormous crowds. There are uniformed staff members standing on the train platforms whose primary job is to push people into the trains in order for the doors to be able to slide closed properly. It was only a few stops from Shinjuku to Akasaka, where the New Otani hotel is, and we were back to the hotel by 8:30 am to start the workday (after a short WALK to the Sonic office :). It was enough to get a feel for what its like to be a daily commuter in and out of Tokyo. Many Japanese business people ride the commuter rails in an out of Tokyo every day for up to a 2 hour train ride each way, while packed in a standing room only train car. Most of the ‘real’ business gets done at night, when business people go out to dinner with their prospective clients, which ends in taking a long train ride back home at 11:30 at night or so. Such is the life of the Japanese business person. I suppose this is not unique to Tokyo. There are people who do the same between Manhattan and New Jersey every day. Perhaps I’m spoiled with my 45 minute car ride to the office, or my hour+ Harley ride into work during the summer. I should consider myself fortunate.

BTW, there are a fair amount of Starbucks installed around Tokyo, but none of them seem to open up before 7:30 or 8:00 am. What’s up with that?
Dave

Related link: http://jcp.org/en/jsr/detail?id=208

The JSR 208 EG has just released a public draft of the JBI spec for review. The 30 day review will close on April 4th with the last seven days also being a JCP EC ballot.

The full spec can be downloaded from -
http://jcp.org/en/jsr/detail?id=208

Here is an excerpt from the spec which describes the purpose of JBI –

“Enterprise application integration (EAI) and business-to-business integration (B2B) solutions have traditionally required the use of non-standard technologies to create functional systems. This has required end users to either “lock in” to a single vendor of such technologies, or create their own. Each approach has disadvantages. No single vendor can cover the vast functional space of EAI and B2B (consider the thousands of applications and protocols to be supported). This leaves users of integration technologies in the uncomfortable position of selecting less than ideal solutions to their integration problems, and paying dearly for them.

Java™ Business Integration (JBI) seeks to address this problem by creating a standards-based architecture for integration solutions. This infrastructure allows third-party components to be “plugged in” to a standard infrastructure, and allows those components to interoperate in a predictable, reliable fashion despite being produced by separate vendors. It is anticipated that this ability to interoperate will create a multivendor “ecosystem” which will give rise to large pool of integration-related technologies that can be sourced by end users. In addition, this ecosystem will foster new innovations in integration technologies, since it will permit innovators to concentrate on a particular technology or problem area, without having to worry about providing all the other pieces needed to build a complete integration solution.”

I would also like to add, as we at Sonic are often fond of saying, that JBI will help drive the shape of ESBs much like the EJB spec drove the shape of appservers in the late 90’s. JBI will add an additional degree of standardization and pluggable interoperability to a key architectural component of the ESB, which is the ESB Service Container.

For more information on the synergy between JBI and ESB, here are a few links to articles –

SOA Integration Java Specs Due Midyear
Integration Developer News
January 18, 2005
http://www.idevnews.com/IntegrationNews.asp?ID=150

The New Integration Standard: How Java Business Integration and the ESB Will Truly Standardize Integration
Business Integration Journal
http://www.bijonline.com/PDF/Chappell%20oct.pdf

For those who like to listen and watch a narrated slideshow rather than just read, here is a replay of a session I did at JavaPro Live! on the subject -
http://www.ftponline.com/javapro/2004_10/online/chappell/

For those of you who have been following along on the progression of the spec, here is a summary of changes made to this specification since Sept 2004 when we published the Early Draft Review –

- The Normalized Message Service has been renamed Normalized Message Router (NMR). The messaging model has been reworked to allow bindings and engines to work as service providers, consumers, or both. There are no longer separate DeliveryChannel types for bindings and engines.
- The management chapter has been rewritten. It previously used a use case-driven set of illustrations of system management. It now uses functional definitions of the JBI-provided management pieces, and how they interact with management tools as well as the JBI components being managed.
- The three separate framework chapter have been consolidated into one, reflecting the convergence of bindings and engines in the NMR.
- The the separate binding and engine packages have been combined into one javax.jbi.component package.
- A new “component” interface has been added, and the component-provided interfaces refactored to use it
- All JavaDocs for all JBI-defined APIs and SPIs are included in this specification.

Cheers,
Dave

Interesting and concise post by Grady Booch on SOAs (posted some months ago) and while I share Grady’s views on most items, especially that SOA’s are underpinned by good component-oriented aerchitectures. I don’t particularly agree that services are most applicable to large grained/low frequency interactions. Grady’s point on large grained interactions through course grained service interface is very valid but his point on low frequency interactions goes against current trends. Google, Amazon, Yahoo and eBay all seem to have developed highly available and scalable platforms to support high frequency interactions.

Good software architecture is part of the answer but it is the combination of good software and hardware architecture that underpins the success of these new platform companies.

Related link: http://www.gazettetimes.com/articles/2005/02/27/news/opinion/edit07.txt

I’ve been working for Novell for about six weeks now, and it’s been quite an adventure. Here’s a summary of what I’ve seen so far:

I was in Minneapolis in late January and early February. When I tell them that, people tend to pity me, but in reality it wasn’t very cold. In fact, one day it was colder when I left Atlanta than it was when I arrived in Minneapolis.

In Minneapolis, I worked with an existing team that was designing an Identity Management-based service-oriented architecture for a very large company. Quite interesting stuff, and it helped show me that I still have a lot to learn about IDM. Unfortunately, that project was put on hold for a while, so I only got to spend two weeks in a city that I’d like to get to know better.

After a week at home doing administrative tasks, it was off to the home office in Waltham for advanced JBoss training. There I met some fellow Novell consultants, and learned about how JBoss is put together under the covers. Interesting stuff, though it could have been more hands-on for my taste.

While I was in the Boston area, I spent a day at LinuxWorld Expo, where I finally got to meet Erik Dasque and Miguel de Icaza in person. I’m sure we’ll be collaborating more in the future as I help Novell push Mono out to our clients in the field.

Speaking of Mono, I also travelled to Omaha this week to do a one hour presentation on Mono to a very large financial services organization. While I’m not sure they bought into it, it was a good chance to work on my presentation chops, and to see what the customers really think of Mono. There were some really good questions about moving to Mono from J2EE and/or Microsoft .NET.

Omaha was a very interesting city. From the architecture, I would guess that the city enjoyed a boom in the 1950s. Many of the buildings are faced with what I suppose is native rock; reddish-colored, and with interesting veins. Access to the city from the airport was very easy, just one long, straight, well-maintained road.

In between travels, I’ve contributed to a couple of proposals, including another SOA for a government agency and a couple of desktop Linux migrations.

In March I’ll be going to Novell’s annual BrainShare event, where I’ll be co-chairing a BOF with my boss, Gary Ardito, on the subject “Service Oriented Architecture: .Net, Mono and J2EE - Co-exist or Pick A Path?” The questions from that client in Omaha gave me some ideas for the panel.

I’ve got some other leads for future work: a state government agency, a large entertainment company, a Canadian government agency, some firms in California. All in all, it’s been a good start to my new job, and I look forward to more frequent flyer miles.

Where will my work take me next?