This series of blog entries makes an argument for a new emphasis on dynamic media as presenting a unique opportunity for client side Java, maybe one of its last best chances. Act II will consider current Java media offerings and the ideas behind them. Act III will take the best of these to advocate a new direction for Java media, and attempt to figure out who’ll pay for it.

This set of blogs started as ideas for a session I’d hoped to do at the Java Posse Roundup, which I unfortunately won’t be able to attend, and which I then turned into a JavaOne BoF proposal that I believe has absolutely zero chance of being accepted (”Hi, Sun. I think you guys suck at media. Do you mind giving me a pass to your conference so I can tell that to all your attendees?”). I’ve also been studying Flash and trying to port a C-based tween sample to QuickTime for Java, so some of that stuff is heavily on my brain right now.

As I’m writing this, my four-year-old is on the other computer, touring Noggin.com Go take a look at that site’s The Schmancy Smashup Game, a Flash game in support of The Upside Down Show. In this game, you pick one of three videos, and then use a simple in-line paint application to create a bitmap image:

Once you’ve done this, and you add a sound effect to be played at key times in the video, you can watch your video, with David and Shane playing with the object you drew:

Did I say this was a game? Sure, but it’s something else. It’s a multi-track video editor. With tweens! Your image moves at key points in the video, presumably by being added as a sprite or a second video track (with only one video sample), with its motion sync’ed to the existing movie by means of a tween track, which interpolates movement (or other transformations) between specific locations at specific times.

Is this particularly special or difficult? No, it’s a typical, if quite polished, Flash app. But I do think it gives a taste of what’s coming when we start to apply the ideas of Web 2.0 to dynamic media. One of the key concepts of Web 2.0 is “the read/write web”. In other words, if Web 1.0 media was just about watching, Web 2.0 should be about creating, modifying, personalizing, and sharing media. Web 2.0 media is “rip, mix, burn”.

It’s in this context that we need to understand and consider the prospects for Java Media.

Why Java Media? Oftentimes, this discussion is carried out by developers who are already working in Java, and start thinking about how they’d like to have better media capabilities than the platform currently affords. I think this is the wrong way to frame this discussion, because it presumes a ubiquity of client-side Java that does not in fact exist. It’s more useful to turn the question around: we know we live in a world that is inundated with media… so what role can and should Java play in helping us create and consume media?

Should it have a role at all? Is there anything about Java that makes it particularly well-suited to address media problems?

Consider some of Java’s plusses: it’s cross-platform, which should make it advantageous to move media applications across platforms alongside your media, or to make media capabilities (playback, capture, editing, export) ubiquitous. This should be useful to media device manufacturers (there are a lot of them!), since any needed applications to enable their devices could be written to a Java Media API and be available on multiple current and future platforms. Java is intrinsically networked (good for distributing media files or streams), is free (as in beer and soon as in speech), and has lots of capable developers on tap. And Java code can run in the browser, as a double-clickable, on the desktop, on the small device, on the server, etc.

Perhaps most importantly, media is complex, and Java is well-suited to complex problem domains. If writing a Java application is hard, maybe that’s not such a bad thing when you’re dealing with concepts that are intrinsically deep and challenging. You don’t, after all, write something like Final Cut Pro with 10 perl scripts communicating through pipes — curly-brace languages have proven themselves as the proper tool for such intricate efforts. I’m making an analogy to Kathy Sierra’s Power versus Ease-of-Use argument here, because I think when you’re dealing with all the challenges and conceptual depth of dynamic media:

• Science
• Persistence-of-vision
• Psychoacoustics
• Information entropy
• etc.
• Theory
• Compression
• Color spaces
• Z-order
• Alpha blending
• Media references
• Timekeeping and synchronization
• etc.
• Craft
• Scale
• Composition
• Lighting
• Pace
• Motion (primary, secondary, tertiary)
• etc.

To put it bluntly, anyone who talks about “just opening a file and playing it” simply doesn’t know what the fsck they’re talking about.

I think that Java may be better suited to media applications than scripting languages that hit their complexity wall sooner, and the C variants that are harder to use and less productive than Java.

On the other hand, there’s an elephant in the room, namely Flash. The use of Flash for web media has exploded in the last year or so, oftentimes displacing the classic Real/WindowsMedia/QuickTime troika even for basic playback scenarios. The reasons are easy to understand: Flash has better cross-platform distribution than all of them, and can can offer whatever level of interactivity the developer chooses to provide, from a simple “click button to play” to more sophisticated uses, as in the Schmancy Smashup.

It may be too late to compete effectively with Flash. But if Desktop Java doesn’t make a stand here, then frankly, where is it going to assert its relevance? Webapps have eliminated many of the use-cases that applets once targeted, and distribution frustrations (among other factors) continue to make double-clickable Java desktop applications a tough sell. Ajax is all but the final insult: at the end of the day, script manipulating UI widgets in a browser isn’t that different than Java bytecodes manipulating AWT or Swing widgets… except for the fact that Ajax is infinitely more popular than any of the Java client technologies ever were.

Some Java pundits assume that Ajax will eventually hit a wall, be unable to deliver a suitably rich user experience, and that will set the stage for a Desktop Java renaissance. I see that as one of four possibilities:

1. Ajax fails not because of richness, but because it just sucks and everybody decides it would be nice to have more than two viable web browsers. This puts everything in play, even the old reload-the-whole page model.

2. Ajax continues to evolve, through improvements to JavaScript and the browser DOM, and keeps up with user expectations. Hey, <canvas> is in HTML 5, so anything’s possible in this world.

3. Ajax hits a wall, and rich Java clients gain in popularity. This is the scenario that makes us Java partisans feel warm and fuzzy. Still, nobody has figured out what form these Java clients take: applets (which the market has already rejected), Java Web Start, or something else. For those of you who said Web Start, do let me know how cleaning up all those .jnlp files (or the new apps in the Start menu or /Applications) works out.

4. Ajax hits a wall, and a rich client technology other than Java takes over.

Some other cross-platform client technology? Who else is in the running? Well, obviously Flash again, in various forms of Flash presentations making remote calls (Flex, Laszlo, etc.). Competing against both Ajax and these Flash-plus-remoting approaches, it’s not hard to see Desktop Java’s opportunities being almost totally eclipsed.

Deliberate pun, of course. It’s utterly damning that the two most prominent Desktop Java applications are developer tools — NetBeans and Eclipse — as if there’s no value in Java applications beyond the audience of fellow Java developers. The internecine battle between SWT and Swing has been a disaster for Desktop Java, dividing the community into a patently useless battle of hollow evangelism and feeble name-calling. NetBeans or Eclipse, you ask? A pox on both your houses! All this time that we could have spent writing apps that matter to real people — the apps that are instead being delivered as native double-clickables, or with Flash — we’ve spent in a ruinously expensive corporate pissing match, motivated by the wild goose chase of native widget fidelity (whose value is utterly repudiated by the popularity of the web, Ajax, and Flash, none of which resemble their host platforms in the slightest).

Where are the cross-platform device libraries that would liberate device-makers from having to write multiple drivers (or couple their products to one platform), and liberate users to move devices across platforms? Where’s Java’s support for USB (spec’ed years ago as JSR-80, but only fitfully implemented)? Where’s the multimedia support?

Flash is a more significant rival for Desktop Java than is generally understood. It is quite remarkable that the abililty to create Flash applications is so desirable that Adobe can charge hundreds of dollars a copy for its development tools, while Sun can hardly give theirs away (woo hoo, at least IBM can give theirs away). Why? Largely because it handles media so well. And this is serving to grow a generation of web-based JavaScript/ActionScript developers who see Java as “your father’s programming language”, much like C++ was for some of us who picked up Java 10 years ago.

The natural tendency would be for Java and its corporate backers (principally Sun) to concentrate on Java’s strengths, on the enterprise server-side, and let the desktop side fall away. That certainly appears to be the current approach, the Rumsfeldian strategy of applying just enough engineers to lose. But it’s a strategy to keep others from stealing Java’s dominance on the server, a strategy that ultimately comes out as minimizing losses instead of taking new ground.

But why should Java embrace media, and thereby pick a major fight with Flash? All online media seems to be going Flash, for the reasons described above: it has superior distribution and superior integration with the browser (which itself is winning as an application distribution strategy over double-clickables). Considering that Flash is a VM-based browser plugin that’s now reaching out to the server with remote calls (did I mention some of those Noggin.com games let you upload your work to Noggin for use on-air? Very Web 2.0), Flash is turning into what client-side Java was supposed to be.

Media is important. Media matters. Media could help make Java matter… to end users! There’s something to be said for attacking the enemy’s strengths, and with Flash, that’s distribution and media support. If we want Desktop Java to have a future, it needs to solve the distribution problem and support the problem domains that people are interested in today, namely media. Java could be better than Flash for this — Flash isn’t as strong on the small device or outside the browser — but Adobe isn’t standing still, and Java is.

Next time: good ideas and bad ideas in today’s Java media frameworks