O'Reilly Digital Media

Related link: http://news.com.com/New high-definition DVDs to use old video technology/2100-10…

This is pretty sad - so they’re trying to get blu-ray vs. HD-DVD off the ground; we missed shipping it for xmas 2005 because of the argument.

Now, Warner is thinking of making 9GB blu-ray disks at first instead of 25GB, using Microsoft’s VC-1 competitive codec. (Existing DVD’s already hold 9GB, and some DVD players already play VC-1. But no DVD players natively support HD resolutions of VC-1 on DVD).

And, Sony has proposed to use MPEG-2 (instead of MPEG-4 AVC or Windows Media), since at the 25GB level, you don’t need to use a more efficient codec, you can do good-old MPEG-2 which the studios are all familar with, and the larger disks will hold HD.

On top of all this is the fear of what will happen when these incredibly high-resolution versions of movies hit the market, and consumers arguably never need to buy another version of the movie.

Like Vietnam or another recent war who’s name I forget, the HD DVD wars linger on with no timetable for conclusion…

Related link: http://www.magnetictime.com/

This company uses the same voices (from a European company called Acapela) that I licensed for my in-car computer company. It’s a very good idea; i’ve seen a few implementations but usually they’re too annoying to listen to at length. A good trick is to use a UK English voice if you’re american, or vice versa - then, the roboticness of the voice fades even more and your email sounds like a BBC broadcast.

Related link: http://emusician.com/web_clips/web_clips_december_2005/

The December 2005 issue of Electronic Musician includes my cover story, “The Art of Podcasting.” It’s based on tips and insights from several accomplished podcasters and radio producers, including O’Reilly’s Daniel Steinberg and Spencer Critchley. I also describe how Daniel and I created the theme for O’Reilly’s Distributing the Future podcast.

The issue won’t be online until January, but you can see a bunch of my source material now—including essays by Daniel and Spencer, a podcast of my interview with NPR theme composer BJ Leiderman, and links galore—at the EM site. (Scroll down to the heading “The Art of Podcasting.”)

What’s on your podcast?

Related link: http://www.pspfanboy.com/

Weblogs, Inc. just launched PSP Fanboy and yours truly is blogging for the new site. Here’s my first post. Check it out if you get a chance. Should be lots of good PSP goodness leaking out from there.

Related link: http://www.mapserverfoundation.org/open_letter.html

This is the year of fulfilled dreams for me. The first one was finishing my book. Today’s announcement brings my count to two!

The creation of a non-profit organisation around MapServer technologies has been a dream of mine for a couple of years. No longer do you need to listen to my perennial pining about the need for an independent, non-profit, support organisation for the MapServer community. It is finally being launched with some strong momentum and promising support. For more details, please read the open letter.

New to the open source web mapping world is Autodesk. They are releasing their new web mapping platform as open source and donating the code to the MapServer Foundation. They are also graciously providing the necessary funding to get the foundation up and running. Welcome aboard Autodesk!

I’m really excited about the potential that the foundation has. For one, I’m looking forward to a third-party organisation that can provide long-term stability to developer and community resources. This will be good for MapServer and the open source geospatial community.

So get into the discussion, sign up to the mailing lists and make your voice heard. What key roles do you see a foundation being able to help fill?

Related link: http://www.microsoft.com/office/xml/covenant.mspx

Common interchange formats allow synergy: you adopt them so
that instead of being tied to the capabilities of one application (or suite) you can shop around different applications and find the best value, capabilities and performance for each niche. Think of HTML editors. It is not too difficult to foresee that at some stage Microsoft will refactor its larger Office applications into smaller applications, notionally tied together by common document formats. Indeed, the new Office GUI revision is entirely predicated on the idea that they cannot sell any more products by adding more features, because users cannot find the existing ones. But festooning Word’s GUI with ribbons and galleries can only get you so far: the fanout is still too great to support convenient access to thousands of functions.

In that respect, I think Tim Bray’s recent call that Microsoft should
>adopt ODF with additions in whatever namespaces are necessary, is quite sensible. The whole issue of good markup languages for word processors is not new: indeed since the late 80s there has been a lot of experience in the merits of WP-centric dump formats versus “pure structure” formats. People interested should google for rainbow DTD for example.

I am not an ODF or O12F partisan. Microsoft is a big ship, so it is more reasonable to expect that they will find it easier to apply more power to go forward in whatever direction they decided to take a few years ago, than to demand them to react with agility. So let them “standardize” their document format through ECMA for what that is worth; the more that they publicly use the rhetoric of standards, the more that customers and developers will demand general adherance to real standards. But I don’t see what they would lose by also supporting enriched ODF import and export too.

So what to make of the
"http://www.microsoft.com/office/xml/covenant.mspx"
>Microsoft Covenant Regarding Office 2003 XML Reference Schemas?

I always try and welcome anything warm and fuzzy!
But I would like to have a few issues clarified,
for us non-lawyers.
First, what exactly is a
>convenant? Or, more particularly, which country
or state’s law of covenant is meant? (Washington State?)
Is a chose in action being granted, and if so with whom or on what, and by which deed?
Due to noxious US proactivism in its “free” trade negotiations, many countries
recognize parts of US IPR law: do they also get the benefit of this covenant? Since IPR law is determined nation-by-nation, does a covenant made in Washington
State cover someone in New Zealand, Canada, Spain, Nigeria, China or Japan?

Does international law need to catch up
for when IPR owners want to give up these ill-gotten
or star-crossed “rights”?
Presumably this is also an issue for Sun, IBM and all the
other large businesses who also see how much runaway
IPR laws can impede their business.

More concretely, should every negotition between nations
on accepting IPR monopolies also incorporate laws for when the owner of the right wishes to accede, covenant away or license away portions of their rights to the public, an increasingly common issue in
this age of open source, open APIs and open standards?

In economics we have a theory called the
"http://en.wikipedia.org/wiki/Velocity_of_money"
>velocity of money. A nation’s prosperity is
not only dependent on the amount of money, but how
much it circulates: if you have the richest country
in the world but can only spend money once every 25 years, you may
as well be in a poor country. In software, the
same principle applies: the “velocity of ideas”.
Software patents effectively reduce the velocity
of ideas, since software is ideas.
An impeded velocity of ideas hinders everyone.

Which word is sillier: festooning, fanout or industry standard?

Related link: http://www.oreilly.com/catalog/wirelesshks2/

Rob Flickenger told me at the beginning of the project to bring this book out that “writing a Hacks book is part writing and part project managment.” He was certainly right.

I’m really proud of this book. We had a great bunch of contributors who have put lots of work into their hacks, and as a result, Wireless Hacks v2 is packed full of really fun and cool things.

A few of the new hacks in the book:

• Broadcast Your GPS Position (Abuse the 802.11 beacon frame)
• Mobilize Your WRT54G with the WiFiCar
• Analyze Elevation Profiles for Better Long-Range Wireless Networking (The newest revision of NoCat Maps)
• Build an Online Community in Your Offline Neighborhood

These are just some of my favorites that we added. However, there is a lot of great material from the first edition that has been expanded and updated as well.

There are also five sample hacks available from the O’Reilly site. Please look over them to get a better idea of what’s available in the rest of the book.

Do you have a wireless geek in your life? Wireless Hacks v2 would make a great Xmas present! You can order it directly from O’Reilly through the link above. The online book retailers don’t have their copies yet… considering I just got my copy on Wednesday, that’s not suprising.

Let me know when you see any of the online book retailers carrying Wireless Hacks, Second Edition.

As I wrote in earlier postings, video enabled iPods and Internet-connected computer screens are certainly not the only venues available for independent filmmakers seeking new and low cost ways of reaching an audience. Nokia and Palm are both touting new mobile devices that are made not only to capture and transmit, but to play back high quality video. On top of that, every time I walk into one of the local outlets of a large drug store chain the person behind the counter is watching video on a Play Station Portable (an entire industry has sprung up to offer people ways of converting DVDs and other video to a PSP compatible format — to see what I mean go to download.com and type in “psp video”).

In many ways, video is the most powerful medium of our time. No offense to video games or the Internet (after all, without the power of the web as a communications medium I wouldn’t be able to share this posting with you) but think of how much time people spend watching TV and how deeply it influences a person’s preferences and view of the world. As an example, a good friend of mine came to the United States as an 8-year-old boy and the first thing he did was sit down in front of his family’s TV to soak up American culture. When it came time for his first Thanksgiving, he not only made sure his mother bought a Butterball turkey, but he made sure he got the drumstick. As he explained to me recently, he never thought about turkeys of any brand until he got here, and then learned about them solely through advertising and sitcoms. That was in 1978, when video stayed in the house. Now video comes with us everywhere we go.

So, now that the medium is becoming even more of a ubiquitous force in our lives, how can you make something that stands out enough for people to watch? My suggestions are as follows:

1. Tell an interesting story. Lots of people spend too much time adding snazzy transition effects or creating dazzling animated titles while they neglect the most important part of their video — the story that an audience is supposed to care about. These days, anyone can pick up a video camera and shoot something. What separates good filmmakers from everyone else is not simply their technical ability but their ability to engage an audience by developing a storyline that people want to watch from beginning to end.

2. Decide who your audience is, and make your editorial decisions accordingly. A video that amuses your family and friends may not be a hit with people you don’t know. It’s fun to watch your friends on screen. However, if your audience isn’t friends with the actors who star in your movie, your viewers may not automatically develop a strong emotional attachment to the people in your film. If your target audience is people you don’t know, it helps if you give them a reason to care about your work.

3. Don’t ignore production values. Audiences spend so much time watching TV and other forms of professionally produced video that they can instantly recognize good and bad production values (for example how good a picture looks or how clear an audio recording sounds). People instantly notice the quality of a production and it plays a large role in shaping their opinions of a person’s work. If you have a really strong story, it can make up for glitches in production (a grainy image or a badly positioned mic) but ensuring that your work looks and sounds professional is always a good place to start, and it doesn’t need to cost more — it just requires some attention and practice.

4. Make something really good. If you go to the trouble to shoot and edit a video, make sure it lives up to your high expectations as a director. Don’t settle for second best, make something you’ll feel good about when people watch it.

And with that, I wish you an enjoyable and safe holiday.

Bon appetit.

Related link: http://www.mtv.com/news/articles/1513022/20051104/index.jhtml?headlines=true

So EA’s Steve Schnur has announced that they are making ‘commissioned’ music from their games available for sale. And, that future games will have these tracks available at the same time the game is released.

Okay, so not really big news. Game audio CD’s have been available for a while now. So why talk about this one? Because with the big money being generated by download music, and EA’s stated goal of claiming some of that for themselves, I worry about the future of audio IN THE GAME. I mean, when there is money to be made PER TRACK of the tunes you commission, then you’re not using them to sell the game so much anymore, but to sell themselves. So it becomes just another co-marketing thing where we end up with songs that have about as much to do with the actual game content as “Into the West” had to do with the “Lord of The Rings” movie.

So follow me here - If a game developer is going to make money off of some name composer’s 4 minute tracks, neatly packaged for CD delivery and download - then is that the kind of tracks we’ll be stuck with in our games? No more interactive music (too hard to render to CD appropriate tracks), no more music that comments on specific gameplay, no more dynamic music driving the action and sucking the player into the emotional undercurrent of the game. Just a pile of songs that someone has built that will work good on the ol’ download site.

Or, perhaps there will be both. We’ll pay a “known” composer a bucket of money to create the 3 or 4 tracks that will be available for download - and bury them in the game somewhere. Maybe they’ll be lucky enough to be title themes, but songs like this just won’t work in long form interactive games. So then they’ll have to hire some “unknown” composer, and pay him a quarter of a bucket of money to write the bulk of the music you actually hear in the game. Basically - if it has no commercial value outside of the game, it’s not worth investing in. There is enough of this feeling in the game audio world NOW that this doesn’t seem so far fetched.

Shouldn’t we be more interested in adding value TO the game, from WITHIN the game? That’s the $64,000 question. Those of us that have been proponents of Interactive Audio have been asking this for quite a while now, so maybe the answer has been there all the time, and we’re just in a state of self-created denial.

Of course, this is all supposition - who knows what this will mean. There is a possibility that nothing about the way game audio is created or licensed will actually change. So, maybe it’s nothing… but it does make me wonder. Do we care about the audio IN the game anymore? Or do we just want a hit outside the game?

Am I paranoid? Or are they really out to get me?

Inspired by
>Kurt Cagle’s live blogging of XML 2005 in Atlanta or perhaps San Diego. Except I wasn’t there, didn’t see it, and am just looking over some slides! Here are my random virtual thoughts, without the benefit of jet lag, and without the distraction of hearing the speaker actually explain their bullet points.

>Sam Ruby’s talk is pretty daft early on. He seems to feel that standards should be made so that people can implement them by telepathy: if a hacker needs to actually pay attention to what a standard says (e.g. read the thing) it is a bad standard.

>For example, someone’s XML Schemas tool does not implement XML Schema’s boolean correctly and converts “1″ to a Java “false”: Sam apparantly thinks that is XML Schema’s fault. Similarly, whenever a parser has something implemented wrong, it is XML’s fault…Many times the same pattern comes up…if someone writes non-well-formed XML and the parser complains, that seems to be XML’s problem…if someone uses incompatable and wrong encodings, XML’s problem not theirs (no mention of application/* versus text/*, hmmm)…that you cannot feed any arbitrary XML to HTML readers seems some kind of shortcoming or gotcha too to Sam…

Later…is Sam pulling a rabbit out of the hat? Ada’s definition of erroneous: interesting idea to apply this to the XML stack. Sam’s example is XML Schema’s datatypes precision requirements. No, he’s completely lost me here: it looks like he is discussing XML Schemas, but he actually discussing Java. I really don’t get what he on about here: I think he is saying that because different platforms use different precisions (and may truncate incoming data etc), chaos is stalking the land and therefore XML is hard.

One good link in the talk was to "http://www.netcrucible.com/blog/Wellformed+RSS+And+RFC+3023.aspx"
>Joshua Allen’s comment on MS and RFC 3023. It is high time that this was explicit, so that we all can build our systems so that text/html is for friendly HTML that kind of muddles through and application/xml is for rigourously marked-up (and encoding-labelled) XML that attempts to prevent bad data or documents propagating and fails early. In that sense, HTML should be easy and XML should be hard.

"http://bitworking.org/projects/XML2005/presentation/atom-publishing-protocol.html"
>Joe Gregorio give astraightforward introduction to Atom.

Good to see the examples use application/atom+xml.
Browsing around…apparantly the schema for the Java web applications >web.xml don’t allow + in a media types, such as
such as >application/xhtml+xml. This must be another example of the programmer not reading the spec for characters allowed in MIME media types; or is it more evidence for Sam Ruby’s case?

I see Josh Lubell from NIST is giving a slide on
standardized naming rules, but I cannot find the slides on Google.
I suppose this is part of their
>XML Schema validation Process for CORE.GOV work (also
in >PDF)

Another paper I would really like to see is
>Analysis of XML schema usage
(Ralf Lämmel and Stan Kitsis and Dave Remy)
which relates to
schema metrics. I presented on my
>Document Complexity
Metric at Cambridge UK last year and
>Sydney this year,
and have been getting good feedback on other factors that
people have found useful. As XML matures, more attention will be paid to metrics.

Interesting to see the escalations as results of an informal
>poll
become treated of as a
>survey in a blog, which is then written about as
an >article. Chinese whispers anyone?

Related link: http://www.keyboardmuseum.org/d_machines/vdrums.html

It’s my 50th
O’Reilly blog (!), so for this nostalgic occasion, I thought I’d
cover something old but cool. Coincidentally, I just stumbled across the Virtual
Drum Machines site, where you can play Flash emulations of dozens of electronic
rhythm boxes.

A couple of them are classics, but most are wonderfully
obscure thrift-store discoveries, unearthed and sampled by Joseph Rivers of
the Audio
Playground Synthesizer Museum. The emulations, apparently built five years
ago, are primitive-looking as well, but I found their fizzy sounds and jerky
rhythms strangely inspiring. Consider the 1970s Conn
Min-O-Matic, with its wacky “Teen Beat”:

…Or the Boss
DR-110, with its classic Roland programming grid:

After you’ve explored the three dozen drum machines, check out the virtual
keyboards, such as the unbelievably grungy Bee
Gees Rhythm Machine, built in 1978 by Mattel:

A note on the site says you can buy a CD containing high-quality samples of
these museum pieces for just $35. I e-mailed Rivers yesterday for more information
but haven’t heard back yet. In the meantime, you could try using a streamripper
application to grab the audio from the online emulations and use it to inspire
your own music.

All in all, a golden opportunity.

Found any other pawnshop prizes lately?

Related link: http://tinyurl.com/d545u

Risk averse game design is the enemy of fun. In other words, it’s the enemy of, um, game design.

Yet we keep hearing that games can only be made with high budgets, and the game company’s responsibility to the high-dollar investors necessetates risk averse policies.

But that will not stand, says I.

Guy Whitmore, who sent me this link, agrees:

“These are the types of avenues that will allow true innovation (and emotion!) into games. A $150,000 risk is easier for a developer/publisher to swallow than a $20 million risk”

The only thing that bothers me in this article is the use of the word “hungry,” to describe the developer willing to produce a low-budget game. “Low budget” does not mean “low-profit” or “low-quality.” Using the word “hungry” to describe somebody willing to use an alternate business model implies that somebody is going to be taken advantage of by Microsoft.

Not that that would ever happen.

I expect that, with or without the “hunger” dynamic, this kind of business model could be sustained in a way that would benefit everybody involved.

Please explain to me again all of the ways in which I simply don’t understand “how business is done these days.” But beware: most of the people who have patiently explained it to me over the last two decades are no longer in the business.

Related link: http://www.ilikejam.dsl.pipex.com/audiophile.htm

This lovely site is a remarkable source, nay, museum, of “distilled essence of the worst” of the audio industry.

Surely, this is what happens when complex brains get inspired, yet go slightly awry. Unless it turns out that these things work.

audiophile museum

example:
Apparently, “OPUS MM unleashes thrilling levels of performance…”. That may well be so, but OPUS MM also unleashes thrilling amounts of cash from your obviously overfull audiophile wallet. Your shiny new speaker cable will set you back a truly outstanding thirty thousand, seven hundred and fifty dollars, and no cents. I’ll say that again: $30,750.00
Cheers to Ramon Hernandez for sending me a link to this. Top work.

Related link: http://www.popsike.com/php/detaildata.php?itemnr=4703870706

O’Reilly readers know almost everything (and some may know considerably more than everything!) so I wonder if anyone can provide any information on a musical question?

I finally found online the cover of the Nigerian LP I grew up with (Mum says it was the first Nigerian long player):
"http://www.popsike.com/php/detaildata.php?itemnr=4703870706"
>Catchy Rythms from Nigeria

On the cover, it mentions a style “Toy Motion” for a song whose rhythm & feel I really like, and I would like to know more about it: in particular, whether it is associated with characteristic
"http://www.pulsewave.com/glossary/i_rhythm_glossary.html"
>rhythms, tempo or instrumentation?

I found a couple of things on the web: Juju “even developed its own semipopular variant, the “Toy Motion” subgenre of the ’50s. Spearheaded by J.O. Araba and J.O. Oyesiku, skilled railway technicians who had seen the world and were described by one admiring rival as “big men with other jobs,” Toy Motion fused palmwine usages and the latest Latin rhythms into a hip, aestheticized, neotraditionalist style favored by the educated. Disdaining praise singing as a form of begging, they thought it their role “to provide philosophical commentary on everyday life, and reveal the misdeeds of flawed characters.”
( "http://www.robertchristgau.com/xg/bkrev/jujurock-91.php"
>Juju Beats)

"http://digital.lib.msu.edu/projects/africanjournals/pdfs/glendora%20review/vol3no2/graa003002007.pdf"
>Old or New: Palm Wine is Truly Sprawling
mentions the use of a Western parade drum instead of
the talking drum. What else is there to it?
(And what became of it? Synchro System?)

Wind me up!

Wow! Big response to my post, made with tongue only partly in cheek, accusing Microsoft of designing its apps to crash on Macs.

Several people commented that they never have trouble with Office under OS X; others say Word crashes all the time for them, others that they have trouble with other apps. klausp contributed what sounds like a useful discussion of MS’s use of CFM for binding under OS X, but it’s beyond my expertise.

I’m sure users’ differencees are due to differing configurations. But in my case I’ve been watching MS apps, especially Word, crash on a long line of Macs going back to the mid-90’s. I’ve gotten into the habit while using Word of twitch-saving every minute or so, or any time I’ve written more than a sentence or two. I find that Word is especially unstable if I try to do anything beyond basic typing, such as using a text box.

Unfortunately, the times when work should be at its best–i.e. when I’m in a state of flow–are the times when I’m likely to temporarily lose the habit of twitch-saving. And that means that I’m likely to have just done quite a lot of work that I’m happy with when the app crashes, thereby driving the potential for frustration towards its theoretical limit.

The sheer efficiency of it leads to “intelligent design”-style thinking: This is too elegant not to have been planned!

Related link: http://www.intel.com/cd/ids/developer/asmo-na/eng/dc/pentium4/optimization/20833…

I found Chandra Mohan Lingam’s good and accessible article ‘HTTP Compression for Web Applications’ at
>Intel’s site. Age unknown. I think it is good material for anyone wanting to develop a critical perspective on the pro- and anti- ‘XML Binary Infoset’ technologies and standard under discussion at W3C.

The highlights are the numbers: compressing static content is a one-off process that more than halves CPU utilization at servers; dynamic compression adds 25% utilization; compression and halves ‘page’ download times. No mention of the cost of decompression at clients, but rather a claim that IO and network bandwidth is the bottleneck. I would be interested to see architectures and numbers to clarify when and how much this is true.

(What is particuarly interesting is the interaction between SSL and compression. In some cases, there was no CPU penalty in having compression and SSL compared to sending raw data; when dealing with a compressed file, the SSL had less to do.)

It strikes me that one good reason for “XML” Binary Infosets to succeed is the ease in which inefficient XML solutions can be deployed: inappropriately long element names, UTF-16 for essentially ASCII data, indented elements causing spurious node creation by parsers, slow and bloated libaries, and disabled compression. The XML guiding goal of Terseness is of minimal importance is a statement about layering, not a statement of general denial. XML evangelists may need to spend more time promoting an awareness of how and when XML can be efficiently processed or transmitted rather than kneejerking about the evils of binary. The present success of XML is so strong and deep, I think Binary Infoset can only gain ground in people who currently actually cannot use XML (for all sorts of good reasons); the people I meet really would prefer to stay with XML if they could.

Related link: http://www.oreillynet.com/pub/wlg/8433

It has to be deliberate: Almost nothing crashes on my Macs except Microsoft products, which do so frequently and with gusto.

Word routinely dumps and throws away my work in progress, obliterating rescue files on its way out. Powerpoint? Luckily I don’t use it much anyway, because getting it to maintain its shakey sense of self for more than 5 minutes is a trick.

At least with OS X it appears that MS’s ability to screw up other apps is limited. Used to be that installing Office on a Mac was like giving it an infectious illness.

I just want someone who knows to fess up: It’s part of the spec, isn’t it?

Update: There was so much response to do this topic that I did a Part 2.

Related link: http://www.karma-lab.com/vp/

This week at the O’Reilly
Digital Media site, we ran a behind-the-scenes
tour of Korg’s luxurious new Linux-powered music synthesizer, the
OASYS. Due to space restrictions, though, we couldn’t cover one of its
most impressive features: the brilliant KARMA interactive music generator. Fortunately,
KARMA inventor Stephen Kay uploaded three new demonstration videos to his site
today.

You can access them on Kay’s video
player page. (True to form, he taught himself Flash so he could build interactive
demos such as this.)

The videos are derived from a performance and tutorial Kay did at last month’s
Music Player Live show. My favorite is the third, “Guitar Strumming,”
but if you haven’t tried KARMA, start with the second, “KARMA Intro/Discussion,”
to get acquainted. As Kay plays the keyboard and manipulates the OASYS’s numerous
knobs, buttons, and sliders, the music reacts to what he’s doing in subtle
or dramatic ways: Sliding a finger across a ribbon controller triggers a flurry
of guitar samples, emulating a strum. Holding a chord with the left hand while
playing random keys with the right simulates fingerpicking—the right hand
controls the timing, tone, and volume of notes based on the chord held by the
left. Other demos, such as the excellent “Citadel in Space,” showcase
complete interactive backing bands. One player compared KARMA to having a band
member who always suggested cool ideas.

Inventor Stephen Kay demonstrates the KARMA Manual Advance feature with
a moving performance of a guitar sound.

Kay, who has also created some impressive demo songs for other Korg keyboards,
started developing KARMA—the Kay Algorithmic Realtime Music Architecture—a
decade ago to extend his own musicality. It first appeared commercially on the
Korg Karma keyboard in 2001. (See
my review.)

Although the Karma was embraced by adventurous musicians such as Herbie Hancock,
its hardware was based on Korg’s entry-level Triton synthesizer and had
some shortcomings. Then, mere months after the Karma came out, Korg released
another inexpensive Triton, sadly sidelining the Karma.

So it was exciting to see the return of this deep, amazing technology in a
high-end synthesizer. I sure can’t afford the $8,000 OASYS, but I look
forward to playing the more affordable instruments Korg will surely spin off
from it, including, I hope, a Karma II. To hear why I’m so hopeful, just
check out the OASYS KARMA videos.

If you can diagnose this, I dub you a Hero of the XML Revolution. Hint: it is probably the most common problem for English-language XML documents

This story is true; guess what the problem is

• You have an XML file “x.xml” on your file system, with an XML encoding declaration saying UTF-8. The file is not XHTML, just some home-made elements.
• You are demonstrating your spiffy XML application to a hushed crowd.
  When you open the xml file in IE, it stops and gives a cryptic error message about a particular location. This is severely embarrassing and so you phone me and give me a vague report.
• When tell me that when open the file in Firefox, it loads without error but at the point that IE identified there is an odd character, a black triangle with a white question mark.
• When you open the file in Topologi Markup Editor, it reports “Illegal character found: FFFD” and has a processing instruction with FFFD at that point.
• When I check out that location in the hex editor for Topologi I see a non-ASCII byte xA0 in among all the ASCII bytes (which are all less than x80.

Have you figured it out? Which tools have the right behaviour? The answer below, after the space

The problem is that the file has byte pattern, the solitary xA0 byte, that is not possible in UTF-8. This is the non-breaking space code in most or all ISO 8859-1 character sets, and their extended forms used in Windows and IBM systems. UTF-8 has the property that bytes with the high bit set are never solitary. So we can diagnose two possibilities: either the encoding header is wrong and the file is actually “ISO8859-1″ or whatever (simply fixed) or the file was UTF-8 and someone edited it using an ISO8859-1 text editor which has corrupted the file (which is more tricky to fix.)

IE is correct for saying there is a well-formedness error and stopping at that point.

Where does the black triangle with question mark come from? It is the standard glyph (picture) for the Unicode character U+FFFD, with the semantic “buggered if I know”. Firefox must be using a transcoder that does not halt on encoding errors, but replaces it with this character. This is fair enough for browsing HTML, but not right for XML. But it is much better than the behaviour of old transcoders, which would silently strip the character or replace it with ‘?’

The transcoder in Topologi gives the warning and puts an inline PI to show where the error occurred. But, as is typical, it gets the information from the transcoder in the form of the Unicode character FFFD not in the form of the bad byte. But this is correct behaviour for a markup editor, which deals in text not WF XML or the infoset.

Did you get that? If you didn’t, don’t panic. But this is a very common problem. If you are receiving data from an AJAX system with forms, you may well get data with the wrong header: using UTF-8 as the default is better than using ISO 8859-1 as the default, because a UTF-8 will tell you when something is screwy while you can expect a UTF-8 or CP1252 transcoder to be as silent as the grave and promiscious as the town bike.

Different applications handle this problem in different ways. Many encoding libraries still do not generate errors or exceptions that the XML processor can use to report the well-formedness error of a bad encoding. Some people dislike that XML requires that a WF XML document should have no encoding errors; they are typically people who can write their names using ASCII characters only.

Related link: http://www.film-headquarters.com.au/video.htm?PUSA_Some%20Postman.mov

Photos by Ashley Bennett About a month ago, I heard about the first all-cell phone shot video of The Presidents of the United States of America (P.U.S.A.). This development is more than a novelty. It’s a working demonstration of the natural artistic progression towards the integration of new mobile video technology within existing art forms. I predict that it will soon be commonplace for bands to shoot mobile phone-based video of interviews, practices, performances, songs-in-progress, or whatever, and post it to their websites. I interviewed director Grant Marshall to find out even more of the gory details behind how this video was produced. (See below.)

The video was shot last July at the Seattle Grip and Lighting studio on Queen Anne hill in 10 hours using 12 two megapixel Sony K750i camera phones. All the phones in the U.S. seemed a few generations behind those of his native Australia, so Grant had to import the cellphones from Australia. (They had only been out in Australia for one week.) The phones were set up on tripods to record 9-12 angles for each shot and the footage was sent via Bluetooth to a Mac. The footage was then brought up to twice its speed to make it look “normal,” and of high enough quality to look professional enough to go back to back with the “high end” music videos.

Numerous technical obstacles had to be overcome in the process. Mobile phone cameras can only record at 1/3000 of standard broadcast quality, and don’t capture movement very well. In addition these phones only recorded at 10 frames per second, even though the manufacturer had promised that they’d record at 15 fps. During the shoot, the phones were so temperamental that they’d just turn off at any point without warning, so Grant had the band play the song 24 times at half speed in order to provide enough footage to edit together one good take of the song.

Director Grant Marshall

“The colors were tweaked and enhanced and/or softened a bit…and touched up with a little telecine to produce the final product.” That part of production took three people about two weeks, spearheaded by Tim Nikkinen. “We defocused parts to give it that soft look, and we ran noise reduction over the clip,” Grant explained. The video has been a huge success. On iFilm.com (now owned by MTV), it’s been viewed over 460,000 times and is already in iFilms top 20 most watched video’s of all time. The video will soon be available for download at iTunes, and is expected to premiere on MTV in November. Contrary to popular belief, although the project seems like the ultimate promotional tool for Sony’s camera phone line, the company didn’t pay a cent for its production. The whole thing was funded by Blast Records, P.U.S.A’s record label.

Here’s the complete interview: L.R.: So why shoot on Mobile phones? G.M.: The idea came to me when we were joking about the shrinking budgets of music videos in Australia. Where previously we would shoot on 35mm for two days, now we had to consider 16mm, HD or video (all in an 8 hour day). My director of photography complained, “What’s next?… shooting on VHS? Soon we’ll have to shoot on our mobile phones.” So I took this concept and wrote several treatments that would lend itself to a song. I originally pitched this idea over 18 months ago (when they started banning phones in change rooms and toilets) to several Australian bands, who instead chose clichéd music videos (like rocking out in an airport hangar… like we haven’t seen that before). Shooting on mobile phones was the most low-fi idea I could think of. I told PUSA that it would be like recording their album on an answering machine. In a hi-def world, this clip is Low-Def!

L.R.: How did you decide on the Sony K750i camera phone? G.M.: I did a lot of testing on mobile phones (I actually walked around shopping centres trying out the video functions of display models). I realised that to make the clip work we were going to have to utilise many tricks and use a heap of phones. The phones we used were the latest Sony Ericsson K750i with a 2 megapixel CCD camera that had only been out in Australia for one week. Despite the 2 megapixel camera, it still only recorded at 176 x 144 resolution at 10 and 15 frames per second (randomly switching, depending how long they were on standby). We had 12 phones on set, but not all were used for every take.

L.R.: How did you have to block your shots for them to come out right in the video?

G.M.: We had to think about the size shot we wanted, eg. if it was a midshot of Chris, the leader singer, we had to break the shot down into a grid and arrange the phones so that there was coverage of the shot. Each camera angle captured a section of the shot, and these sections overlapped so that a whole image could be pieced together. For example the top left phone would capture just the corner of his head and an eye, and the camera beside it would be focused on the other side of his head and shoulder. It’s not a difficult idea, it just needs a lot of time (we’re talking days to just sync the footage) and planning to piece together.

L.R.: So it took three people two weeks in post production to get it to look as good as it does?

G.M.: Yes. And I also came on board to help edit and online the clip, not because I didn’t trust anybody else to do it, but because I had done the research, I’d set every phone’s position and we had 9 – 12 angles for each shot and before we knew it we had 30 hours of rushes… on memory sticks… not something a professional post house is set up for. I would hate to think the amount of Avid time we would’ve needed to book. We had an editor with two assistants using three separate Macs working around the clock on different sections of the clip simultaneously. Final Cut Pro and some G5s were perfect for the muti- angle work as it handled it all in real time, something the Avids at the post house weren’t set up to do.

Overlapping shots (shot from the same position) were overlaid in the edit, doubling or tripling the resolution. This combined with speeding up the footage (the band played at half time) and combining all the multiple windows; now we’re getting close to something that is able to be broadcast. When you download the footage off the phones (via Bluetooth and USB) it is in its native .3gp format (which is approximately 8kbs per second, that’s 1/3000 the quality of SD) , which is a Quicktime file that can be used directly in Final Cut Pro for editing. The clip was exported out of Final Cut Pro and transferred to a Digi Beta tape. We then spent a few hours in Telecine enhancing the colours. We crushed it right down bringing down the black levels and softening them, and defocussed the highlights using a Davinci 2K+. We also ran a noise reduction over the whole piece and gave it a blue wash that matched in with our ‘behind the scenes’ footage as the LCDs on the phones had a blue tint to them. I still wanted the clip to look good as it would go back to back with big budget US clips on Video Hits and MTV.

L.R.: But you still think that people could do this kind of thing themselves at home, using consumer equipment?

G.M.: Yes. In fact there was nothing that hard about what we did at all, any kid with a Mac and a $300 phone could make something like this. And I hope we inspire people like this to give it a go.

Seeing a mobile phone on a tripod is really bizarre, it amused the whole set. I was totally amused we had an A.C.S accredited D.O.P Tim McGahan ACS shooting with a mobile phone. Everything about this shoot and post production was bizarre. In theory I could’ve text messaged the editor (in Australia) the footage from Seattle. No courier needed.

L.R.: What were some of the technical obstacles you encountered while filming the video?

G.M.: We encountered many problems while filming. Occasionally a phone would turn off, we were working with a very new technology in its infancy; its impossible to change the exposure as they are set on auto and are nothing more than toys. The mobile phones’ camera function was low quality and had a low frame rate, which was troublesome as some drum hits and guitar strums were totally missed by the phones as the footage would jump, not running as smoothly as the usual 25 frames per second. To remedy this problem and double the quality of the footage the band performed at half time, (something that is very hard to do 24 times while keeping their performance levels up) therefore their movement was twice as slow, which when sped up to normal speed increases the quality and smooths out their movement, also giving it a unique high shutter film look. This was a key technique, as most of the footage recorded at normal speed was virtually unusable or had to be shrunk down to a very small size on screen. The simple white background also helped us as busy backgrounds would tend to break up and artifact.

The clip could very easily be distributed onto phones (in fact I have a copy on my phone and if you were standing next to me I could Bluetooth it across to you). A fan of the band commented, “This is great, because the lower quality is much more suitable for file sharing, just like MP3.”

L.R.: When did the idea of doing this kind of project first pop into your head?

G.M.: When I was at university (Queensland College of Art – Film and Television- 2000) my lecturer told us that we would need to think about framing and the growing importance of telling stories through close-ups as laptops and LCD screens are getting smaller and smaller. The class scoffed, but sure enough 5 years later with video-ready ipods and mobile phones doubling in resolution every year (such as the new Nokia about to come out), this could be the beginning of a new format and the future of filmmaking as a readily accessible medium that can be viewed by anyone with a mobile phone or an electronic device such as an ipod or Playstation Portable.

L.R.: What was your favorite part of this experience?

G.M.: Although the clip was shot on mobile phones, the most compelling element is the band’s rockin’ performance. I knew after take three that we were onto a winner. The PUSA are the funniest band I’ve ever worked with. Although it was a great experience shooting a music video on mobile phones and I’m glad I was the first to do it, but I’m definitely looking forward to shooting 35mm again.

Resources

1. ifilm - Some Postman.

2. High res version of the video (requires quicktime 7) http://www.film-headquarters.com.au/video.htm?PUSA_Some%20Postman.mov

3. The Making of “Some Postman” — First All Camera Phone Shot Music Video

4. Camera phone enters new creative territory - from BillBoard.

5. Can you see me now? Presidents film video on cellphones By Judy Chia Hui Hsu for the Seattle Times.

6. Make Music Videos On Your Mobile by Shots.net.

What do you think about the implications of do-it-yourself broadcast quality cell phone-shot music videos?

The new video enabled iPods are getting lots of press and deservedly so. As I wrote in this space last month podcasting video opens up a new world of opportunities for the independent media producer. (In two earlier postings I explored the mechanics of video codecs and compression for the iPod, and I also illustrated the power of iPod friendly close-up shots with a memorable pastrami sandwich.) However, not everybody owns an iPod, and not everybody wants to watch video on a handheld device.

Hence, my praise of the Flash player. Encoding video as a Flash file enables a developer to place video directly into an HTML page. If you’re a filmmaker you can make your independent Sci-Fi epic, hip urban comedy, or social issue documentary project available to anyone with a broadband Internet connection, anywhere in the world. If you’re an educator you can use interactive video to create online learning environments that not only engage your students, but encourage them to use their critical thinking skills and take an active role in learning.

(For more on working with video in Flash, Macromedia offers an excellent overview of the process along with some very nice tutorials on adding Flash video to your HTML pages using Dreamweaver, and customizing the video playback interface. You can also download a free 30 day trial version and decide for yourself if it’s worth using.)

Combining Flash video with ActionScript, the coding language Flash developers use to create interactivity, enables people to design user experiences custom tailored to the specific needs of each project. For example, I recently worked on a large production where after viewing a carefully edited Flash video segment online, the user was automatically directed to a survey about their experience. I’m also currently discussing another project where people would view a Flash-based video presentation in a health care setting — the Flash environment would interface with the facility’s bio informatics system to record and store patient data, and also to page a doctor to enter the room for a personal consultation when the video ends.

I don’t claim that Flash is the perfect answer to every question about online video distribution, but it is a highly flexible and affordable way to get your work to an audience. I list some pros and cons below, but first take a look at two examples so you can get a better idea of what I’m talking about — each opens in a new browser window, and the video may take some time to appear on your screen depending on the bandwidth and processor power you have available. To play the video you need to have the newest Flash player installed on your computer, which you may view as one of the cons if you don’t like to install new software. (It’s also important to check that you have the newest Flash player installed on the browser you’re using to view the video — a friend recently installed the new Flash player on Explorer and not Firefox, and then discovered he couldn’t use Firefox to watch the video until he updated that browser’s Flash player as well.)

• The first clip is an excerpt of Big Luca, a short film I created for DV Filmmaking: From Start
  to Finish. (If you’re inspired by the composite effects, the book walks you step-by-step through their creation.)

• The second is a clip from Mi Querida America, an ongoing documentary project following a group of immigrant teens as they adjust to life in the United States.

Pros:
Unlike other media players, such as the RealPlayer or Windows Media Player, I can use Flash to display video in a Web page without using an external media player that has a fixed interface. Using Flash I have the option of custom designing my own interface so it integrates perfectly into my layout or I can use one of the preconfigured video playback components that ship with the application. (A few years ago I worked with an art director who asked if we could design an interface that would place a video playback window inside an image of an elephant — I explained that we could but probably wouldn’t want to. Then I came across this very inventive combination of Flash-based video and Polariod-style images, and since then I’m thinking maybe a playback window inside an elephant might not have been such a bad idea.)

The main benefit of the new Flash video codec is how good it looks. The video in my two example links is encoded at 300kbps, which is an unbelievably low bit rate for this level of image quality. (In short, the bit rate is the amount of information required to play one second of digital video. Higher bit rates require greater processor power and can cause computers to hang-up or even freeze. Lower bit rates, such as 300kbps have traditionally looked pixilated or like blocky mosaics.) Another benefit is Flash lets me encode the audio tracks as MP3 files, which also deliver impressive quality at low bit rates. In addition, Flash encoding software integrates with a number of video editing/compression applications, so I can now export to Flash video format directly from Final Cut Pro and QuickTime Pro.

Cons:
Not everybody has the latest version of the Flash player. If a person tried to watch my video clips with an older version, say one they downloaded two years ago, it simply wouldn’t work. (Flash Hacks explains how to check if someone has the Flash Player installed, and how to check which version.) Some people don’t have the Flash player at all, and aren’t interest in installing one — you can provide people with a link to download the latest version, but if they don’t want to there’s not too much you can do to convince them.

Unlike televisions, DVD players, and iPods that are built to standardized video specifications, computer operating systems and hardware process video in surprisingly different, and often frustrating, ways. Video that plays back easily on one computer may freeze another. Likewise, video that looks great on one computer may look really ugly on another. Although the Flash player is entirely cross-platform, Macs and PCs handle color differently. Sometimes an image might look great on a Mac but show up too dark to watch on a PC. To put it simply, there’s no substitute for checking your work on the same type of equipment your audience is using. To check this video I emailed friends around the country and asked them to watch the clips and report any problems before I made this posting public — one friend who uses an older PC had to wait about 20 seconds before the video showed up, but once it did everything worked nicely.

Lastly, Flash video won’t play on an iPod, so you miss out on part of the coolness factor. Of course, if you’re really motivated, you can encode multiple versions and offer different media formats for different members of your audience (it may take a little more effort, but if it adds to the quality of your project and makes your work accessible to a larger number of viewers it may be worthwhile). Either way, Flash and the iPod aren’t the only ways you can deliver video to a large audience — but I’ll save that for another posting.

Stay tuned.

Visiting Disneyland last week for the first time in about 16 years, I was struck
by the sound of the place—both what was there and what was missing.

It seemed like the park designers were afraid of silence. Music and sound effects
were everywhere. Spotting the hidden speakers became a game for my
son and me. Bushes, fake rocks, trees, light poles, and even the seats on the
rides blared happy tunes. During a boat ride down a miniature river, we heard
frogs and crickets beaming from the riverbank. As we floated past the stiff,
robotic wildlife—“Those ones? They’re gnu” quipped the
guide—I imagined a future where nature had been scrubbed away and replaced
by digitally enhanced facsimiles. It was kind of creepy, like the decoy fireplace
in that old Star Trek episode that emitted no heat because the alien
copied only superficial aspects of the Earth.

Into the belly of the beast….

The multiple speakers allowed some nifty acoustic mixing tricks, though. In
one room, a movie played on a screen while the voice of projectionist Donald
Duck wafted up through the projector hole in the wall, synchronized with hand
shadows Donald was supposedly making on the screen. (Does he even have
hands? I didn’t think about that until now, so I guess the effect was
convincing.)

In the fantastic Main Street parade, the audio on the floats synced with the
roadside speakers. How cool would it be to compose for multiple moving speakers that pass
by stationary ones?

Newer rides have speakers in the seats, for a soundtrack that follows you.

Overall, and particularly on the rides, I found the sound was far too loud
and bright to be realistic. Of course, I was there on a weekday in November.
Maybe bigger crowds would absorb or compete with the sound during most of the
year.

The other thing that struck me was how much the sound quality varied between
the old and new attractions. The audio on older rides was ugly—very
harsh and midrangy. But then, Disneyland has always juxtaposed old and new. Even
on my first visit 26 years ago, I was impressed by how the environment was futuristic
and anachronistic at the same time. It was like a ’50s view of the future
that had never been updated as that future arrived.

And the sound that was missing? That turned out to be an unexpected bonus: The
“It’s a Small World” exhibit was closed for remodeling, so
its syrupy song (perhaps the most annoying of all time) was blissfully absent.

What juxtapositions inspire you musically?

A good C++ programming technique that has almost no published material available on the WWW relates to using the special pipeline instructions in modern CPUs for faster text processing.

Typically compilers (such as for C++ and Java) find it hard to detect possible parallelism in programs: indeed, this is because typically you have to structure the algorithm to suit the CPU’s capabilities, which goes against the desired requirements for cross-platform, Write Once Read Anywhere code. It used to require assembler programming, but modern compilers support a special class of functions called
intrinsics which are more-or-less macros that expand to assembler code sections.

Lets take the UTF-8 to UTF-16 conversion function used in
"http://xmlsoft.org/">libxml
as our example. Its pretty typical code; it doesn’t look too bad to me. UTF-8 is a variable-width encoding of Unicode: it uses one byte for ASCII characters (and with exactly the same codes), two bytes for most Western characters (both bytes have their top bit set), three bytes for most Chinese characters, and four bytes for new rarer characters. UTF-16 is a fixed width encoding of two bytes. (We ignore the issue of surrogates here.)

Here’s the original code (with some dead code removed):

/**
* UTF8ToUTF16LE:
* @outb:  a pointer to an array of bytes to store the result
* @outlen:  the length of @outb
* @in:  a pointer to an array of UTF-8 chars
* @inlen:  the length of @in
*
* Take a block of UTF-8 chars in and try to convert it to an UTF-16LE
* block of chars out.
*
* Returns the number of bytes written, or -1 if lack of space, or -2
*     if the transcoding failed.
*/ 

static int
UTF8ToUTF16LE(unsigned char* outb, int *outlen,
			  const unsigned char* in, int *inlen)
{
	unsigned short* out = (unsigned short*) outb;
	const unsigned char* processed = in;
	const unsigned char *const instart = in;
	unsigned short* outstart= out;
	unsigned short* outend;
	const unsigned char* inend= in+*inlen;
	unsigned int c, d;
	int trailing;

	/* UTF16LE encoding has no BOM */
	if ((out == NULL) || (outlen == NULL) || (inlen == NULL)) return(-1);
	if (in == NULL) {
		*outlen = 0;
		*inlen = 0;
		return(0);
	}
	outend = out + (*outlen / 2);

	// PARALLEL OPTIMIZATION TO GO HERE !!

	// Output the rest the usual way
	while (in < inend) {
		d= *in++;
		if      (d < 0x80)  { c= d; trailing= 0; }
		else if (d < 0xC0) {
			/* trailing byte in leading position */
			*outlen = (out - outstart) * 2;
			*inlen = processed - instart;
			return(-2);
		} else if (d < 0xE0)  { c= d &a