That title bar is not in vector format - it's three bitmaps! Your reason for thinking that it is in vector form is superficially plausible, but it turns out to be fallacious:
"the gradients used in the title bar are all smoothly scalable."
I'm assuming that you used the OS X Preview application to look at it - that applies some kind of interpolation (it looks like bilinear interpolation) to any bitmaps it finds in a PDF. Try saving out a whole screenshot using the Shift-Command-3 key press. That generates a bitmap of the screen stored in a PDF. Zoom in on that, and you'll see exactly the same effect - the gradient remains perfectly smooth as you zoom in, despite the fact that these screenshot PDFs just contain one big bitmap.
This gradient smoothness, which you have mistaken for evidence of vector imagery, is just the result of the interpolating scaling algorithm being used to scale the bitmap - linear gradient fills in bitmaps happen to come out looking just great if you use either bilinear or bicubic interpolation to expand them.
If you open your PDF up in Adobe Illustrator, you'll be able to see clearly that the title bar is just three bitmaps - the main area and the two end caps. (And the traffic lights are another partially transparent bitmap drawn on top of the main title bar bitmap.)
Alternatively, if you don't have Illustrator, open it up in a PDF viewer that uses nearest neighbour scaling rather than a smoothing interpolation. The current PC version of Adobe Acrobat Reader does this. When you zoom in with such a view, it becomes perfectly clear that the title bar is just three composed bitmaps.
So this remains unconvincing - you have produced a PDF containing a bunch of bitmaps. This simply appears to add further support to my two hypotheses (1: that OS X cannot produce scalable window content, and 2: that this is because once you get outside of a Quartz 2D surface, Quartz works by composing bitmaps).
I tried a variation on this experiment. I wrote a Cocoa application with an NSImageView on it. I drew a scalable image in OmniGraffle and made that image the NSImageView's image, and also set the NSImageView's autosizing to make it resize with the application.
When resizing the window, the contents of the NSImageView get redrawn every time. This of course gets Quartz 2D to redraw the contents from scratch every time. So on each redraw Quartz 2D rerasterizes the image to the approprate size and then hands it over to the compositor. So the imagery appears to be scalable. But look what happens when I print to PDF:
(I apologise for the exceptionally poor drawing skills, but Dammit Jim, I'm a progammer, not a graphic designer!)
Even though you mistook the bitmaps in your image for vector imagery, surely you cannot fail to see that this PDF is clearly made up of bitmaps - the image view that I gave scalable image data to is quite clearly a bitmap in this PDF. (And I used exactly the same technique to generate this bitmap as you described by the way. I just added the NSImageView. And a button, to see what that would look like. I notice that scales poorly too, although that's presumably because it's done as a bitmap just like the traffic lights.)
This leaves just the text in vector form. But even Win32 can do that. In fact it can do slightly better (even if you use the old GDI32 API) - if you set up an Enhanced Metafile device context and then pass it using a WM_PRINT message to a window, it will write a scalable representation of the window into the enhanced metafile. (This is the closest equivalent I could get to of printing a window using Save as PDF... under Windows, because Windows doesn't have built-in PDF support.)
If generate a scalable EMF file of a window in this way on Windows XP, all the window decorations suffer from the same problem as your OS X example exhibits - the decorations all come out as bitmaps. Any text will come out in text form rather than bitmaps, as with your example. But unlike your example, in Win32, the resulting enhanced metafile will also put any vector drawing primitives (like ellipses, bezier curves etc) in vector format too. (Contrast this with what happened in my NSImageView example on the Mac.)
So it's possible to do a slightly better job with Windows than you have done here. I therefore remain unconvinced. (But I'm still retaining an open mind. If anyone can show me a PDF of a UI that really is scalable, rather than simply being a bunch of bitmaps, then please do!)