As I’ve written about before, in the last few months I’ve been using lots of different tools to work with my digital photographs. For example, I’ve been maintaining my main image library in Aperture, but using Lightroom to create large prints with. With the recent release of the Photoshop CS3 beta and the full release of Lightroom 1.0, I’ve been working with multiple applications even more. And the more I work with all of these applications together, the more that I wish that the incredibly important metadata about these photographs was fully interoperable across the set of applications I use.
Adobe’s applications, most notably Lightroom, Bridge, and Photoshop, are fully interoperable with RAW file metadata. When you rank and keyword images in Lightroom, that information is reflected in Bridge. This is accomplished not through some secret API sauce on Adobe’s part, but instead through XMP metadata. This metadata is either embedded into the image files, or in the case of camera RAW files such as NEFs and CR2s, as side car XMP files.
It takes a lot of time to add metadata to your photographs. This is important stuff that you don’t want to have to perform time and time again. It’s information that should be able to travel with your image data when you hand it off to somebody else. It’s data that needs to be preserved for the future. And, it shouldn’t be locked up in any one application’s silo. It’s just data.
So, given all of that, my number 2 feature request for the next version of Aperture—right behind performance improvements—is full support for XMP in externally referenced files. When a keyword is added to an image in Aperture, it should be written to an XMP sidecar file so that if you open your images in Lightroom or Bridge, all of your metadata is right there. I don’t expect, of course, for Aperture to support Adobe Camera Raw settings. That would be asking for Aperture to emulate exactly how Adobe Camera Raw works. But, for keywords and IPTC metadata, it would be a much appreciated feature addition.
Hi guys,
I thought I'd share a couple of points in brief:
* Right now photographers who want to use DNG mostly rely on Adobe software to do the conversion, but that's not a requirement: the format is publicly documented, and Adobe provides open-source code for implementing DNG reading and writing (via the DNG SDK).
* There's no relationship between the DNG Converter being able to convert a file to DNG, and DNG-reading software's ability to read DNGs from that camera. Even if Adobe were to stop supporting conversion from a particular format (something that seems unlikely, but which is possible), DNGs made from that format would remain perfectly readable by DNG-aware apps.
* It's true that the DNG Converter does need to be updated for new proprietary raw file formats. That's the benefit that Adobe is providing: the translation of an unknown to a defined standard. And beyond the conversion experience, ask any photographer using a Leica M8 or Pentax K10D how much they appreciate instant support from the moment their first raw file is captured.
* I've heard from certain camps that DNG is a bit of an empty promise, that these companies really have to do custom work for each camera & that they therefore can't support DNGs made from cams they don't support. If that's the case, why are DNGs compatible with Camera Raw in Photoshop CS1, which was last updated some two years ago? It may be that a developer will *want* to do custom work for a camera, even if its images are in DNG format, but that isn't a requirement.
Hope that's useful info,
J.
My number one request for Aperture is that they fix the #$!$ dates! It can't handle arbitrary date assignment.
Mercifully, Lightroom can -- though there is a bug. If you assign a date before 1500 or so the month/day values change erratically. One might do this, for example, when working with scans of historic documents or artwork.
You do have a beef with DNG don't you? I agree people are crazy if they throw away their NEFs - disc space is cheap - but your series of hypothetical arguments can equally apply to the camera makers themselves collapsing and support for their formats being dropped. Which can safely hold all sorts of descriptive or editing XMP? Whose embedded thumbnail/preview can safely be written by other applications? Which is archival - the undocumented or the publicly documented format?
Well, Windows Explorer's the apogee of image editing, isn't it? Try Notepad - no use for raw conversion but you'll see all the metadata.
Gio,
Well, as I said, don't know where you get your information.Can you factually back up your argument? Don't know if you read fully the links I sent below, but here is a clip from a Q&A from the John Nack on Adobe blog:
Katrin Eismann -- 04:16 PM on February 26, 2007
"Hi John,
I was wondering why does the new ACR support JPEGs and TIFFS (even layered TIFFS) but not PSDs?
Curious Katrin!
[Great question, Katrin. In simplest terms, ACR is designed to handle images from cameras, and cameras don't shoot PSDs.
More technically, ACR handles flat documents, and 9 times out of 10 a PSD contains multiple layers (otherwise, why bother making it a PSD?). Therefore, upon opening that file in Photoshop, ACR would either have to turn it into a flat file, or ACR would have to learn how to edit all the layers individually while honoring their complex blending options (i.e. it would have to become just like Photoshop in its layer-handling savvy). Neither of those things seems too appealing, so we are keeping ACR focused on the mission of editing flat files from cameras.
This is an area in which ACR and Lightroom differ. LR needs to be able to view and edit PSDs so that you can edit a file in LR, throw it to Photoshop, add layers, then save and return to LR & keep editing. I think users would accept LR subsequently handing Photoshop a flattened version of the PSD, whereas they wouldn't accept that behavior from ACR (which is part of Photoshop). Does that make sense? --J.]"
This makes clear sense that in order for Adobe's DNG converter to make a .dng file of a TIFF, it has to be one made from a camera, NOT a scanner. And for those with sizable scanned film archives, yes indeed, inability to convert to DNG is a significant drawback to adoption. The operative phrase from John Nack's answer is "ACR is designed to handle images from cameras, and cameras don't shoot PSDs." So the DNG initiative leaves the scanned film archives out in the cold. It's only designed for digital camera files.
As I said, if you offer proof I'm wrong about Adobe having to first support a camera model (and, updating ACR + DNG Convertor) before a conversion can be made, please do...and I would hope I'm wrong. Unfortunately, I'm not. If you get your hands on a Canon 1DS Mark III, and shoot a few test RAW files before Adobe can provide an update to the new RAW format, I will bet you'll have to sit on those files, until an update is available.
This is a VERY big caveat, and roadblock to widespread DNG adoption. I too was excited at the prospect of a "standard", but it comes at the price of Adobe controlling the show (for now, at least). See this interesting link:
http://news.com.com/Adobe+taps+the+power+of+negative+thinking/2100-1041_3-6136875.html?tag=nefd.lede
So, if Adobe doesn't support it (camera model), and Adobe is the only company providing DNG conversion software (even if they license it to other companies), then you can't even create "archival" DNGs for other applications (Aperture, Lightroom, ACR, Bridge/Photoshop, iView MediaPro, Bibble, Capture NX, DxO,...name your RAW app.) to even use. It all hinges on Adobe providing camera/model support.
As stated in some articles justifying DNG adoption, one concern is what happens when camera manufacturers drop support for older models? As Adobe's ACR/DNG Converter are primarily updated to include new models, the real question is will Adobe choose to drop older cameras off it's list? If the code is already there for RAW conversion in ACR, who cares if a camera company no longer supports or discontinues a model? It's only if Adobe drops a camera it already supports from its list (perhaps to save code/avoid "bloat"?), that you would really have trouble. Few people really like using the camera provided RAW processing software (but at least it's there in emergencies). Loads of people rely on Adobe for RAW support, one way or another.
A lot here relies on Adobe, not the other (above mentioned) software providers with regards DNG. It's not ommissions, or "lack of support" on Apple/Apertures side. Adobe is in control and responsible for the DNG initiative. Don't just read the hype, if the camera model is not Adobe supported, or worse, your model gets dropped from Adobe RAW support, your future "Archival" DNGs are useless. THAT to me is a big caveat...So how much do you trust in Adobe?
You claim DNG's "archival strength comes from leaving the image data untouched, documenting its origins in a publicly-readable format, and carrying within the file any XMP format data that anyone wants to add. It's a whole lot more archivally-robust than believing in Apple and leaving your descriptive and adjustment metadata inside the Aperture database."
Really? sounds like a whitepaper soundbite from Adobe. Looking at DNG files in Windows Explorer doesn't show me anything but a single file, with no way to "see" the contents, nor even "extract" the original "untouched image data (read: RAW file it was made from)". If people are converting to DNG and tossing the RAWs (and some do, from internet postings!), feeling their images are "archivaly safe", they should re-think the matter.
Not sure if we're clearly talking at cross purposes - you're pretty vague about "the application" and maybe I'm wrong in assuming you mean Aperture. Where Adobe doesn't support the camera format of course DNG converter will not make a DNG. Hold the front page! Or maybe not - Adobe lead comfortably in supporting new cameras and has a wider range of existing models. The editing application itself, which is what I thought you meant, has no need to support the original camera - that is Apple's omission. DNG's archival strength comes from leaving the image data untouched, documenting its origins in a publicly-readable format, and carrying within the file any XMP format data that anyone wants to add. It's a whole lot more archivally-robust than believing in Apple and leaving your descriptive and adjustment metadata inside the Aperture database.
Not wanting to use a more open format like TIF instead of PSD for scans is your problem, not a remotely-significant drawback of DNG.
Gio
Ah Gio...where do you get your information?
"There is no need for the application to support the (camera) model which originally created the image data that is in DNG"
...really? Try using Adobe DNG Converter to convert a RAW file from a model that Adobe does NOT support. It can't be done! Apple isn't the only company that has issues with DNG...so does Adobe. As for TIF or PSD, my workflow is geared towards PSDs and RAW. TIFs have issues, but we won't get into that, and contrary to the way some on the Adobe team think (based on some of their posts), not ALL PSDs are of the layered flavor. I'm not asking to convert layered PSDs to DNG, Just my flattened ones. I don't intend to, or care to, add another file format like TIF to my mix. Just not sold on the "Convert - to - DNG" will "save our assets" evangelism. (BTW: Adobe DNG Converter failed to allow a conversion of a scanned image saved as a TIF...oh well!)
Trace
That's a pretty small caveat. There is no need for the application to support the model which originally created the image data that is in the DNG - that is merely how Apple have chosen to "support" DNG. Second, as you note, Adobe support for formats is about the best around - including new ones like the R8 and Phase backs. And forgive me, but why would you want to put a PSD in a DNG - scan to TIF, which can be put into DNG.
Gio
One big caveat with DNG is that if the camera model is not "supported" by the application, neither is the DNG file. Try converting a RAW file that is NOT on Adobe's camera list, and see if you can even make a DNG file, let alone open it in Bridge/ACR. And yes, Aperture's supported camera list is smaller than Adobe's.
Another unfortunate draw back is PSDs (read: images from scanners), cannot be turned into DNGs. Glad I don't switch around multiple apps, save Aperture/CS2-3. Too modular a way for me.
Good discussion here. One point of reference I have is the way iView syncs metadata - it can read or write. Part of the problem, in my opinion, is Apple's design philosophy, which shows up here. Aperture says: "I'm the one managing photos here, why do I need to talk to anyone else?" It's the old closed-systems approach (I know, that only goes so far, but I think you know what I mean). Witness the package library format!
Sheesh, metadata standards... if you think this is bad, you should see things on the video side. It's way worse. =)
These links may help to clarify a few things...or muddy them further, but the whole RAW/XMP/DNG business is an interesting situation:
http://www.imaginginfo.com/article/article.jsp?siteSection=34&id=67
http://blogs.adobe.com/jnack/2007/02/nondestructive.html
James
The small difference between us here is that you want XMP updating to be live, while I want to launch it when I want. I dream of commonly understood processing values too, obviously not everything but just as Aperture writes ratings can be shared with Adobe/iView, we could also have a generic white balance to which LR/Ap could refer, just like they refer to the camera WB. Users could implement this themselves, manually at least, if Ap/Lr supported extensible XMP - which "pro" tools should do (yeah, I know photographers don't understand data).
Gio
"1) Import into my library directory, whether in Aperture or some other program.
2) Make meta data changes in Aperture
3) Open up Bridge to pull in images into InDesign via Photoshop. Sure, I don't mind reprocessing the image in Adobe Camera Raw at this point. But I'll also probably put a keyword in that indicates that I used it in a layout.
4) Go back to Aperture and see the new keyword show up there."
Got it James...think we are making the same points (I was simply putting it more verbosely!)
Seems you're wanting to import a RAW file into a library (say Aperture), add metadata info, then open the same RAW file in Bridge (with all the new metadata from Aperture visible...say keywords, ratings, copyright, etc.) It still being the RAW format, it will need processing in ACR before being opened (rendered) into InDesign via Photoshop, correct?
But before InDesign (or any other Adobe Suite app.) can use that file, it has to be saved out by Photoshop as something other than RAW format. The only part of your process that can fiddle with the RAW file is ACR, or Aperture, and both need to render the file out as a format the other apps can play with. So why not simply process the RAW in the processor of your choice (presumably Aperture, unless it is only being used for library purposes), and render a version, with all metadata intact for Bridge to see and InDesign/Photoshop to tweak? Why take two bites of the RAW apple ('scuse the pun!) for essentially what is the same outcome?
I think what you are saying is changes to metadata made in Bridge do not show up in Aperture. Perhaps, I haven't tested it, but then I don't use Bridge to manage that data, that's why I have Aperture in the first place (among other reasons). But I see your point, and they should play nice together...
Guess embedding the XMP rather than sidecaring it is the easiest way to go.
Trace: Also, to answer your question about why you wont' see the processing instructions from an Aperture export show up in Adobe's software... It's all about the fact that different RAW decoders support radically different features. Sure, things like luminance and curves could be standardized, but unless you are running the same processor, you would be hard pressed to try to "emulate" the behavior of a different processor. It _could_ happen, but.
I'm happy to start with metadata. :)
Trace: Actually, I don't care too much about the processing changes to the file. Things like curves and levels aren't that important to me to transition between programs. Sure, it'd be nice. But that's not what I'm asking for at all.
What I care about are keywords and IPTC metadata being shared via XMP. If Aperture wants to put more stuff into the XMP data (whether in sidecar or in an embedded data block), that's fine. But here's the use case I want to be able to do:
1) Import into my library directory, whether in Aperture or some other program.
2) Make meta data changes in Aperture
3) Open up Bridge to pull in images into InDesign via Photoshop. Sure, I don't mind reprocessing the image in Adobe Camera Raw at this point. But I'll also probably put a keyword in that indicates that I used it in a layout.
4) Go back to Aperture and see the new keyword show up there.
Metadata shouldn't be in a silo. That's all.
Trace
If Apple chose to put its processing XMP into that sidecar file it could do so. It would just have to put them under its name - the X stands for extensible. You too could have a trace namespace, with info that are important to you.
In this case, for reasons best known to itself, Apple don't put the processing instructions in the XMP. Adobe have done so for 2-3 years.
Gio
Ok,
After testing this I think I know James' concern: If you create any processing changes to a RAW image file (in Aperture, and presumably that other software), And you later decide to export that Master RAW, along with the resultant XMP sidecare file (which includes, among other things, said processing instructions), in order to use it in another RAW application, the RAW file "should...ideally", open in the new application with complete processing instructions (sharpening, color balancing, cropping, B/W conversions, etc.), intact.
Only problem is, it doesn't. I can export a RAW master from Aperture, complete with separate XMP sidecar file (after processing image), and even see both the RAW and XMP files in my folder using Windows Explorer (I'm dual platform, bear with me here), but opening the folder in Bridge (CS3) reveals only the RAW image without the previous application's (*Aperture) adjustments. All other data does transfer nicely to Bridge, including my star ratings!
Why is this? Isn't XMP a "standard" format, and an "open standard" at that? Why can't one application "read" the XMP processing instructions of another? I could be wrong (often am...), but I suspect that instructions like "minus 18 red channel levels/ sharpen plus 27/ Grey point plus 53.24/ etc...." mean different things in different applications, because none of them are using a "standard" zero point!
A plus 2 in Luminance in Aperture does not translate to the same setting in Lightroom, or any other RAW converter. Until RAW conversion software (and presumably camera manufacturers) get a consortium to implement an absolute standard from which to "zero-out" settings, there will always be conflict.
At least in the "old days" of film, we had international standards. So that ISO rated film manufactured by Kodak, Fuji, Ilford, Agfa and all met a standard of tolerance that photographers could rely on...and camera manufactures were on board too.
Now it's Proprietary Hell, with DNG as a supposed alternative. I do miss the "everybody on board with standards, let's make some images!" feel of the film days...sigh
Gio: I sure hope it's an open door. But even if Aperture supports import of XMP information, it's still not quite right. It needs to capture changes to the XMP information from other applications when the metadata is changed, say in Bridge.
Trace: What you describe works well when you're willing to let Aperture do the RAW decode. But at that point, you've got multiple copies of your image data and are starting to walk into a more complicated DAM situtation. What I really want to see is the ability to have the RAW files on disk with XMP metadata that can be shared by all applications without metadata silos. I'm willing to accept RAW image processing silos since different RAW decoders work differently, but not for keywords and IPTC data.
Allan: Yes, I agree that the metadata should be in the file. It makes for transmission to other users much easier. DNG is the way to do this for RAW data, but that requires full support for DNG in Aperture. Maybe that's number 3 on the request list for Aperture. :)
Allan
XMP is a data format - it can be embedded or in sidecars. Apple and Adobe don't want to write any metadata into raw files - they consider the risk of writing to an undocumented formats too high. Many customers disagree - and have backups anyway. But what can be embedded isn't as extensive as needs to be carried in XMP format. It would be great if Apple supported DNG - it is documented and plenty of non Adobe programs can write to it.
Gio
While I think XMP support would be great, I generally prefer my file metadata to be embedded in the files themselves. This enhances portability and reduces chances for errors or loss, and makes it app-neutral (as you mention above). This works great for JPG, TIFF, and PSD images; unfortunately most RAW formats don't support embedded metadata (for some reason!). XMP sidecars for RAW are great.
This is why Trace (below) can see his metadata in Bridge - it's embedded on export.
James,
I use an Automator script I created to get selected images from Aperture and create full size 16Bit PSDs and put them in a folder on an external drive. The drive is then connected to a Windows XP desktop (where my copy of CS2/CS3 resides), and the images are opened in Bridge (CS3 Beta). Violà, Bridge shows ALL my Aperture keywords, metadata, copyright notices, IPTC entries, etc. Even tells me the focal length used and if the flash fired.
I do not use the "File/Export" command in Aperture for my roundtripping to Photoshop, as the Automator script takes care of everything. Gotta' love the Mac platform! So a Mac/Aperture-to-PC/Photoshop workflow is very fluid and works beautifully!
I'm sure you're kicking at an open door there. The weird thing is, it currently exports XMPs with export masters, but amazingly doesn't import XMP (odd of Apple not to lock your data in, but lock others' data out). So I'm sure there will soon have an option to export sidecars without exporting the masters. This needs to merge with any existing sidecars, not overwrite. And this metadata should include editing instructions too - after all, some bright spark might be able to translate at least some of it into ACR and vice versa.
Gio