Legal
The views expressed in this blog are my own and do not necessarily reflect the views of Adobe Systems Incorporated.
Search
February 26, 2007
Non-destructive JPEG: An oxymoron?
When cameras capable of shooting digital raw files started hitting the mainstream (roughly five years ago, give or take), one of the advantages of shooting raw was that editing had to be non-destructive. That is, because the pixel data hadn't yet been converted into traditional RGB channel data, applications like Photoshop couldn't poke at it directly. This in turn meant that conversion parameters had to be stored as sets of instructions, rather than as burned-in as pixel edits.
Photographers have now become familiar and comfortable with the idea of moving & storing the captured bits along with the "special sauce" used by their raw processing app of choice. The XMP files that are (optionally) parked next to images by Adobe Camera Raw & Lightroom make this particularly easy. The fact that the DNG format supports built-in metadata & rendered previews turns it into a kind of envelope (or "job jacket," to borrow Peter Krogh's phrase)--a container that stores your negative, your processing instructions, and your rendered print. As editing tools get richer--for example, with Lightroom's ability to store multiple settings per file--the benefits of this approach grow.
But what about non-raw files? Both Lightroom and Camera Raw now offer the ability to edit JPEG and TIFF files, so that no matter what format(s) your camera generates, you can use the same non-destructive tools. So now a photojournalist or sports shooter, say, could shoot JPEGs, apply edits in the field (soft crops, non-destructive dust busting, tonal corrections, etc.), and upload the original files plus their processing instructions.
This poses some tricky questions, however. Fundamentally, is it okay that Adobe is putting "special sauce" into the metadata of JPEGs, causing them to appear differently when viewed in the latest Adobe editing tools than in other apps? Is it okay to extend the JPEG standard? A few things to consider:
- Adding this metadata to JPEGs doesn't damage the files in any way, or degrade other tools' ability to read the pixels. The data is simply ignored by tools other than Lightroom/ACR/Photoshop/Bridge. Adobe tools are leveraging the flexibility that's already in the format.
- Generating a copy of the image with the edits burned in (i.e. with the pixels changed) is a one-click task.
- Putting the metadata into the files makes it more easily portable than requiring a sidecar file.
- One alternative would be to bake Lightroom/ACR edits into JPEGs immediately, thereby negating the advantage of non-destructiveness. Another would be to force the JPEG to be converted to another format, making it clear that something had changed, but rendering those images unreadable by other tools. Forcing either approach, however, seemed like a bad idea.
So, there are pros and cons to any approach, but the one we're pursuing makes it possible to enjoy the portability and non-destructiveness of raw editing using non-raw files. It's done in a way that lets JPEGs be extended easily & without damage. If you're concerned about using this approach, you can convert JPEG & TIFF files to DNG (an option I'll address separately in a bit)--but that conversion isn't forced on anyone.
My take is that the flexibility it opens up is more than worth the cost. What do you think?
Comments
It adds additional functionality to an existing format... I agree with you... it's more than worth the cost. This is definitely a smart way to go.
Hi John,
I was wondering why does the new ACR support JPEGs and TIFFS (even layered TIFFS) but not PSDs?
Curious Katrin!
[Great question, Katrin. In simplest terms, ACR is designed to handle images from cameras, and cameras don't shoot PSDs.
More technically, ACR handles flat documents, and 9 times out of 10 a PSD contains multiple layers (otherwise, why bother making it a PSD?). Therefore, upon opening that file in Photoshop, ACR would either have to turn it into a flat file, or ACR would have to learn how to edit all the layers individually while honoring their complex blending options (i.e. it would have to become just like Photoshop in its layer-handling savvy). Neither of those things seems too appealing, so we are keeping ACR focused on the mission of editing flat files from cameras.
This is an area in which ACR and Lightroom differ. LR needs to be able to view and edit PSDs so that you can edit a file in LR, throw it to Photoshop, add layers, then save and return to LR & keep editing. I think users would accept LR subsequently handing Photoshop a flattened version of the PSD, whereas they wouldn't accept that behavior from ACR (which is part of Photoshop). Does that make sense? --J.]
Hmm, great question... clear explanation too, thanks.
I understand this as "The non-destructive edit modes developed for digital raw files can also be used on JPEG. What issues would arise from embedding that processing metadata with the unedited original image data within the .JPG file, in a file which any JPEG viewer could still view?" Am I on-track here?
[Yep, exactly. The question of "what issues would arise" has less to do with other apps mishandling the data (which I'd expect them simply to ignore) and much more to do with the fact that JPEGs will potentially look different depending on what application reads them.
This state of affairs actually exists today to some degree: ICC profiles (a form of metadata) are honored by some apps and not by others. But ACR/Lightroom settings go further, specifying cropping, healing, etc. --J.]
If so, then is there any difference from other uses of metadata within a .JPG file?
http://www.w3.org/2001/sw/BestPractices/MM/resources/Tools.html
http://www.google.com/search?q=jpeg+metadata
[I'm a little out of my depth trying to address that one, as I'm not a metadata expert. But offhand I don't see any conflicts with the spirit or the letter of the specs. --J.]
I don't know all the JPEG standards... do they have guidance on the use of metadata, other than comments, stored within the file wrapper?
I rarely heard support problems with Fireworks' use of editing instructions within .PNG files -- like you describe above, other readers will just ignore chunks they can't process.
[That's a very good parallel. The only difference I see is that I think Fireworks updates the main PNG so that it matches the appearance you'd see inside FW. That is, the crop, colors, etc. wouldn't differ--only the presence or absence of layers, frames, and other editing constructs. --J.]
Is the above on-track to what you're seeking...?
[Yes, thanks. --J.]
Thanks John - Oddly enough ACR could 'see' a layered TIFF file which threw me for a loop. Katrin
[Yeah--I think that's actually a bug, and the shipping version of CS3 won't allow ACR to handle layered TIFFs. Those files aren't from cameras (at least not directly), and the question of what to do with the layers upon opening them it too dicey. --J.]
I agree that your use of the extensible features of the JPG standard are right on the money. That's why formats like JPG and TIFF have these extendable formats in the first place. There is no way to know how formats are going to be used as technology advances so adding metadata into the file is the best way to go.
The fact that only Adobe products can read it is actually a "good thing" here, as long as Adobe makes the information at least somewhat public so that other applications can get basic information from the metadata (at least cropping info and the new thumbnail). Just some thoughts.
[Converting JPEG to DNG can enable this. --J.]
I'll cast another vote for encouraging ACR to work with all file formats. I find the ACR adjustment tools to be incredibly intuitive and easy to use for the first batch pass before getting into heavy editing in Photoshop. Most of the time these are a batch of shots from a camera, but sometimes they're not (i.e.: preparing a bunch of photos for inclusion in an Indesign layout).
Now that you mention ACR interface questions, it also occurred to me that I often find myself in the position that most but not all of the shots in a batch don't need to be fully processed in Photoshop - ACR is sufficient. For those, it would be really nice to be able to "dip into Photoshop" for just a few frames in a batch, then back out to ACR for the jpeg conversion and xmp save.
But maybe this is an argument for rolling all of this stuff together into one program...
I had the option of outputting TIFFs in my last camera but never used it. The overheads were just too expensive when compared to a full quality JPEG that would write in less time and take up less storage space on the CF card. So, though ACR can open both file formats, I suspect the majority of them will be JPEGs.
Is there a case for opening PSDs and TIFFs that include a flattened version? It would open up another workflow: perform non-destrcutive edits not available in Photoshop. For example, clone, heal, crop, HSL, Grayscale, Split Toning without overheads. Just thinking out loud...
Shangara.
John,
Is that a G-Love reference?
[Not intentionally. ;-) --J.]
I know you guys like those music references. So if I understand you correctly, only Adobe products at this point are reading the JPG adjustments. So, if I were to send it to a lab without doing some time of processing, all my edits would not be picked up by my lab.
[They will if those folks are using CS3 or Lightroom. But the safest bet is to generate a version (using Save As from ACR, or Export from Lightroom) with the edits baked in. --J.]
That is a great little detail to keep in mind before spending the extra money or yelling at the lab when it's not their fault.
So is it correct that if I open a JPEG in CS3 that I edited in LR without exporting that it will open with the LR adjustments assuming I have LR set to update the XMP.
[Correct. --J.]
Is adobe going to consider open sourcing the code that reads the XMP data to allow other apps to see the JPEG the right way?
[I doubt it, as doing so would mean open-sourcing Camera Raw. Instead, we'll enable things like converting JPEG to DNG, which would update the embedded JPEG preview. That way, any software could read the preview data. --J.]
Tom
Tampa Bay Photographer -- Tom Moore Fine Art Photography
John, did you give an ambiguous answer to Tom Moore?
If you update the XMP in the JPEG using Lightroom, you can open it in ACR 4.x and the settings will be applied.
But if you open it DIRECTLY with Photoshop CS3 (not ACR 4.x) they won't be. (Perhaps I am being pedantic?)
[Actually, if you open the file with Photoshop CS3, it'll notice the Lightroom/ACR settings in the file's XMP block & will pop up ACR. This isn't working properly in the public beta, but it will work in the shipping version.
Note that you need to tell Lightroom to update the JPEG's metadata. By default the edits are stored only in the LR database (for the sake of performance), but updating the metadata of one or more selected files is a one-click affair. --J.]
Not apropos of your main issue here, but just FYI I purchased the Canon D30 upon release in October 2000, so that's how long RAW data has been available, at least in my experience.
[Oh yeah--raw has been available for quite a while, but I would say it wasn't mainstream until four or five years ago. FWIW, Camera Raw 1.0 came out four years ago. --J.]
In addition to the issue of possible data corruption (which I admit is minimal for a well-tested program), writing the data to the orginal file has other undesirable properties such as:
1. It causes externally stored checksum/CRC checks to become useless.
2. Is highly undesirable in backup situations with normal backup tools because if even 1 bit in an image is changed, the entire image must be backed up (causing lots of unneccessary disk/network activity to accomplish backups).
If you have hundreds of thousands of images and do backups across the internet, this is especially important.
I wish Lightroom at least had a preference to disable its modification of image files, and work seamlessly with XMP sidecar files for jpeg files.
Also I wish Lightroom had a view that mirrored the file structure except that I could just add multiple directories as roots, and have the program watch those directories (and subdirectories) for the addition of new images. (Multiple root directories are necessary because my images fill up multiple hard drives.) No photo/DAM app seems to get this right.
Another thing that bothers me is that its way too easy to adjust the star ratings on images. iView has this problem too. If you have several hundred thousand images, clicking on an image wrongly and clearing a star rating by accident is almost the equivalent of losing a photo forever! Meta-data should not be changed by an accidental sneeze.
Both Aperture and Lightroom are very sluggish as image browsers, even for jpegs. With ACDSee I can zip through 3 or more full-screen images per second, even on a several year old computer. I've never found a photo browser than can go through images as fast as ACDSee.
I use LR for thousands of photos, some jpeg, but now i'm shooting raw.
I really like the external file approach for raw files. The reason i like this is because inline file changes (even if they are "non-destructive") cause the file checksum to change, and the offsite backup system i use is based off of MD5sums (rsync).
I just updated the metadata on about 1500 JPEGs, and now i have to re-sync all 1500 files (even though only a couple of bits have changed per file). This will take about 2.5 days (given average file size). If the XMP files were separate like they are in RAW, the backup would be much faster.
Could you make the ability to save XMP data as external files an option for TIFF, jpeg, and DNG files?
Thanks.
...alex...
Raw to log RGB conversion should really be another kind of "adjustment layer" in Photoshop. Also, the history of changes made in Lightroom should be visible and adjustable in Photoshop. These are so obvious user experience simplification things, why don't you see that? (I guess Adobe should've hired me)
[I guess. --J.]