December 27, 2006

The Secret Life of Smart Filters

Smart Filters–i.e. those that can be adjusted or removed at any time, leaving the underlying pixels unaffected–address what is probably the single longest-standing feature request in Photoshop.  Customers’ response to them has been quite good, but the details of how & why they work as they do may be a little subtle.  For example,

  • Why can’t you paint directly onto a surface that has a Smart Filter applied?
  • Why are you limited to having one filter mask per layer (instead of having one per filter)?
  • Why do Smart Filters add file size?

If you’re interested in the story of how and why Smart Filters came to be as they are, read on.  I find the whole topic of how Photoshop is evolving from a simple "a pixel is a pixel" app into a dramatically more powerful editing pipeline fascinating, but I recognize it’s not everyone’s cup of tea. :-)

Around four years ago (while building CS1), we took a crack at enabling non-destructive filtering.  After talking with quite a few customers, we opted to enable "Filter Layers"–just like adjustment layers, except containing filters.  We quickly encountered a number of problems, both technical and conceptual:

  1. Performance: It’s one thing to run something well on a screen-resolution document, and quite another to run it well on the huge multi-layer images that Photoshop is often asked to handle. Yes, computers keep getting faster, but cameras and other devices generate ever more data, and our collective patience keeps getting shorter.  There were too many times when performance was lousy–say, whey trying to paint layer 20 out of 50 with a bunch of filter layers applied.  We therefore required filter layers to turn off while editing the layers to which they were applied, after which they would be re-applied.  This made for a pretty lame user experience.
  2. File Size: There’s no free lunch: you can either pay as you go (storing both "before" pixels and "after" pixels in a document), or you can pay later (not storing the pixels in the doc, and requiring the filters to be re-applied when opening the doc).  If you don’t store the filtered "after" pixels, then the PSD file can’t be opened by apps that don’t have the same filters installed (bad for backward compatibility, cross-app integration, etc.). We opted to store the after pixels, but because adjustment layers are very small, people were then surprised that something that looked similar would add file size.
  3. Editability: This may be the toughest of the three, because it’s conceptual, not technical.  If you apply a transformation that substantially changes the appearance of the underlying pixels (e.g. Polar Coordinates is one extreme; see screenshot), what should happen when you attempt to paint or otherwise edit those pixels? As you drag a straight line to the right, what should you be seeing through the prism of the filter(s)?

So, with knowledge in hand but not much else, we pulled the plug on filter layers.  As work progressed on CS1, we thought about ways to build a solution that not only addressed the pain points above, but that solved other Photoshop limitations (scaling and warping non-destructively, integrating with Illustrator and other apps, etc.).  That’s when Smart Objects were born.

Okay, about the name: I technically work in marketing, but I hate cheesy names, and making everything "live" this or "smart" that starts to have a whiff of fromage to it.  Finding a simple, factual name proved difficult for a feature that’s really general–one that enables, among other things:

  • Turning one or more layers into something that can be scaled, rotated, or warped losslessly
  • Creating multiple instances, all of which update when one is edited
  • Preserving complex vector data & re-editing it in Illustrator
  • Preserving raw data & re-editing it in Camera Raw
  • Potentially enabling linking to external files
  • Potentially lots more that I can’t get into here/now

So, we went with a pretty generic name–"Smart Objects." The concept is identical to the Smart Objects feature that’s been in GoLive for several versions: place something the app doesn’t understand natively, and on the fly turn it into a native version, while keeping the source & output linked together. People sometimes get hung up on the one implementation difference (GL links whereas PS embeds), but doing so misses the main point: on-the-fly conversion from {whatever data} to native pixels.

As the basis for Smart Filters, Smart Objects address the three pain points of Filter Layers listed above:

  1. For performance, they don’t set the expectation that you can be drawing/painting/moving things while watching the results update on the fly.
  2. For file size, they create a visible, new entity, and they require a conscious choice to create it. And it’s the same hit you take for enabling re-editable scaling and warping.
  3. For editability, again they don’t set the expectation that you can paint while seeing the results applied.  You simply edit the un-filtered/-warped/-scaled pixels, hit save, and then see the transformed results.

That’s why you can’t paint directly onto a filtered surface, and why you can have only one filter mask per Smart Object (because multiple masks would lead you to expect that filters would update while painting on masks underneath them).

This entry is pretty long already, so I’ll cut myself off, but in the next day or two I plan to post an entry that discusses some of the challenges involved in making Smart Objects easier to use.

Posted by John Nack at 5:37 PM on December 27, 2006
Copyright © 2014 Adobe Systems Incorporated. All rights reserved.
Terms of Use | Privacy Policy and Cookies (Updated)