Archive for October, 2007

October 31, 2007

Cool hydra filters

So, I’ve been meaning to post about some of the cool hydra filters that folks have been posting to their blogs and the Hydra Filter Gallery. I’ve been pretty busy working on the next release of the AIF Toolkit, but the whole team has been really digging some of the cool filters that everyone has been posting. Here are some of my favorites so far (in no particular order)… This is just a subset of what has been posted to the gallery…

Double Plasma by Martin ‘Cifko’ Stefcek – This is just so wonderfully trippy.
Fuzz by Tyler Glaiel – This would be the start of a cool cross-disolve filter.
Colored Spotlight by Andy Zupko – I like this one because it is simple, but very effective.
Spherize by Joa Ebert – The first non-Adobe hydra developer contribution is something pretty cool: a simple, yet powerful spherize filter. Joa has posted a ton of filters, I just had to pick one to be fair.
AIF’s own Mangler mashed up two of Joa’s filters: Technodots and Twirl and came up with this beauty: TechoTwirl
http://www.quasimondo.com/hydra/sineNoise1.jpg SineNoise: The lack of a built-in noise or random function (currently) in Hydra prompted Mario Klingerman to see what he could do to create noise. Very cool. He’s also posted some great gradient generators in the gallery.
Polar Coordinates: This is another fun kernel to play with. by wf
Zeh has created an adjustable threshhold filter that is pretty cool and yet pretty simple and easy to understand.
The Vortex filter by Jan is also amazingly trippy, this thing looks awesome animated.

 

In a future post, I’ll link to some of the stuff that the hydra developers have been posting on their blogs.

 

What are your favorite filters? Also, what kinds of things would you like me to cover in future posts? Let me know in the comments or on the forum.

4:55 PM Permalink
October 23, 2007

The FadeToHistory Hydra Filter in action

I figured that the static pictures were boring, so I decided to show what the FadeToHistory might look like in action.

yes. this is a movie, not some secret feature in Flash Player 9. I left in the rewind Flash to make it explicit.

3:21 PM Permalink

Writing a FadeToHistory Hydra Filter, part 3

Previous parts to this tutorial:
Steps 1-3
Steps 4-6

At this point, we have a hydra filter which does an interactive cross-disolve from a color image to a luminance based black and white image: Download Hydra Filter

The current version of the filter is:

kernel FadeToHistory

{
parameter float crossfade;
const float3 lumMult = float3(0.3, 0.59, 0.11);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float luminance = dot(dst.rgb, lumMult);
dst.rgb = mix(dst.rgb, float3(luminance), crossfade);
}
}

Step 7: Adding the Sepia Conversion

We’re about halfway to being done. We are fading from color to black and white. Time to add the sepia toning so that we can finish this sucker off. There are a bunch of different methods for doing sepia conversion, but essentially the general idea is to mix the red, green and blue channels in different amounts to preserve the original image while giving it that old-time photographic feel. Unlike computing the luminance, we’ll need to produce all three color channels, so the calculation is a bit more cumbersome:

        float3 sepia = float3( dst.r * 0.4 +
dst.g * 0.769 +
dst.b * 0.189,
dst.r * 0.349 +
dst.g * 0.686 +
dst.b * 0.168,
dst.r * 0.272 +
dst.g * 0.534 +
dst.b * 0.131 );

I’ve modified the filter above so that we can test the sepia toning.

kernel FadeToHistory

{
parameter float crossfade;
const float3 lumMult = float3(0.3, 0.59, 0.11);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float luminance = dot(dst.rgb, lumMult);
float3 sepia = float3( dst.r * 0.4 +
dst.g * 0.769 +
dst.b * 0.189,
dst.r * 0.349 +
dst.g * 0.686 +
dst.b * 0.168,
dst.r * 0.272 +
dst.g * 0.534 +
dst.b * 0.131 );
dst.rgb = mix(dst.rgb, sepia, crossfade);
}
}

So now we’ll run the filter and made sure that it compiles without error and does something like what we want. If you set the crossfade parameter to 1.0, you should see something like this:

step 8: doing the full fade

We have black and white, we have sepia, now we want to fade from color to black and white and then to sepia. First, I’ll show how I would do this with Flash compatibility mode off and then I’ll show how I would do it with Flash compatibility on.

With Flash warnings and errors off, I would write the code like this:

kernel FadeToHistory

{
parameter float crossfade<minValue:0.0; maxValue:2.0; defaultValue:0.0;>;

const float3 lumMult = float3(0.3, 0.59, 0.11);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float luminance = dot(dst.rgb, lumMult);
float3 sepia = float3( dst.r * 0.4 +
dst.g * 0.769 +
dst.b * 0.189,
dst.r * 0.349 +
dst.g * 0.686 +
dst.b * 0.168,
dst.r * 0.272 +
dst.g * 0.534 +
dst.b * 0.131 );

float3 startMix = dst.rgb;
float3 endMix = float3(luminance);
float mixValue = crossfade;
if ( crossfade > 1.0 )
{
// normalize mix value to the range of 0-1
mixValue -= 1.0;
startMix = float3(luminance);
endMix = sepia;
}
dst.rgb = mix(startMix, endMix, mixValue);
}
}



If you want to try this, be sure to uncheck the
"Turn on Flash warnings and Errors" in the build menu, or you’ll get the error "ERROR: (line 30): If statements not supported in Hydra byte code"

A few things to note:

  • I’ve added metadata to the crossfade parameter to specify that its range is now from 0.0 to 2.0 instead of the default 0.0 to 1.0
  • I’ve added temporary variables for startMix, endMix and mixValue to simplify the code. By default, they are the original color, the luminance color and the crossfade value.
  • If crossfade is larger than 1.0
    • I overwrite the temporary variable mixValue to be mapped into the range of 0.0 to 1.0 so that we can use that value in the mix function
    • I overwrite the startMix value with luminance color so that is where we mix from instead of the original color
    • I overwrite the endMix value with the sepia color

Now, about that error we get if we try to compile with Flash warnings and errors on. In order to support older graphics cards, we will not support conditional branching (if statements) in Astro. That makes things a bit harder, but in no way impossible. This is how I would rewrite the above filter to get around that limitation:

 

kernel FadeToHistory

{
parameter float crossfade<minValue:0.0; maxValue:2.0; defaultValue:0.0;>;

const float3 lumMult = float3(0.3, 0.59, 0.11);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float3 luminance = float3( dot(dst.rgb, lumMult) );
float3 sepia = float3( dst.r * 0.4 +
dst.g * 0.769 +
dst.b * 0.189,
dst.r * 0.349 +
dst.g * 0.686 +
dst.b * 0.168,
dst.r * 0.272 +
dst.g * 0.534 +
dst.b * 0.131 );

float3 startMix = dst.rgb;
startMix = (crossfade <= 1.0) ? startMix : luminance;
float3 endMix = (crossfade <= 1.0) ? luminance : sepia;
float mixMinusOne = crossfade - 1.0;
float mixValue = (crossfade <= 1.0) ? crossfade : mixMinusOne;
dst.rgb = mix(startMix, endMix, mixValue);
}
}

Instead of doing conditional branching in Flash, I can do conditional assignments of pre-computed values. What this means is that I have to do a little more computation, but I can still get the results I want. Instead of only doing computation in an if or else statement, I do all the computation in the filter and then only assign the result to the final variables if the condition is met. Of course you can do conditional assignments in regular Hydra too to be clear.

The big changes in this version of the filter is that I have a couple of extra statements and an extra variable, but the output is exactly the same.

All we have left to do is a bit more cleanup and then we’re done!

Step 9: final cleanup

That sepia calculation is annoying. There is a lot of math there and it takes up a lot of space. Doing these kinds of calculations can be pretty common in image and video processing and there is a simpler way to do them in Hydra. Doing this kind of operation where we combine each channel of a color with a blend of all of the colors added together is exactly the same thing we do when we multiply a vector by a matrix. Matricies are built-in to Hydra and you can use them like any other type. This is how that looks:

 

kernel FadeToHistory

{
parameter float crossfade<minValue:0.0; maxValue:2.0; defaultValue:0.0;>;

const float3 lumMult = float3(0.3, 0.59, 0.11);
const float3x3 sepiaMatrix = float3x3(0.400, 0.769, 0.189,
0.349, 0.686, 0.168,
0.272, 0.534, 0.131);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float3 luminance = float3( dot(dst.rgb, lumMult) );
float3 sepia = dst.rgb * sepiaMatrix;

float3 startMix = dst.rgb;
startMix = (crossfade <= 1.0) ? startMix : luminance;
float3 endMix = (crossfade <= 1.0) ? luminance : sepia;
float mixMinusOne = crossfade - 1.0;
float mixValue = (crossfade <= 1.0) ? crossfade : mixMinusOne;

dst.rgb = mix(startMix, endMix, mixValue);
}
}

Once again, I’ve used a const for sepiaMatrix because it stays the same from pixel to pixel. Now everything looks cleaner and simpler. It is easier to understand and more importantly, it is easier to mess around with and make changes. Download the final version and try it for yourself!

Thanks for following this tutorial and let me know in the comments if you have questions or would like other tutorials.

12:13 PM Permalink
October 18, 2007

Writing a FadeToHistory Hydra Filter, part 2

When we left off in part 1, we had a filter that took a color image and then created a black and white image by using the red channel from original image for the red, green and blue channels of the output image. Going from this:

to this:

And here is the hydra filter that we left off with:

kernel FadeToHistory

{
void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord()).rrra;
}
}

If you would like to download what we have so far, a link to the above hydra filter is here: FadeToHistory2.hydra

Now, onward and upward!

Part Four: Making it interactive

We now have a filter that takes a color image and does a simple conversion to black and white. This is nice, but we could do it in Photoshop and use that instead. Where Hydra gets cool is by allowing you to interact with the filters, automate and animate them. Let’s add a parameter to the filter that will allow us to animate the image from color to black and white. The new filter looks like this:

kernel FadeToHistory

{
parameter float crossfade;

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float3 bw = dst.rrr;
dst.rgb = ((1.0 - crossfade) * dst.rgb) + (crossfade * bw);
}
}

There are a few changes here:

  1. parameter float crossfade; This line tells the compiler that you want a parameter to your filter that will be exposed to the application hosting your filter. In this case, a float parameter. If you run the filter above, you will see a slider in the area to the right of your image. Moving this slider changes your parameter value.
  2. dst = sampleNearest(src,outCoord()); We’ve reverted this back to what it used to be when we did new filter. We’re going to use the dst variable to store the color version of the image.
  3. float3 bw = dst.rrr; I’ve created a temporary variable to store the black-and-white version of the rgb channels from the original image. Float3 is an array of three float values. In Hydra, pixel values are stored as floats.
  4. dst.rgb = ((1.0 – crossfade) * dst.rgb) + (crossfade * bw); This is a simple crossfade calculation using the crossfade variable to determine the amount. When the crossfade parameter is 0, you get the color image. When the crossfade parameter is 1, you get the black and white image. We overwrite the red, green, and blue colors from the original image with the mix of color and black and white values based on the crossfade parameter.

Part Five: Better Black and White conversion

If you’ve tried this Hydra kernel on any image without a lot of red content in it, the results are pretty unsatisfying. Time to do a better conversion.

kernel FadeToHistory

{
parameter float crossfade;

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float luminance = dst.r * 0.3 + dst.g * 0.59 + dst.b * 0.11;
dst.rgb = ((1.0 - crossfade) * dst.rgb) + (crossfade * float3(luminance));
}
}

There are a couple of important things here:
float luminance = dst.r * 0.3 + dst.g * 0.59 + dst.b * 0.11;

This line computes a single float value that is a combination of the red, green and blue colors from the original image. This formula is pretty similar to how color TV images are converted for old black and white TV screens. The values are based on how your eyes perceive color. If you used 0.33 for each of the multipliers, you’d actually have something that looked kind of wrong because your eyes don’t perceive blue and red in the same way you see green. For more info, check out this wikipedia article.

float3(luminance)
Here we create a new float3 variable in-line where each of the three indicies are set to the value of luminance.

Part Six: A Bit of Clean Up

For this last part of this post, I’m going to make a couple changes to clean up the code. These will simplify some things as well as show some more built in functions for Hydra.

kernel FadeToHistory

{
parameter float crossfade;
const float3 lumMult = float3(0.3, 0.59, 0.11);

void
evaluatePixel(in image4 src, out pixel4 dst)
{
dst = sampleNearest(src,outCoord());
float luminance = dot(dst.rgb, lumMult);
dst.rgb = mix(dst.rgb, float3(luminance), crossfade);
}
}

With this iteration, I’ve created a const variable:
const float3 lumMult = float3(0.3, 0.59, 0.11);
This variable contains the numbers I multiply red, green, and blue by to get the luminance value. Since these numbers do not change ever, I can put them in a const variable. This simplifies the code and allows the compiler to do some optimization because I’m telling the compiler that this value will not change.

float luminance = dot(dst.rgb, lumMult);
I’ve changed from multiplying the red, green and blue components of the input image by the luminance calculation and then adding them together to using the built-in dot function which does exactly that. Dot specifies the dot product of two vectors, which is computed by multiplying the components of the vectors and then adding them up (which is exactly what we were doing to compute the luminance value). Fore more info on dot products, this wikipedia article is extensive.

dst.rgb = mix(dst.rgb, float3(luminance), crossfade);
Since operations like the crossfade that I was doing before are pretty common in image processing, we’ve have a built-in function to do linear interpolation between two values. This is the mix function, and I use it here rather than doing the computation myself.

At the end of part 6, we have a kernel with a parameter that lets us smoothly animate from a color image to a luminance-based black and white image. To play with the filter, you can download the Hydra file: Download the Hydra file

5:13 PM Permalink
October 17, 2007

Writing a FadeToHistory Hydra Filter, part 1

So, I promised a walk through tutorial of the code I wrote in our MAX talk. It’s taken me a while to get some time, but here I go.

So the filter I’m going to write works like this: I want to take a color image and then fade it to black and while and then continue to fade into a sepia-tone. Sort of a Ken Burns-ish thing. I’m going to walk through how I would do this iteratively, starting from the default new filter you get by choosing “New Filter” and then expand on it until I get my final goal.

To Illustrate

we’ll create a filter that goes smoothly from this:
start image

To This:
middle image

And finally to this:
end image

Continue reading…

5:32 PM Permalink
October 8, 2007

AIF Toolkit and Hydra slides from MAX posted

Hey all, I’m still getting out from under the mountains of mail I received during the MAX conference, so I’ll be posting more on MAX and Hydra (tons of blog postings in the community!) soon. In the meantime, I wanted to let you know that we’ve posted our slides from our talk at MAX as a PDF. Download it here.

I’ll be posting the code that I wrote live on the stage to the labs wiki soon.

4:47 PM Permalink
October 1, 2007

Joa Ebert becomes the first 3rd party hydra developer

Joa Ebert has published the first non-Adobe hydra filter, while we’re still updating the Wiki! It’s simple, but it’s pretty cool considering that the toolkit has only been available for a couple of hours!

What can you make between now and Wednesday?

Our talk again…
Wednesday, October 3rd @ 3pm
Room W-183B

2:07 PM Permalink

It’s Alive! (Hydra and Astro)

This morning, at the close of the MAX keynote, we formally announced that the AIF Team’s image processing language, code-named “Hydra”, will be part of Astro AKA Flash Player 10. Emmy and Justin from the Flash team demoed an application called the Adobe Image Foundation Toolkit Technology Preview. That tool is available from Adobe Labs NOW.

We are super-excited to get this technology into your hands to see what you can do with it!

The application that we have made available will let you build hydra filters and see them run on your images. The AIF team wants your feedback on the language and the tools so that we can continue to improve them before their final release.

While this technology is new to Flash, it isn’t new to Adobe products, we first shipped this technology in After Effects CS3. That said, there has been a ton of effort to get this release ready for you to play with and it feels great to see it on the big screen in front of a crowd of enthusiastic developers!

Now that the cat is out of the bag, so to speak, I’ll be posting more on this blog about Hydra and Astro so stay tuned. You can also post questions and comments on the labs forum.

Don’t forget our Talk about the AIF Toolkit and Hydra! Wednesday, 3pm in Room
Room W-183B

11:47 AM Permalink