An update of the Pixel Bender Toolkit is now available via the Adobe Updater and from the Pixel Bender Developer Connection area on adobe.com. If you have installed After Effects CS5, Flash Professional CS5 or one of the suites that includes these applications, you will see the update listed in the available updates. If you installed the Pixel Bender Toolkit via downloading it on from Adobe.com, you will need to download the update and install it by following the installation instructions.
The majority of effort in the release was on bug fixes found late in the CS5 development cycle. There have been some good bugs reported in the Pixel Bender forums, some are fixed in this release and some will be fixed in a later release.
There has been one major change in the behavior of the Flash preview in this release. In the 2.0 version of the Pixel Bender Toolkit, if you have a kernel with two inputs, the filter was treated as a Flash blend shader. This resulted in some unexpected results, especially when compared to CPU or GPU previews. Now, when you have a two input kernel, we instead create a ShaderJob and use the two input images as inputs. Hopefully, this will make things a bit more consistent. In a future version of the Toolkit, we plan to make this behavior selectable, but the blend case seemed to be very uncommon.
Thanks for using these tools and providing good feedback! If you find any problems or have feature requests, please post them to the Pixel Bender forums!
This is the Spotlight filter from Ryan Phelan. This one isn’t new, I’ve been meaning to post to it for a while. The filter itself is nice, but I also like the UI that Ryan created for editing the spotlight parameters.
Don’t forget to tweet me your links: @pixelbender!
The subject of using Pixel Bender for audio processing on the Flash platform comes up a lot. Audio processing is very processor-intensive and math-heavy, so it would seem natural to leverage Pixel Bender to improve performance of audio within a SWF. At some point, last year, I was talking to Justin Everett-Church about doing a demo for Flash Player 10.1 multi-touch features. A synth seemed like a good idea, and it would let me kill two birds with one stone. So I coded up a synth (with a lame Flex UI) and Justin took that and made it pretty and added multi-touch support. That 2nd part of the demo never worked out the way we meant it too (missed the MAX 2009 keynote by this much). After that, I planned on cleaning up the code and posting it, but I got busy shipping CS5 and well, 7 months later, I finally got around to posting it. Right now it is just the playable demo with the crummy Flex UI.
It is a total processor hog, on purpose. I basically wanted to use it to push the limit of what could be done in the player, so I kept adding more filters and processors to it until the audio started to break up on my Core Duo 2 Mac Book Pro and then stepped back just a little. It turns out that you can actually do a ton of audio processing interactively in the player leveraging Pixel Bender. This was also designed to run as an AIR app, which means that if you really want to play with it, close all your other tabs. Really.
Pixel Bender Synth Experiment
I’ll be posting all the code on the page eventually.
Some techy details:
Each Pixel Bender shader is encapsulated within an Actionscript class and launched via a ShaderJob.
Each Pixel Bender filter processes two samples at once to double throughput.
The synth uses three Pixel Bender Shaders as does the Reverb, the other filters use one.
The delay-based filters (reverb/delay) are using simple bit-bucket style delay in actionscript, but the delayed signals are mixed back with the main signal via Pixel Bender shaders.
All signal paths are stereo. When mixing two stereo inputs, a pixel4 representing the 2 sets of stereo channels is used. When processing/generating a single stereo signal, a pixel4 representing two sequential stereo samples is used.
The filters are ordered as such:
Will post more info soon. Ask any questions in the comments. Thanks!
Pixel Bender’s heritage is from the 3D Pixel Shading languages, but we carefully removed most of that to make it specialized for signal processing. The lineage is there though, right below the surface, and Pixel Bender’s utility for doing a lot of math fast makes it really attractive for doing 3D. I’ve written about 3D experiments from the community in the posts Some cool experiments with Pixel Bender and More Cool Pixel Bender Experiments. There has been a lot of really cool work done in the community since then, and I wanted to highlight that:
I’ve been doing some playing around with processing audio using Pixel Bender in Flash and I realized that it was hard to find some working code to get started with. So i wrote up this sample. I tried to do a minimal app that actually did something interesting and would be a start for someone else. To that end, this AIR app sample loads an MP3 file and then the embedded Pixel Bender kernel lets you change the level of the individual channels separately.
The MXML code is below:
All the action is in the ProcessAudio function, that pulls samples from the input file and executes a ShaderJob across them. There is something important to reference here:
I pass the buffer into the shader job as a 2D buffer instead of a buffer with a height of one. This may make less sense logically, but the Flash player breaks the data up by rows for multi-threading, so this should make that perform faster.
Here is the kernel:
One thing to notice here is that rather than using an image2 as input and a pixel2 as output (which may make more sense logically again), I instead just use the buffer layout and process 2 stereo samples at the same time. This should also give you better performance for filters that can do this.