Pixel Bender’s heritage is from the 3D Pixel Shading languages, but we carefully removed most of that to make it specialized for signal processing. The lineage is there though, right below the surface, and Pixel Bender’s utility for doing a lot of math fast makes it really attractive for doing 3D. I’ve written about 3D experiments from the community in the posts Some cool experiments with Pixel Bender and More Cool Pixel Bender Experiments. There has been a lot of really cool work done in the community since then, and I wanted to highlight that:
I’ve been doing some playing around with processing audio using Pixel Bender in Flash and I realized that it was hard to find some working code to get started with. So i wrote up this sample. I tried to do a minimal app that actually did something interesting and would be a start for someone else. To that end, this AIR app sample loads an MP3 file and then the embedded Pixel Bender kernel lets you change the level of the individual channels separately.
The MXML code is below:
All the action is in the ProcessAudio function, that pulls samples from the input file and executes a ShaderJob across them. There is something important to reference here:
I pass the buffer into the shader job as a 2D buffer instead of a buffer with a height of one. This may make less sense logically, but the Flash player breaks the data up by rows for multi-threading, so this should make that perform faster.
Here is the kernel:
One thing to notice here is that rather than using an image2 as input and a pixel2 as output (which may make more sense logically again), I instead just use the buffer layout and process 2 stereo samples at the same time. This should also give you better performance for filters that can do this.