Kevin Goldsmith

Pixel Bender’s heritage is from the 3D Pixel Shading languages, but we carefully removed most of that to make it specialized for signal processing. The lineage is there though, right below the surface, and Pixel Bender’s utility for doing a lot of math fast makes it really attractive for doing 3D. I’ve written about 3D experiments from the community in the posts Some cool experiments with Pixel Bender and More Cool Pixel Bender Experiments. There has been a lot of really cool work done in the community since then, and I wanted to highlight that:

David Lenaerts has been busy combining Away3D and Pixel Bender, especially for Normal maps: Away3D 3.4.2: Pixel Bender to the max!

Kris Temmerman has also been experimenting with Pixel Bender and Normal Maps.

This is a demo video of a different ray tracer written in Pixel Bender by macduyhai.

Here is an article on integrating Pixel Bender and Papervision3D

Tom Beddard has been doing really cool fractal tools with Pixel Bender. His new Mandelbulbs are really beautiful.

Jerzy Drozda Jr did some experiments with Pixel Bender for UV Mapping in After Effects.

Stay tuned for more links!

I’ve been doing some playing around with processing audio using Pixel Bender in Flash and I realized that it was hard to find some working code to get started with. So i wrote up this sample. I tried to do a minimal app that actually did something interesting and would be a start for someone else. To that end, this AIR app sample loads an MP3 file and then the embedded Pixel Bender kernel lets you change the level of the individual channels separately.

The MXML code is below:

All the action is in the ProcessAudio function, that pulls samples from the input file and executes a ShaderJob across them. There is something important to reference here:

effectShader.data["source"].width = BUFFER_SIZE / 1024;
effectShader.data["source"].height = 512;
effectShader.data["source"].input = shaderBuffer;
effectShader.data["volume"].value = [ leftSlider.value, righttSlider.value ];

var effectJob:ShaderJob = new ShaderJob( effectShader, event.data, BUFFER_SIZE / 1024, 512 );
effectJob.start(true);

I pass the buffer into the shader job as a 2D buffer instead of a buffer with a height of one. This may make less sense logically, but the Flash player breaks the data up by rows for multi-threading, so this should make that perform faster.

Here is the kernel:

One thing to notice here is that rather than using an image2 as input and a pixel2 as output (which may make more sense logically again), I instead just use the buffer layout and process 2 stereo samples at the same time. This should also give you better performance for filters that can do this.

Here are the files:

• PixelBenderAudioDemo.mxml
• Volume1Channel.pbk

For more info, the following references might be helpful

• Tinic Uro’s original post on audio processing with Pixel Bender
• A post on the mediabox forum which has a working version of Tinic’s code
• A post from Саша Рудан that goes into more detail on sound production
• Some Pixel Bender experiments from Mihai Pricope, including an audio mixing sample
• Flash Magazine wrote up a great tutorial on using Pixel Bender for number crunching which got me thinking about doing this sample
• Ryan Taylor also has a great post on doing number crunching and audio processing with Pixel Bender

Just a note for Seattle folks: I’m speaking at the Seattle Flex User’s group meeting on Thursday.

More info: http://seaflexug.org/