by Kevin Goldsmith
I’ve been doing some playing around with processing audio using Pixel Bender in Flash and I realized that it was hard to find some working code to get started with. So i wrote up this sample. I tried to do a minimal app that actually did something interesting and would be a start for someone else. To that end, this AIR app sample loads an MP3 file and then the embedded Pixel Bender kernel lets you change the level of the individual channels separately.
The MXML code is below:
All the action is in the ProcessAudio function, that pulls samples from the input file and executes a ShaderJob across them. There is something important to reference here:
effectShader.data["source"].width = BUFFER_SIZE / 1024; effectShader.data["source"].height = 512; effectShader.data["source"].input = shaderBuffer; effectShader.data["volume"].value = [ leftSlider.value, righttSlider.value ]; var effectJob:ShaderJob = new ShaderJob( effectShader, event.data, BUFFER_SIZE / 1024, 512 ); effectJob.start(true);
I pass the buffer into the shader job as a 2D buffer instead of a buffer with a height of one. This may make less sense logically, but the Flash player breaks the data up by rows for multi-threading, so this should make that perform faster.
Here is the kernel:
One thing to notice here is that rather than using an image2 as input and a pixel2 as output (which may make more sense logically again), I instead just use the buffer layout and process 2 stereo samples at the same time. This should also give you better performance for filters that can do this.
Here are the files:
For more info, the following references might be helpful
- Tinic Uro’s original post on audio processing with Pixel Bender
- A post on the mediabox forum which has a working version of Tinic’s code
- A post from Саша Рудан that goes into more detail on sound production
- Some Pixel Bender experiments from Mihai Pricope, including an audio mixing sample
- Flash Magazine wrote up a great tutorial on using Pixel Bender for number crunching which got me thinking about doing this sample
- Ryan Taylor also has a great post on doing number crunching and audio processing with Pixel Bender