by Kevin Goldsmith


August 7, 2009

We tried to be semantically agnostic in the original design of the Pixel Bender language. We’d seen other languages go down rabbit holes of over-specification around what parameters really meant, locking them into archaic and insufficient implementations or clumsy hierarchies of meanings. We didn’t want to limit Pixel Bender developers into what we could conceive at the time of the invention of the language.

We were being a bit too idealistic :). It is completely true that the community of Pixel Bender developers continues to blow our minds at what they accomplish with the language and we never would have anticipated half the stuff that you guys are using Pixel Bender for. However, having some generally useful semantic meanings for Pixel Bender parameters would definitely help those who design general user interfaces for Pixel Bender filters.

One smart thing we did (if I do say myself) was to allow parameter and kernel metadata to be extensible. It provided developers like After Effects and Picnik a way to add custom metadata to Pixel Bender that specified semantics or actual UI controls for Pixel Bender kernels and graphs. The picnik metadata and the After Effects metadata are different though. I started to get concerned that by not having something around this that we could end up with many different Pixel Bender semantic metadata mechanisms floating around. To that end, I created the proposal below and started floating it around Adobe and some of the sites that are heavy users of Pixel Bender.

In this design, I tried to follow some general rules:

  • Pixel Bender is not only a way to write image and video filters for Adobe applications, but also a way for you to host 3rd party filters in your Flash-based applications. Any guidelines we create need to be implemented by us, but also by any independent Flash developers creating apps that use Pixel Bender kernels. To that end, I tried to keep the design relatively simple so that it wouldn’t be too difficult to implement in Flash.
  • This proposal adds semantic metadata, but avoids specifying specific user interface controls. Rather than specifying a slider as an editor for a parameter, I think it makes more sense to say that the parameter is a percentage and allow someone designing a UI to make a good percentage editor. We are definitely thinking about how to specify custom editor UIs for Pixel Bender filters, but this proposal does not approach that.
  • The proposal is not a complete answer for all Pixel Bender filters. I’m trying to get the most universal semantics represented. The most generally useful, as opposed to trying to give the complete solution. If you have an application that is uses Pixel Bender kernels for something that makes sense to augment the metadata you can still choose to do so.
  • The proposal represents guidelines, not requirements. As a developer consuming Pixel Bender kernels, you can choose to enumerate the metadata as described in the proposal or not. The suggested metadata is metadata. It is not required. The goal is that if you wish to take advantage of it when it is present, that you can provide a more compelling user interface to your users because you understand the intent of a parameter, not just its type.

There are a couple open questions in the proposal that I’d like feedback on. They are called out pretty clearly. I’d really like your feedback on this proposal. I hope to issue the final version soon. Reply to this post or in the Pixel Bender forums on Adobe Labs. If you decide to post a reply on your own blog, please post a link to your post here or in the forum. Right now trackbacks are off so I won’t know about it otherwise. You can also tweet a reply to the Pixel Bender twitter account (if you can fit it in 140 characters 🙂 ).

Pixel Bender Metadata Hinting for User Interface Guidelines public proposal.pdf