Adobe

decor

Web Platform Team Blog

Making the web awesome

A look into Custom Filters reference implementation

Note: Custom Filters is no longer supported in Chrome or WebKit Nightlies.

A shader example

Over the past two years, my team in Adobe has been actively working on the CSS Custom Filters specification (formerly CSS Shaders), which is just one part of the greater CSS Filters specification. Alongside the spec work, we have been working on the CSS Custom Filters WebKit implementation, so I’ve decided to write this blog post to explain how this is all being implemented in WebKit, giving an high-level conceptual overview of the game.

Support for OpenGL shaders on general DOM content has been one of the noted features for the latest Chrome releases. Users and web developers are excited about the ability to apply cinematic effects on their pages.

Shaders, Shaders, Shaders

I’ve often been asked “what is a shader?”; well, shaders are short programs that run on the GPU that – unlike regular programs – solely deal with bitmaps (textures) and 3D meshes, performing specific operations on pixels (called fragments) and vertices. You can get a more in-depth explanation of shaders on Wikipedia.

Roughly speaking, the reason why these small programs run on the your graphic card’s processor (the GPU) instead of the regular CPU is that GPUs are very good at performing ad hoc math operations in parallel. While the GPU is busy performing graphical operations, your CPU is idle and ready to perform other tasks. Everything is faster and snappier.

In our case, the texture passed to the graphic pipeline is styled DOM content, your page content. This means that with one line of CSS and a shader, you can get splendid and astonishing results, just take a look at the image at the beginning of this post.

HTML5Rocks has a great post covering the whole idea behind CSS Custom Filters with an example of their capabilities so I won’t spend too many words on how to use them, instead I’ll be focusing on what is the general idea behind the implementation. Let’s start!

Parsing and Style Resolution

The life of a CSS Shader Custom Filter starts with the CSS Custom Filter declaration. With the currently implemented WebKit syntax, it might look like the following:

#myElement {
    -webkit-filter: custom(url(vertex.vs) none)
}

This CSS is firstly encountered by WebKit’s CSSParser, which – as the name might suggest – is the component within WebKit responsible for CSS parsing. Raw text data gathered from this object is then computed by WebKit’s StyleResolver and from there, a CustomFilterOperation is created. This class will contain all the major information about the Custom Filter you’ve asked to use, such as its parameters. It also references the program, a CustomFilterProgram, which is a vertex shader and fragment shader pair, that will be applied to the texture (your styled DOM content).

Here’s a picture of the happy event:

PassRefPtr StyleResolver::createCustomFilterOperation(WebKitCSSFilterValue* filterValue)
{
    // ...snip...
    RefPtr program = StyleCustomFilterProgram::create(vertexShader.release(), fragmentShader.release(), programType, mixSettings, meshType);
    return CustomFilterOperation::create(program.release(), parameterList, meshRows, meshColumns, meshType);
}

Computing and Rendering

The RenderLayer tree is one of the many trees in WebKit and it’s currently being used, among other things, to deal with group operations such as opacity, stacking contexts, transforms and to do things like scrolling and accelerated compositing: it’s a huge huge piece of code that defines the word “complexity”.:-)

What happens after the parsing and the style resolution is most interesting: when a DOM element has a filter associated with it, it receives its own RenderLayer. When this happens, the relevant DOM content is painted in an image buffer and then sent to the GPU for the “shading treatment” as a regular texture. All this starts to take place thanks to RenderLayerFilterInfo which, among other things, downloads the shaders as external resources and calls the RenderLayer back to notify it that everything’s ready to repaint.

The computing magic begins when a RenderLayer starts to iterate on all the filters associated with it:

FilterOperations RenderLayer::computeFilterOperations(const RenderStyle* style)
{
	// ...snip...
        for (size_t i = 0; i < filters.size(); ++i) {
        RefPtr filterOperation = filters.operations().at(i);
        if (filterOperation->getOperationType() == FilterOperation::CUSTOM) {
	// ...snip...
            CustomFilterGlobalContext* globalContext = renderer()->view()->customFilterGlobalContext();
            RefPtr validatedProgram = globalContext->getValidatedProgram(program->programInfo());
	// ...snip...
}

For every custom filter, a validated version of the program (see next paragraph) is requested; this is then appended to the list of the associated filter operations.

The actual rendering will take place in the FilterEffectRenderer class where the “filter building” operation will start in a surprisingly straightforward manner:

bool FilterEffectRenderer::build(RenderObject* renderer, const FilterOperations& operations)
{
	// ...snip...
        effect = createCustomFilterEffect(this, document, customFilterOperation);
	// ...snip...
}

FilterEffectRenderer::createCustomFilterEffect is the method that will rule them all. Take a look at it, it’s just gorgeous! :)

A word about security

Among the other things, shaders deal with pixels which means that a texture can be sampled (read). This translates into a potential security risk: we don’t really like the idea of malicious shaders reading your page content. Imagine to what would happen if anyone was able to push a shader capable of making a screenshot of your private messages on some social network.

In light of this we do enforce restrictions on shader code before it’s sent to the GPU. We do this by validating and rewriting the shaders in a controlled and secure fashion, and CustomFilterValidatedProgram serves just this purpose, one of its goals is to avoid texture sampling, meaning that the shaders cannot read pixels from your page content anymore. Most of the validation work actually relies on ANGLE, an open source library used by many other projects, including WebKit and Gecko, for WebGL validation and portability.

If you want to have an idea of the kind of validation we perform, pay attention to CustomFilterValidatedProgram: it might prove a very learning reading.

Wrapping up

Currently, we’re working with the CSS Working Group to make the CSS Custom Filters syntax more elegant and future-proof. As with any new web feature, you can expect changes. Stay tuned for more information on upcoming improvements!

Comments are closed.