Web Platform Team Blog

Adobe

decor

Making the web awesome

A Visual Method for Understanding WebKit Layout

My last post was an introduction to the WebKit layout code. It was pretty high level, with a focus on documentation. This post is much more hands on: I will explain some changes that you can make to WebKit’s C++ rendering code to be able to see which classes handle which parts of a web page.

Getting and building the code

This is entirely optional, so if you don’t feel like setting this up or don’t have a machine powerful enough to do the build, feel free to skip to the next section. I will put in links to all of the code that I reference throughout the document, so you should be able to follow along with nothing more than the web browser you are using to read this.

Note that this section does not attempt to be a complete guide to setting up a build environment and getting a working build. That would be an entire blog series of its own!

Getting the code

If you’re still with me, before you build anything, you’ll need to get a copy of the source code. WebKit uses Subversion for source control; however, most developers work with the Git mirror. If you’re on Windows, you should jump directly to the¬†Building the code¬†section below, as you need to have the code checked out to a specific location in order for the code to build. Otherwise, you can continue with this section.

Installing Git on Linux and Mac OS X

Installing Git is pretty straightforward on Linux and Mac OS X. On Linux, I’d suggest that you use your distribution’s package manager, but if you can’t do that for some reason, you can download Git from the Git Homepage.

For recent versions of Mac OS X, it is included in the Xcode developer tools, and you will need them to build the code anyways. Once you’ve installed Xcode, you will need to install the command line tools, which is located under Xcode -> Preferences -> Downloads in Xcode 4.5.2, and in a similar location in earlier versions.

Downloading the source

Once you have Git installed, grabbing the source is pretty straightforward. First, make sure you have at least 4 gigs of free space. Then, open a terminal, find a suitable directory, and run the following command:

git clone git://git.webkit.org/WebKit.git WebKit

This will take awhile, but when it’s done, you’ll have your very own copy of the WebKit source code.

That’s great, but I still don’t get this Git thing

Explaining how to use Git is beyond the scope of this post, but here are a couple of resources:

Building the code

WebKit is designed to be able to run on many different platforms. To facilitate this, functions like graphics and networking have abstraction layers that allow an implementation for a specific platform to be plugged in. This combination of the platform independent part of WebKit with the low level bindings for a specific platform is called a port.

WebKit Port Diagram

Note that while there are ports that are operating system specific, like the Mac and Windows ports, there are also ports for platforms like Qt, which is a library that can be used on many different operating systems. The short of it is that in order to build WebKit, you need to choose the proper port to build for your purposes.

The best place to start to learn how to build WebKit is to take a look at the documentation on installing the developer tools to build webkit. It lists out the most common ports with either instructions in the page or links to the Wiki documentation on how to set up a development environment for that port. I would suggest using the Mac port if you’re on a Mac, and the Windows port if you’re on Windows. If you’re on Linux, you can choose between the Gtk, Qt, and EFL ports. If you don’t care or don’t know which one to choose, you could use the Gtk port if you’re using Gnome, and the Qt port if you’re using KDE.

If you’re building a port other than the Mac or Windows port, all of the build instructions are on the wiki page about that port, which I linked to in the paragraph above. Otherwise, you will want to read the build instructions for the Mac and Windows ports.

There are more ports than the ones I mentioned here; however, when doing WebKit development, it is probably easiest to start with one of the ports I mentioned above. If you would like to know about others, there is a list of ports on the WebKit wiki.

Help! I followed all the instructions and it still doesn’t work!

If you can’t get your chosen port to build, or if you’re unsure of which port to choose, there is a page of contact information for the WebKit project. I would suggest starting by sending a message to the webkit-help mailing list, as that will get you in contact with people that should be able to help with any build problems or questions that you may encounter. You can also contact the developers on IRC: the #webkit channel on irc.freenode.net is WebKit central.

Visualizing layout

The layout process determines what features of a web page get drawn into which locations. However, when starting out, it can be hard to link the C++ code to the visual result that you see in your browser. Probably the easiest way to make this connection in a visual way is to modify the code that draws everything to the screen. This drawing step happens after layout, and WebKit calls it painting.

Basics of painting

After layout has finished, the process of painting begins. It happens in a very similar way to layout: each rendering class has a paint method that is analogous to the layout method we discussed in the last post. This paint method is responsible for drawing the current renderer and it’s children onto the display.

The CSS specification defines the painting order for elements, and the WebKit implementation has painting phases for all of these steps. As with many of these things, it is too complex to explain here in its entirety, but the rules of the painting order ensures that all of CSS’s positioning rules are rendered properly. This must be taken into account when thinking of where in the individual paint methods changes should be made.

The paint method can be instrumented to draw the outline of the area painted by the class. This can make it much easier to understand which parts of the code create which parts of the rendered page.

What does it look like?

To generate the above screenshot, RenderBlock::paint method (Look in Source/WebCore/rendering/RenderBlock.cpp for the definition) was instrumented to draw a green border around the block itself.

Drawing the outlines

The C++ code that generated those lovely green outlines is the following:

Color color;
color.setRGB(0, 255, 0);
paintInfo.context->setStrokeColor(color, style()->colorSpace());
paintInfo.context->strokeRect(overflowBox, 2);

Let’s break this down line by line:

Color color;

This creates a Color object. It is hopefully not surprising that this object represents a color, and will be used to set the color that we are going to draw our border with.

color.setRGB(0, 255, 0);

This call sets the color for our Color object. It takes 3 numbers between 0 and 255, each one representing the amount of red, green, or blue that makes up the color. For simplicity, I just went with entirely green, but you could do any sort of fancy thing you like here. Setting the color to different values could come in handy if you want to instrument multiple classes at once: each could have a different color, making it easy to tell them apart.

paintInfo.context->setStrokeColor(color, style()->colorSpace());

Now’s when things get more complex. paintInfo is an instance of PaintInfo that is passed as an argument to the paint method. PaintInfo is a simple structure that contains the state of the painting operation as well as the object used to actually call the native drawing routines. That object is the context member of PaintInfo. This object is actually a facade to provide a platform independent API for drawing. It is declared in platform/graphics/GraphicsContext.h, and there are corresponding platform specific implementations that it uses depending in which WebKit port is in use.

If you have done graphics programming before (Even using HTML’s canvas element), you will probably find the use of the GraphicsContext familiar. The methods may have different names, but the underlying concepts of the API are exactly the same. For example, this setStrokeColor call sets the color that is used by subsequent drawing operations that use this context.

The last piece of this line that I should explain is the style() call. This gets the CSS style information for the RenderBlock, and queries it for the color space. This is one of the things that will need to change when modifying this code to work in paint methods in other objects, since there are some cases when a renderer delegates painting some part of it’s content to another class that itself is not a renderer. In those cases, that class has a reference to the renderer it is attached to, and the style() method can be called via that member. In most cases, this member is called m_renderer, so this call changes to m_renderer->style()->colorSpace(). I give an example of this case later in this post.

paintInfo.context->strokeRect(overflowBox, 2);

This is the call that actually draws the border rectangle. The overflowBox is the dimensions of the block plus any visible overflow: that’s the entire area of the page that is rendered by this block and its children. The second argument to strokeRect is the width of the line used to draw the rectangleA.

Where is this code added?

Of course, this code cannot just be added at random, you need to put it in a specific place in the paint method. As of this writing, the paint method in RenderBlock.cpp looks like the following:

void RenderBlock::paint(PaintInfo& paintInfo, const LayoutPoint& paintOffset)
{
    LayoutPoint adjustedPaintOffset = paintOffset + location();

    PaintPhase phase = paintInfo.phase;

    // Check if we need to do anything at all.
    // FIXME: Could eliminate the isRoot() check if we fix background painting
    // so that the RenderView paints the root's background.
    if (!isRoot()) {
        LayoutRect overflowBox = overflowRectForPaintRejection();
        flipForWritingMode(overflowBox);
        overflowBox.inflate(maximalOutlineSize(paintInfo.phase));
        overflowBox.moveBy(adjustedPaintOffset);
        if (!overflowBox.intersects(paintInfo.rect))
            return;
    }

    bool pushedClip = pushContentsClip(paintInfo, adjustedPaintOffset);
    paintObject(paintInfo, adjustedPaintOffset);
    if (pushedClip)
        popContentsClip(paintInfo, phase, adjustedPaintOffset);

    // Our scrollbar widgets paint exactly when we tell them to,
    // so that they work properly with z-index. We paint after we painted
    // the background/border, so that the scrollbars will sit above
    // the background/border.
    if (hasOverflowClip() && style()->visibility() == VISIBLE &&
        (phase == PaintPhaseBlockBackground ||
        phase == PaintPhaseChildBlockBackground) &&
        paintInfo.shouldPaintWithinRoot(this))
        layer()->paintOverflowControls(paintInfo.context,
            roundedIntPoint(adjustedPaintOffset), paintInfo.rect);
}

I’m not going to go over this one line at a time, as by the time you read this, this method may or may not look the same. However, I would like to share where I placed the code and the rationale behind it so that hopefully you will still be able to instrument the code yourself even if it has changed, and will be able to think about how to instrument the paint methods in other render classes as well.

When looking for a place to put the code in this method, the first thing I needed to find was where I could find a LayoutRect that represents the area that this renderer will paint into. This is the dimensions of the block plus any visible overflow. This value is usually used early on in the paint method to determine if the current block is actually in area to be painted (paintInfo.rect). In RenderBlock::paint, you can see this in the if statement following the helpful comment “Check if we need to do anything at all.”:

if (!isRoot()) {
    LayoutRect overflowBox = overflowRectForPaintRejection();
    flipForWritingMode(overflowBox);
    overflowBox.inflate(maximalOutlineSize(paintInfo.phase));
    overflowBox.moveBy(adjustedPaintOffset);
    if (!overflowBox.intersects(paintInfo.rect))
        return;
}

So overflowBox is our variable, which you might remember from the instrumentation code example above. Now, we need to determine where to put our instrumentation code. In this case, unless we want to move the definition of overflowBox, we don’t have much choice: it must go in the body of the if statement above. Since we don’t want to draw the outline if the block is not in the area to be painted, we should put it after the return statement at the end, like so:

if (!isRoot()) {
    LayoutRect overflowBox = overflowRectForPaintRejection();
    flipForWritingMode(overflowBox);
    overflowBox.inflate(maximalOutlineSize(paintInfo.phase));
    overflowBox.moveBy(adjustedPaintOffset);
    if (!overflowBox.intersects(paintInfo.rect))
        return;

    Color color;
    color.setRGB(0, 255, 0);
    paintInfo.context->setStrokeColor(color, style()->colorSpace());
    paintInfo.context->strokeRect(overflowBox, 2);
}

And now if we build and run the code, we should see an effect like in the screenshot above. It is also very interesting to launch the web inspector with this active.

More complex paint methods

The paint method of RenderBlock makes for a nice example because it is pretty small and it is pretty straightforward to see how to instrument it. (Most of the complexity is nicely factored out into helper methods.) However, this is not the case for the paint method in InlineFlowBox. InlineFlowBox is used to manage flows of inline content, like text.

I am not going to reproduce the entirety of InlineFlowBox::paint here, but you should be able to follow along by either opening the file in your favorite editor if you opted to download the code, or by looking at InlineFlowBox.cpp on Trac.

While this is a more complex paint method, the check if painting should happen is right at the top of the method:

LayoutRect overflowRect(visualOverflowRect(lineTop, lineBottom));
overflowRect.inflate(renderer()->maximalOutlineSize(paintInfo.phase));
flipForWritingMode(overflowRect);
overflowRect.moveBy(paintOffset);

if (!paintInfo.rect.intersects(pixelSnappedIntRect(overflowRect)))
    return;

From this, we can glean two pieces of information, just like we did in the RenderBox example: that overflowRect defines our painted area, and that we should instrument after the if statement that determines if we should paint or not.

Looking at the rest of the method and the plethora of if and return statements, it is less obvious how far into the method our instrumentation code should go. All of those conditions might look very opaque, but you may be happy to know that you don’t need to understand them all to decide where to put the instrumentation code.

Most of the conditions in InlineFlowBox::paint have to do with the different phases of painting. I mentioned earlier that CSS defines the order in which painting should be done: WebKit uses paint phases to implement this. So all these if statements are doing is ensuring that only the proper things are drawn in a given phase. If we were writing production code, we would need to determine in which phase our outline should be drawn in, and make sure that it is only drawn in that phase. However, since this is exploratory code, we can just draw the outline in every phase, which will ensure that it gets shows up on top of any other content rendered by this object. The simplest way to do this is to place the instrumentation code as the first thing after it is determined that painting should happen.

Great! Now we know where the code should go, and we know which variable contains our bounding box, so we can take the code from earlier, changing overflowBox to overflowRect and changing the color to red just in case we want to have this active at the same time as the changes to RenderBlock:

Color color;
color.setRGB(255, 0, 0);
paintInfo.context->setStrokeColor(color, style()->colorSpace());
paintInfo.context->strokeRect(overflowRect, 2);

That’s the right direction, but if you do this, you will find out that it doesn’t compile: the compiler can’t find a style() method! That’s because we’ve discovered that InlineFlowBox is not a render object, it is a helper, used by render objects like RenderInline.

Luckily, these helper objects have a reference to their render object, accessible via the m_renderer instance variable. This allows us to make one more change to our code which will allow it to compile:

Color color;
color.setRGB(255, 0, 0);
paintInfo.context->setStrokeColor(color, m_renderer->style()->colorSpace());
paintInfo.context->strokeRect(overflowRect, 2);

And then you can see the InlineFlowBoxes in all their glory:

What’s next?

If you didn’t get a working build before reading the whole thing, I would suggest going back, setting up a build, and then trying out the code. It really is a great way to learn how layout (and even a bit of painting) works.

If you have a working build and were following along at home, you should try instrumenting the paint methods of some of the other classes in Source/WebCore/rendering (InlineBox and InlineTextBox are good places to start), and see how these classes map to the visual elements of a web page. You might be surprised by what you discover.

And if that isn’t enough for you, there are plenty of issues on bugs.webkit.org that you could dig into. (And if you can’t find something you find interesting there, I’m sure that someone on the #webkit channel on irc.freenode.net could give suggestions.) After all, there’s no better way to learn than to really work with the code.

Of course, I will be also writing more posts here as well. Stay tuned for my next post on WebKit’s layout tests and the infrastructure that runs them.

4 Comments

  1. February 09, 2013 at 12:29 pm, T. Reiss said:

    Great article! Minor typo :)
    > To generate the above screenshot, RenderBlock::paint method (Look in Source/WebCore/RenderBlock.cpp for the definition)
    Should be “Source/WebCore/rendering/RenderBlock.cpp”. Link is correct, though.

    • February 11, 2013 at 8:49 am, Bem Jones-Bey said:

      Thanks! I’ve updated it.

  2. August 08, 2013 at 4:50 pm, Sunho said:

    One of great, great post !

    Build webkit, and the result of your code was great.
    Your code was great too for pages in japanese vertical writing mode.
    (But I am not a japanese.)

    And I have 1 question.
    Is it possible to draw rect for each character on a page ?
    Is it possible for javascript code to get each rect info for each line box on a page ?
    Or, is it possible for javascript code to get each rect info for each character on a page ?

    Really thanks again for great post !

    • August 14, 2013 at 8:24 am, Bem Jones-Bey said:

      Glad you found the post useful!

      As for your question: I’m not sure if it’s possible to draw a rect around each character on a page. I would suggest asking your question on the webkit-help mailing list, as there are many people more knowledgeable than me on there.

      I’m pretty sure that it isn’t possible to get access to the line boxes or each character’s box from JavaScript. Usually the way we do something similar is to wrap the text we care about in span tags so that they can be accessed from JavaScript.