Web Platform Team Blog

Adobe

decor

Making the web awesome

Testing performance in CEF

Adobe is committed to creating the best tools and services to help you use the web effectively. Whenever possible, we use the web platform itself to build these tools and services. For instance, Edge Code and Edge Reflow are built with HTML, CSS and JavaScript on top of the Chromium Embedded Framework (CEF). CEF is an open source project created by Marshall Greenblatt, to which my team also contributes code to. With the transition from CEF1 to CEF3, we had the opportunity to contribute a feature to CEF that has been directly requested by the community: off-screen rendering.

Off-Screen Rendering

Off-screen rendering is useful for embedders who need more flexibility when interacting with the Chromium webview. Using this feature, the embedding application receives paint notifications, while the actual painting is performed in a bitmap buffer. The embedder is then responsible for drawing this buffer on the screen.

A consequence of this model is that the webview loses control of input events and the embedder is in charge of forwarding mouse and keyboard events to the webview. The performance of the off-screen rendering feature was a primary focus of the implementation from the very beginning. A challenge early on was to find a rigorous (yet not overly complex) method of comparing the performance of off-screen rendering with that of the standard rendering in CEF.

Chromium Telemetry Framework

While looking for a tool to test rendering performance in CEF the team found the Chromium Telemetry framework. Telemetry is a Python framework that runs on top of the remote debugging protocol used for performance testing in Chromium. From the set of benchmarks that Telemetry supports, the scrolling benchmark was the most applicable to our use case. This benchmark scrolls the page and gathers metrics about responsiveness, things like megapixels painted per second, number of dropped frames and the frame rate (via the mean-frame-time).

Telemetry can run on any flavour of Chromium and Content Shell. Since CEF3 is based on the new Chromium Content API, to run the scrolling benchmark we had to expose some private window.chrome APIs in JavaScript. These APIs are required for some of the metrics in the scrolling benchmark.

Running Telemetry with CEF

Telemetry depends on the Chromium source code, so a Chrome source checkout is required. A binary release of CEF can be downloaded from here.

To start the scrolling benchmark in CEF one has to run the run_multipage_benchmarks script from src/tools/perf/, in the Chromium source tree, with the following parameters: --browser=exact --browser-executable=path/to/cefclient --output=outputfile.csv smoothness_benchmark path/to/chromium/src/tools/perf/page_sets/top_25.json 

To run Telemetry on CEF with off-screen rendering and hardware acceleration disabled just append the extra switch:
--extra-browser-args="--off-screen-rendering-enabled --disable-accelerated-compositing"

For reliable test results, it is recommended that the benchmark be run against a set of pages copied locally. To record such a copy of the pages, run:/tools/perf/record_wpr --browser=exact --browser-executable=path/to/cefclient path/to/chromium/src/tools/perf/page_sets/top_25.json

Because of a current limitation of the Telemetry framework the path/to/chromium/src/tools/perf/data directory must already be created before attempting to create a local copy of the pages.

What we tested

Since off-screen rendering doesn’t support hardware acceleration, we compared the CEF test application(cefclient) in off-screen rendering mode with on-screen test application(accelerated compositing disabled).

We used the CSV file output format, concatenated the results into another file and then visualized the results with the viewer from path/to/chromium/src/tools/perf/utils/results_viewer/src/results_viewer.html.

Selecting two columns in the rendered html resulted in a graph like this:
CEF_Telemetry_results_old

Results of the investigation

On the Mac port, comparing the results revealed some unexpected behavior: off-screen rendering reported, on average, a smaller mean frame time than standard rendering. This means that more frames were sent to the screen during off-screen rendering in the same time interval.

After some investigation we found that by scrolling inside an off-screen rendering based cefclient, the renderer receives much more scrolling events than it does for a regular cefclient. This caused the renderer to send more frames to the screen. Digging into the code revealed that Chromium used to skip some of the scrolling events received from the system – an optimization that wasn’t available in off-screen rendering.

After addressing the off-screen rendering performance issues so that they were more closey aligned with standard rendering in Chromium, we ran the benchmarks again and received the following results:
CEF_Telemetry_results_new

Conclusions

Testing rendering performance is always a challenge but fortunately the Chromium team did a really good job with the Telemetry framework. Since CEF3 shares the necessary bits with the Chromium content shell, we got Telemetry support inside CEF pretty much for free.
While these capabilities are not yet widely available, it is easy to see the value such a tool would provide for all major web browsers.

This post has been written in collaboration with Ion Rosca.

Comments are closed.