Inside Adobe Reader Protected Mode – Part 4 – The Challenge of Sandboxing

Hi, Scott Stender from iSEC Partners here.   I was invited by the Adobe Secure Software Engineering Team (ASSET) to comment on our analysis of the sandbox through several stages in its development.  Of course, numerous individuals — at Adobe, iSEC and Matasano — were involved in its assessment, so please take this as one person’s perspective.  Even so, I would be remiss if I didn’t acknowledge the great work of Andreas Junestam, Andrew Becherer, Alex Vidergar, Chris Clark, and Justine Osborne of iSEC Partners, as well as the good folks at Matasano and Adobe who worked closely with us.

Creating a sandbox is perhaps the most difficult security engineering task one can undertake.   Some readers will take immediate objection to that statement – documentation is readily available online for using restricted tokens, chroot jails, and other sandbox building materials.   While it is indeed simple to place a restricted wrapper around a minimal service or piece of demonstration code, placing large applications, with all of their dependencies, in a sandbox presents an entirely different challenge.

A perfect sandbox is akin to the perfectly secure computer – the “…one buried in concrete, with the power turned off and the network cable cut.”  A sandbox is distinguished by the restrictions it places on a piece of running code.  Software, on the other hand, is evaluated based on its usefulness.  Balancing these competing goods – preventing bad software from doing bad things, while allowing good software to be useful – is the impossible challenge the sandbox engineer faces.

The Adobe Reader team has undertaken what may well be the most ambitious attempt to sandbox a Windows application to date:  Millions of lines of existing code, a third-party development platform, every multimedia technology under the sun, all now running in an entirely new execution environment.  It is hard to overstate the challenge of doing so and the accomplishment of getting there.

Securing the Sandbox
Our role as security advisors was to provide three major services to Adobe throughout the development of the sandbox: design review, code review, and penetration testing.

The design review was performed early in the development of the sandbox, with our feedback covering everything from options to provide process-level isolation to the best way to secure an individual API call.

Integrating all supported features from Adobe Reader 9 into the sandboxed version was a priority for Adobe.  As design reviewers we were often called on to suggest mechanisms to support sometimes scary functionality in a controlled fashion.  It was during this stage that we first encountered a recurring challenge – supporting a sandbox across multiple versions of Windows. Let me explain.

Adobe Reader was designed to support every Windows operating system from Windows XP to Windows 7.  Technical readers are well aware that a major change was made to the Windows security model in Windows Vista – User Account Control (UAC).  Less technical readers will know this feature by the pop-ups that request permission to execute sensitive programs.  An obvious challenge is creating a sandbox without the inherent isolation that UAC provides. A less obvious challenge is supporting  Windows XP without system-wide changes that were made to accommodate restricted processes.

iSEC performed reverse engineering and directed testing to identify workarounds that allowed the sandbox to work seamlessly on all supported platforms.  Once a basic level of functionality was achieved, the bulk of our design work went into feature-by-feature reviews of those areas that required coordination with the broker process.  Thankfully, we were able to test our own design suggestions in the code review and testing phases, providing an important feedback loop throughout our project.

The code review was surprisingly limited for a project of this size.  Our scope was the sandbox infrastructure, not the whole of Adobe Reader, so our targets for formal review were only on the order of tens of thousands of lines of sandbox-specific code.  No problem, right? The challenge of code review was that this relatively small amount of critically sensitive code obscured a large amount of underlying complexity.  The broker process is responsible for carrying out sensitive actions on behalf of the sandboxed Reader process; the broker is responsible for validating the request and calling Windows APIs with data passed by the sandbox.  The danger is that these APIs are often not designed to be resilient when called directly by an attacker.

The Windows security model contains several points of control – IPC mechanisms, data received over a network, and so on.  In the general case, calling a library function is not one of those points of control.  For example, take a look at the Platform SDK, find an API that accepts a pointer, and pass a non-NULL, but invalid value.  It is almost certainly going to crash your process, and that’s okay.  In this example, we just caused a denial of service against our own process – not a security bug.  Most programming platforms – UNIX system programming interfaces, the Java SDK, .Net Framework, and others – maintain a similar assumption: A process will not attack itself.

Not so with the sandboxed Adobe Reader.  Put simply, the sandbox and broker introduce a security boundary where one did not exist before.  As the sandbox developer, it is Adobe’s job to make sure that calls across this new security boundary are secure and well-behaved.

Code review had two major goals:  identify implementation-level flaws in the broker process and ensure that API parameter data passed from the sandbox is properly validated before being passed to a Windows API.  The first goal can be accomplished via straightforward code review; the second was an interesting challenge:

The Windows Platform SDK is an abstract API.  Microsoft states that the underlying implementation can change in any version and that only documented functionality is supported for use by developers.  Our goal as security reviewers was to help define what a “safe call” under these circumstances was – across all supported Windows platforms and with all supported callers.  Clearly, documentation review and reverse engineering are helpful. However, we needed to perform some serious testing of our assumptions.  That brings us to the next phase.

Testing provided the most intellectually challenging phase of the project.  In addition to the standard “find bugs” goal of testing, this phase helped define and refine the major attack surfaces of the sandboxed Reader process and its broker.  We focused on three areas: test infrastructure development, directed testing of sandbox and broker, and automated testing of the broker.

Our tests against prototype builds of the Reader process were painfully inefficient.  We used a combination of debugging tools, custom DLL injection, and a lot of process restarts.  After a small amount of testing of this sort, we found that we could eliminate 90% of the labor required by investing in an efficient test infrastructure. iSEC built a suite of tools that hooked directly into the IPC channel between the sandboxed Adobe Reader and its broker process.  These tools allowed us to make raw requests in exactly the same context as the sandboxed process, without any pesky client-side validation of parameters.  Furthermore, our hooks could be controlled by a completely separate process, allowing us to build directed test tools and integrate into Adobe’s existing fuzzing infrastructure.  The combination of fast directed testing and automated testing allowed us to provide broad and deep coverage of broker functionality exposed to the sandbox.  Our findings, in turn, helped modify our code review targets, resulting in a virtuous cycle of improved security on each review/test iteration.

Next Steps for the Sandbox
Engineering is a fundamentally human endeavor.  We do what is useful within our constraints, and iterate through progressively more useful and better-built versions.  It is easy to lose track of this fact amid talk of robust testing and code review.

Engineering the sandbox had its own set of constraints: Some risky functionality was just too important to omit altogether, and some changes to the code base could not be undertaken if we wanted to benefit from a sandboxed Adobe Reader within an acceptable timeframe.  In a threat landscape that constantly changes, there is no limit to the amount of testing one could do. Even as I write this I can think of more tests I would like to run, more investigation into platform technologies, and more isolation mechanisms to employ.  No doubt, some readers of this article have similar ideas and will report them as feedback and flaws to Adobe. Adobe has taken the first step in its attempt to minimize the impact of attacks on Adobe Reader with sandboxing- and it’s a huge step. Now, the Adobe Reader team can continue to refine its new security model based on feedback from users and security experts alike.  I look forward to seeing your thoughts now that Adobe Reader X is in your hands!

Scott Stender
iSEC Partners
scott@isecpartners.com