Archive for November, 2010

Inside Adobe Reader Protected Mode – Part 4 – The Challenge of Sandboxing

Hi, Scott Stender from iSEC Partners here.   I was invited by the Adobe Secure Software Engineering Team (ASSET) to comment on our analysis of the sandbox through several stages in its development.  Of course, numerous individuals — at Adobe, iSEC and Matasano — were involved in its assessment, so please take this as one person’s perspective.  Even so, I would be remiss if I didn’t acknowledge the great work of Andreas Junestam, Andrew Becherer, Alex Vidergar, Chris Clark, and Justine Osborne of iSEC Partners, as well as the good folks at Matasano and Adobe who worked closely with us.

Creating a sandbox is perhaps the most difficult security engineering task one can undertake.   Some readers will take immediate objection to that statement – documentation is readily available online for using restricted tokens, chroot jails, and other sandbox building materials.   While it is indeed simple to place a restricted wrapper around a minimal service or piece of demonstration code, placing large applications, with all of their dependencies, in a sandbox presents an entirely different challenge.

A perfect sandbox is akin to the perfectly secure computer – the “…one buried in concrete, with the power turned off and the network cable cut.”  A sandbox is distinguished by the restrictions it places on a piece of running code.  Software, on the other hand, is evaluated based on its usefulness.  Balancing these competing goods – preventing bad software from doing bad things, while allowing good software to be useful – is the impossible challenge the sandbox engineer faces.

The Adobe Reader team has undertaken what may well be the most ambitious attempt to sandbox a Windows application to date:  Millions of lines of existing code, a third-party development platform, every multimedia technology under the sun, all now running in an entirely new execution environment.  It is hard to overstate the challenge of doing so and the accomplishment of getting there.

Securing the Sandbox
Our role as security advisors was to provide three major services to Adobe throughout the development of the sandbox: design review, code review, and penetration testing.

The design review was performed early in the development of the sandbox, with our feedback covering everything from options to provide process-level isolation to the best way to secure an individual API call.

Integrating all supported features from Adobe Reader 9 into the sandboxed version was a priority for Adobe.  As design reviewers we were often called on to suggest mechanisms to support sometimes scary functionality in a controlled fashion.  It was during this stage that we first encountered a recurring challenge – supporting a sandbox across multiple versions of Windows. Let me explain.

Adobe Reader was designed to support every Windows operating system from Windows XP to Windows 7.  Technical readers are well aware that a major change was made to the Windows security model in Windows Vista – User Account Control (UAC).  Less technical readers will know this feature by the pop-ups that request permission to execute sensitive programs.  An obvious challenge is creating a sandbox without the inherent isolation that UAC provides. A less obvious challenge is supporting  Windows XP without system-wide changes that were made to accommodate restricted processes.

iSEC performed reverse engineering and directed testing to identify workarounds that allowed the sandbox to work seamlessly on all supported platforms.  Once a basic level of functionality was achieved, the bulk of our design work went into feature-by-feature reviews of those areas that required coordination with the broker process.  Thankfully, we were able to test our own design suggestions in the code review and testing phases, providing an important feedback loop throughout our project.

The code review was surprisingly limited for a project of this size.  Our scope was the sandbox infrastructure, not the whole of Adobe Reader, so our targets for formal review were only on the order of tens of thousands of lines of sandbox-specific code.  No problem, right? The challenge of code review was that this relatively small amount of critically sensitive code obscured a large amount of underlying complexity.  The broker process is responsible for carrying out sensitive actions on behalf of the sandboxed Reader process; the broker is responsible for validating the request and calling Windows APIs with data passed by the sandbox.  The danger is that these APIs are often not designed to be resilient when called directly by an attacker.

The Windows security model contains several points of control – IPC mechanisms, data received over a network, and so on.  In the general case, calling a library function is not one of those points of control.  For example, take a look at the Platform SDK, find an API that accepts a pointer, and pass a non-NULL, but invalid value.  It is almost certainly going to crash your process, and that’s okay.  In this example, we just caused a denial of service against our own process – not a security bug.  Most programming platforms – UNIX system programming interfaces, the Java SDK, .Net Framework, and others – maintain a similar assumption: A process will not attack itself.

Not so with the sandboxed Adobe Reader.  Put simply, the sandbox and broker introduce a security boundary where one did not exist before.  As the sandbox developer, it is Adobe’s job to make sure that calls across this new security boundary are secure and well-behaved.

Code review had two major goals:  identify implementation-level flaws in the broker process and ensure that API parameter data passed from the sandbox is properly validated before being passed to a Windows API.  The first goal can be accomplished via straightforward code review; the second was an interesting challenge:

The Windows Platform SDK is an abstract API.  Microsoft states that the underlying implementation can change in any version and that only documented functionality is supported for use by developers.  Our goal as security reviewers was to help define what a “safe call” under these circumstances was – across all supported Windows platforms and with all supported callers.  Clearly, documentation review and reverse engineering are helpful. However, we needed to perform some serious testing of our assumptions.  That brings us to the next phase.

Testing provided the most intellectually challenging phase of the project.  In addition to the standard “find bugs” goal of testing, this phase helped define and refine the major attack surfaces of the sandboxed Reader process and its broker.  We focused on three areas: test infrastructure development, directed testing of sandbox and broker, and automated testing of the broker.

Our tests against prototype builds of the Reader process were painfully inefficient.  We used a combination of debugging tools, custom DLL injection, and a lot of process restarts.  After a small amount of testing of this sort, we found that we could eliminate 90% of the labor required by investing in an efficient test infrastructure. iSEC built a suite of tools that hooked directly into the IPC channel between the sandboxed Adobe Reader and its broker process.  These tools allowed us to make raw requests in exactly the same context as the sandboxed process, without any pesky client-side validation of parameters.  Furthermore, our hooks could be controlled by a completely separate process, allowing us to build directed test tools and integrate into Adobe’s existing fuzzing infrastructure.  The combination of fast directed testing and automated testing allowed us to provide broad and deep coverage of broker functionality exposed to the sandbox.  Our findings, in turn, helped modify our code review targets, resulting in a virtuous cycle of improved security on each review/test iteration.

Next Steps for the Sandbox
Engineering is a fundamentally human endeavor.  We do what is useful within our constraints, and iterate through progressively more useful and better-built versions.  It is easy to lose track of this fact amid talk of robust testing and code review.

Engineering the sandbox had its own set of constraints: Some risky functionality was just too important to omit altogether, and some changes to the code base could not be undertaken if we wanted to benefit from a sandboxed Adobe Reader within an acceptable timeframe.  In a threat landscape that constantly changes, there is no limit to the amount of testing one could do. Even as I write this I can think of more tests I would like to run, more investigation into platform technologies, and more isolation mechanisms to employ.  No doubt, some readers of this article have similar ideas and will report them as feedback and flaws to Adobe. Adobe has taken the first step in its attempt to minimize the impact of attacks on Adobe Reader with sandboxing- and it’s a huge step. Now, the Adobe Reader team can continue to refine its new security model based on feedback from users and security experts alike.  I look forward to seeing your thoughts now that Adobe Reader X is in your hands!

Scott Stender
iSEC Partners

Adobe Reader X is Here!

Since we first announced the development of a sandbox for Adobe Reader on July 20, 2010, there has been a tremendous level of interest in the sandboxing topic — and an equal level of anticipation for Adobe Reader X. Over the last few months, the Adobe Reader engineering team together with the Adobe Secure Software Engineering Team, partners in the software development community such as the Microsoft Office security team and the Chrome team at Google, as well as customers, third-party consultancies in the security community, and other external stakeholders were hard at work to help ensure the sandbox implementation was as robust as possible.

Today, all of the hard work has come to fruition, and we are happy to announce that Adobe Reader X (with Protected Mode, aka sandboxing, on Windows) is now available! To download the new version of Adobe Reader, visit

Adobe’s product security initiatives are focused on reducing both the frequency and the impact of security vulnerabilities. Adobe Reader Protected Mode represents an exciting new advancement in mitigating the impact of attempted attacks. While sandboxing is not a security silver bullet, it provides a strong additional level of defense against attacks. Even if exploitable security vulnerabilities are found by an attacker, Adobe Reader Protected Mode will help prevent the attacker from writing files or installing malware on potential victims’ computers.

For more information on Adobe Reader X and on Adobe Reader X Protected Mode in particular, see the following blog posts:

We are excited to present Adobe Reader X, and — as always — we welcome your feedback and comments!

Inside Adobe Reader Protected Mode – Part 3 – Broker Process, Policies, and Inter-Process Communication

In part three of our technical tour of Adobe Reader X Protected Mode, we’ll examine the broker policies and the inter-process communication (IPC) the sandbox process uses to communicate with it.

The broker process runs with regular user rights and allows the sandboxed processes indirect and secure access to objects based on policies. As the privileged controller, the broker process enables the sandboxed process to get its work done. Some actions are specifically white-listed as part of the broker policies and are allowed to be performed by the sandbox indirectly via the broker. White-listed actions include:

  • Writing to the user’s TEMP folder
  • Writing (saving) Adobe Reader-specific user-modified preferences to the registry
  • Launching a tracker to handle shared reviews
  • Launch a .docx attachment in Microsoft Word

Inter-Process Communication (IPC)

The broker process’ IPC is the sandbox’s gateway to the rest of the operating system. The IPC exposed by the broker is also an attack surface of Protected Mode. Malicious code that runs inside the sandbox process may try to abuse functionality exposed by the broker process or exploit a vulnerability in the broker itself.

Figure 1 - Sandbox and Broker Process IPC

Figure 1 - Sandbox and Broker Process IPC

The Adobe Reader sandbox broker, multi-process architecture requires that processes need to securely communicate with each other. The Adobe Reader IPC uses the standard Windows mechanism of shared memory and events. For example, Figure 1 shows a sandbox process that attempts to write a file to disk. Because sandboxing (“Protected Mode”) is enabled, file creation is routed through the broker process as follows:

  1. The sandbox process tries to create a file.
  2. File creation fails.
  3. The sandbox process sends a request to the broker to perform the create file action on its behalf.
  4. The broker evaluates the sandbox request against its policy-set to decide whether to allow or deny the request. If the request is denied, the broker returns an error.
  5. The broker makes the CreateFile call.
  6. The operating system returns the file handle to the broker.
  7. The broker duplicates the file handle into the sandbox process.
  8. The sandbox process successfully writes the file to disk.

With so many steps involved to perform a privileged action, there is a concern that performance will be affected. It turns out that Reader X with Protected Mode enabled actually performs as well as or better than Reader 9 according to our tests.


The actual restrictions applied to a sandbox process are configured by a policy. The policy is a programmatic interface that allows the broker application programming interfaces (APIs) to define the restrictions and allowances.

The sandbox restrictions are, by design, coarse in that they affect all securable resources the sandbox can touch. But sometimes, a more finely-grained resolution is needed. The policy interface allows the broker to specify white-list rules. White-list rules allow the broker to grant the sandbox access to perform certain operations (e.g. CreateFile) to certain named objects (e.g. the user’s TEMP directory) the sandbox process restrictions would otherwise prevent.

The exact form of the rules for each subsystem varies, but in general, rules are triggered based on a string pattern. For example, a possible file rule could be:

AddRule(SUBSYS_FILES, FILES_ALLOW_READONLY, L”c:\temp\app_log\*.log”)

This rule specifies that access will be granted if a sandbox wants to open a file for read-only access as long as the file matches the pattern expression; for example, c:tempapp_logmyapp.log is a file that satisfies the pattern.

Most broker behavior is controlled via policies. A typical policy would say something like “Allow all writes to the user’s TEMP directory” or “Allow all writes to the registry location HKCUSoftwareAdobeAcrobat Reader10.0.”

In some respects, the broker is a policy manager. It strictly enforces any policies that have been set while sandbox processes simply make requests for resources. If the broker determines that the policy allows such access, the request goes through; otherwise, access is denied.

The policies themselves are tight enough to allow only the desired operations, but lax enough to allow the application to perform its day-to-day work without allowing an exploit to take advantage of it.

In addition to the actions above, the broker is invoked for privileged operations such as running the Adobe Reader Updater and creating out-of-proc Component Object Model (COM) objects.

There are three policy types, and each is created differently:

  • Hard-coded policies: Certain actions essential to the application are encapsulated in hard-coded policies. For example, writing to HKCUSoftwareAdobeAcrobat Reader10.0* or %APPDATA%AdobeAcrobat10.0*.
  • Dynamic policies: Policies are needed to deal with operations resulting from user interaction with a PDF file. Such policies are dynamically created on the fly and are guarded by confirmation dialog. For example, when a user tries to save a PDF file, the broker displays a “Save As” dialog, and the user-specified file name is then added to the list of files to which we allow write access.
  • Admin-configurable policies: An administrator, via a configuration file, can configure white-list rules to allow custom workflows or plugins to bypass some of the sandbox restrictions. For instance, write access could be given to plugin-specific Application Data or registry areas. The details of how to configure this will be outlined in the “Protected Mode and Adobe Reader” section of the Enterprise Administration for the Acrobat Family of Products document once Reader X is released.

Policies are used whenever an access request is made to open the following kernel objects: File, Registry, Named Pipe, Section, Mutant, and Event. It is also used for launching external helper processes such as the Adobe Reader Updater or the comments synchronizer.

Broker API as a Trust Boundary

In the previous blog posts about Adobe Reader Protected Mode, we described all of the restrictions placed on the sandboxed process. These restrictions add an additional layer of defense by containing malicious code inside PDF files within the Adobe Reader sandbox and preventing regular privilege execution on the user’s system. 

Malicious code that is executing in the sandboxed process would have to use a second vulnerability in the broker code to try to gain elevated privileges. While the implementation of a sandbox represents a major additional level of defense by making it significantly more difficult for attackers to succeed, it is important to recognize that sandboxing is not a security silver bullet, in particular as exploit kits that use two or more vulnerabilities and attack methods are becoming more sophisticated.

For the Adobe Reader sandbox, the two logical choices where an attacker might look for an escape hatch are:

  1. Local kernel vulnerabilities
  2. Vulnerabilities in the Adobe Reader broker process

It is beyond the scope of this blog to discuss local kernel vulnerabilities. This section will focus on the attack surface of the Adobe Reader broker process and what we have done to prevent it from being used to escape from the sandbox.

There are a large number of APIs in the Adobe Reader Protected Mode broker to support the rich feature set of Adobe Reader. The vast majority of the APIs are for intercepted Win32 APIs (such as APIs for printing) or access to securable kernel objects (such as sections, events, and mutants). The rest of the APIs fall into two categories:

APIs that provide services which Adobe Reader needs. An example would be launching an executable from a white list of applications.

APIs that pop confirmation dialogs out of the broker process before allowing potentially dangerous things to happen. An example is the dialog that confirms if the user really wants to disable Protected Mode:

Figure 2 - Confirmation dialog to disable Protected Mode

Figure 2 - Confirmation dialog to disable Protected Mode

Obviously, these broker APIs represent a large attack surface. The Adobe Reader broker treats data that comes in via the API as untrusted. To prevent vulnerabilities in the broker process, the Adobe Reader team employed all of the following secure software engineering best practices:

  • Input validation on each parameter
  • Limiting the allowed parameters to a small, white-listed set
  • Code review and code inspection
  • API fuzzing
  • Static code analysis
  • Penetration testing

Much of the above work was done with the help of third party security researchers.

(Note: The final blog post in this series will discuss building security into the broker in more detail.)

Exploitation of Adobe Reader X Versus Adobe Reader 9

In previous blog posts, we described the restrictions placed on the sandbox process. In this blog post, we have described what we have done to prevent an exploit from using the broker process to gain privileges. We’ll end this post with a comparison of what it takes for an attacker using a malicious PDF file to exploit a user running Adobe Reader version 9 on Windows XP versus a user running Adobe Reader X on Windows 7.

First, let’s review what mitigations Adobe Reader 9 provides:

An attacker could easily exploit an Adobe Reader 9 vulnerability using a malicious PDF file and any of a number of techniques to bypass DEP. The following figure illustrates the most common steps in an attack that installs malware via Adobe Reader 9 on Windows XP, where users ran by default as Administrators:

Figure 3 - Typical attack to install malware on Adobe Reader 9 / Windows XP

Figure 3 - Typical attack to install malware on Adobe Reader 9 / Windows XP

Now, let’s take a look at what it would take to exploit Adobe Reader X Protected Mode on Windows 7:

The broker process and the sandbox process both have the following platform mitigations enabled:

These mitigations combined provide a significant hurdle for an attacker using a malicious PDF file to get code running in the sandbox based on a vulnerability in the sandbox. Assuming these hurdles are overcome, the attacker would then need to use the malicious PDF file to exploit a second vulnerability in the Adobe Reader broker process in order to attempt to elevate privileges. The broker process, however, also has all of the above platform mitigations enabled (DEP, ASLR, SEHOP). The attacker would also have to use the malicious PDF file to overcome those hurdles in the broker process to get code running in order to elevate privileges. Even if the attacker gets this far, on Windows Vista and later, he will still not be able take full control of the machine. The broker process does not run with an administrator SID enabled on these platforms, so the attacker would have to exploit another local privilege escalation vulnerability at this point to successfully complete the attack.

Figure 4 - Attack to install malware on Adobe Reader X / Windows 7

Figure 4 - Attack to install malware on Adobe Reader X / Windows 7

With Adobe Reader Protected Mode, the attacker now needs to overcome five different hurdles instead of two in order to succeed in this traditional attack vector for installing persistent malware.

There are many forms of attack, but this illustrates how much more difficult the traditional “front door” attack used against Adobe Reader will be to exploit now with all of the different Protected Mode protections in place.


To summarize, we’ve seen how the very restrictive sandbox process hosts the PDF file (and any exploits contained therein) and uses a set of APIs exposed by the broker to perform some white-listed privileged operations (like file system access). The broker access-checks the API calls using the white-list policies that allow access to a specific set of system objects.

Any bug in the large surface area of the broker API set can potentially be abused by an exploit to escape out of the sandbox. And that’s why we’ve taken every precaution, ranging from extensive input validation to code reviews to fuzzing to pen-testing, in order to keep the broker code secure.

With all of these restrictions in place, we believe it will be significantly harder for exploits to gain user rights code execution since any successful exploitation would most likely now require hijacking not just the PDF-hosting sandbox but also the broker process.

Our next post will talk about the numerous steps we’ve taken to harden the broker, so stay tuned…

-Liz McQuarrie, Ashutosh Mehra, Suchit Mishra, Kyle Randolph, and Ben Rogers