ASSET

A running theme on this blog is that ASSET and Adobe care a great deal about keeping our products secure and our customers safe. On Tuesday Adobe announced a corporate network security issue and since then we've seen media coverage and headlines indicating that vulnerabilities in Adobe Reader may have been the attack vector in this incident.

Just like we always do in the case of reports of security vulnerabilities in an Adobe product, we have been actively tracking down samples or other information regarding potential vulnerabilities in Adobe products related to this incident. The most definitive public description of the incident that we've seen thus far is the McAfee post here.

Similar to the McAfee researchers, we have not been able to obtain any evidence to indicate that Adobe Reader or other Adobe technologies were used as the attack vector in this incident. As far as we are aware there are no publicly known vulnerabilities in the latest versions (9.3 and 8.2) of Adobe Reader and Acrobat that we shipped on January 12, 2010.

This is a complex incident, the investigation is ongoing, and we will continue to work our partners in the security community and the other firms affected. We will continue to use the Adobe PSIRT blog as the first line of communication to our customer base regarding any product security vulnerabilities. Even though we don't have any information regarding a zero day vulnerability in an Adobe product the sophistication of this incident also serves as a reminder to all of us the importance of layers of security to provide the best possible defense against those with malicious intent.

Since the vast majority of successful attacks against all software products are using known, already-patched vulnerabilities we strongly encourage all of our users to update to the latest version of Adobe Reader and Acrobat by visiting get.adobe.com/reader or selecting "Check for updates" from the Help menu.

Kyle Randolph here. I work closely with the Adobe Reader and Acrobat engineering team as we continue to work hard on the security initiative first announced back in May 2009. Today, the team announced new security improvements in Adobe Reader and Acrobat 9.3 and 8.2. This is the third quarterly security update for Adobe Reader and Acrobat and we are starting to roll out to users the configuration options and features that we began designing last summer to mitigate the evolving security threats we were seeing. Let me explain the security geek coolness factor of the improvements in this release as well as the improvements in the October quarterly security update.

New Adobe Reader Updater / Acrobat Updater
We introduced the new updater in the October Adobe Reader and Acrobat 9.2 and 8.1.7 update as beta technology, and today, we are testing the new technology with a real-world security update to users participating in the beta program. (Since we are still conducting the pilot, only users who are participating in the beta program are receiving today's update via the new updater.) The new updater improves the user experience and helps users stay up to date with the new option of receiving security updates automatically, via background updates, which have been shown to have better patch adoption. Some customers, such as corporate IT administrators, need to know and manage which updates are installed and when. But a lot of customers, particularly consumers and individuals who don't have the autopilot luxury of a managed desktop environment, just want to have the most secure and up-to-date version, and don't want to be interrupted when it is time to install an update. By allowing customers to select an update process that automatically runs in the background, we can help protect more users from attacks against known, patched vulnerabilities.

JavaScript Blacklist Framework
Over the past two years, a significant number of external vulnerabilities found in Adobe Reader and Acrobat have been in JavaScript. The Adobe Reader and Acrobat engineering team has been busy creating new ways to help protect against this attack vector. The new Adobe Reader and Acrobat JavaScript Blacklist Framework, which was added with the October update, is great for security because it provides a method to disable a specific vulnerable API instead of disabling JavaScript completely. This allows Adobe Reader to be configured in a way that is not vulnerable if a 0-day vulnerability that exploits a JavaScript API is identified. Better still, the new blacklist is stored in the registry and can be configured centrally in enterprise environments using Group Policy Objects (GPO) to prevent end users from overriding it. As an example, the recent vulnerability CVE-2009-4324 could be mitigated by blocking the DocMedia.newPlayer API.

For more info on the JavaScript Blacklist Framework, check out http://kb2.adobe.com/cps/504/cpsid_50431.html.

Yellow Message Bar
The Yellow Message Bar was added in the October security update for Adobe Reader and Acrobat (9.2 and 8.1.7), but it is cool enough to mention here. This makes the user experience much more pleasant when a dangerous API is selectively blocked by the JavaScript Blacklist Framework or due to the Enhanced Security configuration. Previously, you'd get a modal dialog box asking users if they would like to re-enable some unsafe behavior, as shown in the screen shot below:

Now the Yellow Message Bar appears at the top of the document as shown below:

Since the Yellow Message Bar stays out of the way, it enables users to interact with the PDF without exposure to a disabled feature's security risk, if you don't need that functionality. Additionally, the choices are more granular in that the Yellow Message Bar decision is to trust a document one time or always, as opposed to a decision to turn the entire feature back on for all documents. These changes should reduce the frequency and impact of accidental click-throughs or users getting into the habit of clicking through warnings without reading them, which can lead to social engineering and phishing attacks. This same type of change in security notification has been adopted in other vendors' desktop products, such as Microsoft Office, as a security best practice. The Yellow Message Bar will appear when an action is blocked by Enhanced Security in Adobe Reader or Acrobat or by the JavaScript Blacklist Framework.

For more info on the Yellow Message Bar, see http://kb2.adobe.com/cps/504/cpsid_50432.html.

Multimedia (Legacy) off by Default
Another effective technique to reduce security risk for our customers is to reduce the attack surface of the product. Legacy multimedia is a set of rarely used features which have a broad attack surface. The Multimedia (Legacy) features are no longer trusted by default. Users that open PDFs that contain legacy multimedia will see a Yellow Message Bar at the top of the document.

Conclusion
This January update for Adobe Reader and Acrobat builds on the good work put into the October release to continue increasing the security protection for our customers with each quarterly security release in addition to fixing externally reported vulnerabilities. We're excited to evaluate the results for the pilot of the new Adobe Reader Updater with its automatic mode for background updates. The Yellow Message Bar notifications provide an improved user interface to help protect users. And we're providing more fine-grained control for any future JavaScript API vulnerabilities with the JavaScript Blacklist Framework. Finally, disabling Legacy Multimedia by default protects users against any potential security vulnerabilities identified in these rarely used features.

We posted an update to Security Advisory APSA09-07 that reflects the target ship date of January 12, 2010 for the update to remediate vulnerability CVE-2009-4324. I thought folks might be interested in some of the analysis that went into developing the schedule for the fix, so let me share some of the details in this post.

We evaluated two different options for patching this vulnerability:

1. Stop everything else and start work immediately on an out-of-cycle security update to resolve this vulnerability with a one-off fix. We made major investments as part of our security initiative earlier this year that allow us to deliver patches more quickly. We estimated that delivering an out-of-cycle update would require somewhere between two and three weeks. Unfortunately, this option would also negatively impact the timing of the next quarterly security update for Adobe Reader and Acrobat scheduled for January 12, 2010.
2. Roll the fix for vulnerability CVE-2009-4324 into the code branch for the scheduled January 12, 2010 release. The team determined that by putting additional resources over the holidays towards the engineering and testing work required to ship a high confidence fix for this issue with low risk of introducing any new problems, they could deliver the fix as part of the quarterly update on January 12, 2010.
   

Two important considerations that contributed to our decision to select the second option:

• JavaScript Blacklist mitigation - This new feature, introduced in Adobe Reader and Acrobat versions 9.2 and 8.1.7, with the quarterly update in October, allows individuals as well as administrators of large enterprise managed desktop environments to easily disable access to individual JavaScript APIs. More details on the JavaScript Blacklist mitigation are available here. The feature design and our testing for this specific vulnerability indicate the JavaScript Blacklist is an effective mitigation against the threat without breaking other workflows that rely on JavaScript or other JavaScript APIs.


• Customer schedules - The next quarterly security update for Adobe Reader and Acrobat, scheduled for release on January 12, 2010, will address a number of security vulnerabilities that were responsibly disclosed to Adobe. We are eager to get fixes for these issues out to our users on schedule. Many organizations are in the process of preparing for the January 12, 2010 update. The delay an out-of-cycle security update would force on the regularly scheduled quarterly release represents a significant negative. Additionally, an informal poll we conducted indicated that most of the organizations we talked with were in favor of the second option to better align with their schedules.
  

Kyle Randolph here. Today we have a guest post by Pat Wibbeler, who led a fuzzing tiger team as part of the Adobe Reader and Acrobat Security Initiative.

As a Software Engineering Manager on Adobe Reader, I take it personally every time an exploit is found in our code. I've always taken pride in my work, and I'm particularly proud to work with many brilliant engineers on a product installed on hundreds of millions of desktops.

This ubiquity makes Adobe Reader an appealing target for attack. We're going to great lengths to protect our users through the Adobe Reader and Acrobat Security Initiative. One major component of this ongoing initiative is application fuzzing. I led a task force in fuzzing Reader when the security initiative began. We have always had fuzzing efforts within the Reader team, but this project scaled our efforts in key areas. I thought I'd share some of our experiences.

First - What is "Fuzzing?" One of the most common application vulnerabilities is caused by improper input validation. When developers fail to validate input data completely, an attacker may be able to craft application input that allows the attacker to gain control of the application or system. PDF files are the most common type of input consumed by Adobe Reader. Developers and testers intuitively test PDF file format parsing with valid cases - can Reader properly render a PDF generated by Adobe Acrobat or another PDF generator? Fuzzing automatically provides invalid and unexpected PDF data to an application, probing for cases where the PDF format may be poorly validated. For more information on fuzzing, you can read the following Wikipedia entry.

There are several existing fuzzers, and all fuzzers work basically like this:

Valid data is fed into a fuzzer. The fuzzer mutates the data, creating invalid data, and feeds it to the program, monitoring for a crash or fault. In our case, the fuzzer mutates a PDF file and launches Adobe Reader, monitoring and logging any crashes or faults. Crashes are then analyzed and prioritized according to whether or not they are likely to be exploitable.

When we began, we set the following general plan: First, select a fuzzer, then select areas to target. Next, put the fuzzer into action by developing tools, and then building and running models in a newly developed infrastructure.

Fuzzers have several important features, but the most important for us are:

• Data Mutation Strategies - How does the fuzzer manipulate input data?
• Extension Mechanisms - Can the user extend the fuzzer to provide richer mutations or data definition?
• Monitoring/Logging - How does the fuzzer monitor and sort faults?

• A rich set of data mutators
• Several mechanisms for extension and modification (including extending the engine itself since it's open source)
• Excellent Monitoring/Logging, using the !exploitable (bang exploitable) tool from Microsoft.

Select Targets
It's critical to identify and prioritize specific targets during fuzzing. It's extremely tempting to say there is only one target - "PDF." If you look at the PDF specification, it quickly becomes clear how rich, complex and extensive the format is. It contains compressed streams, rich cross-referential structures, multimedia data, flash content, annotations, forms, and more. Fuzzing PDF is as complex as the file format itself.

We set the following goal:
Think like an attacker - target the areas of PDF that an attacker is likely to target

Since the Peach Fuzzer is a flexible, robust tool, we're able to systematically focus on different areas of the PDF file format, covering the most likely targets first.

Build Models
Peach performs fuzzing mutations based on a model built by the developer. A model is an XML description of the data being attacked. Peach calls these "Pit" files. These models can describe the data in detail or simply treat the data as a blob. Peach mutates blob data by flipping bits and sliding specific data values through the blob one byte at a time. We first built simple models that treat streams as blobs. We referred to these as "dumb" or "naïve" models. These simple models were effective, and for me as a Reader developer, humbling.

We followed each simple model with a "smart" model. The Peach Pit format can describe relationships and offsets between parts of the fuzzed data. For instance, a 4-byte field may describe the length or offset of a stream that follows later. Peach will intentionally provide values for this 4-byte field that are at or near common range limits and truncate or lengthen the stream data in an attempt to overrun buffer lengths that may be naively set using the data input from the stream. These models are also quite effective.

Peach Pit models can describe data in two ways:

• Generational Models - Generational models produce data only from the description of the data. For instance, the description might indicate a fixed-width bit-field of flags followed by a stream with properties indicated by those flags. A generational fuzzer might simply create a random bit-field and a series of random bytes.


• Mutational Models - Mutational models consume template data based on the description of that data. A varied data set can be fed to these fuzzers, and the data type will be mutated. Mutational fuzzers have the distinct advantage that you can create a completely new fuzzing run simply by feeding the mutational model new input data. For instance, the same JPEG mutational model might consume any existing jpg, mutating it differently based on properties unique to that jpg (e.g. color depth, or whether or not the image is interlaced).

We use both generational and mutational models in our testing, broadening our coverage as much as possible.

Develop Tools
We found it extremely helpful to build tooling to assist in model development and results processing. We built several new tools:

• Crash Analyzer/Processor - Peach "buckets" (categorizes) exploits based on the signature of the callstack. It's hard to overstate the value of this bucketing since fuzzing produces many duplicate bugs. We developed a script that iterates through the bucket folders, creating an XML description of all of the results. We built a GUI Crash Analyzer (using Adobe AIR) that allows for visual sorting and analysis of the Peach buckets.


• PIT Generator - We also developed a GUI tool that we could use to quickly select streams within an existing PDF and create a generational "naïve" model based on that data. This allowed us to quickly build simple models that could run while we spent time building richer smart models.


• Enveloper - PDF is a format that contains many other formats. We found it somewhat difficult to build mutational models that ignored large portions of an input PDF. We also wanted to be able to find sample input data without having to manually wrap it in a PDF file before fuzzing. We built an "enveloping" extension to Peach that allowed us to build pit files that described only our targeted stream (e.g. JBIG2). The enveloper then wrapped the stream in a valid PDF structure before Peach launched Adobe Reader.

Develop Infrastructure
Unfortunately, models take an extremely long time to run because they have to launch and close an application for each file we generate. Many models can generate from tens to hundreds of thousands of mutations. To speed the run time we leveraged the Acrobat/Reader test automation team's fabulous grid infrastructure to provision virtual machines, deploy Reader and launch a test. Additionally, Peach has a parallelization feature that allows different parts of a model to be run in different processes. Our automation team worked with the model development team to enhance their automation grid. The grid automatically partitions a Peach run and deploys it across dozens of virtual machines. At peak, we could run hundreds of thousands of iterations per day for a given model, drastically improving throughput.

Run the Models!
It seems obvious, but the most important part of the process is to actually run the models. Developers love to build things, and it's easy to tweak forever, but those developers who ran the models early and often found the most bugs. We also found that it was easy to make a small mistake in modeling that invalidated the entire model. For example, there are several areas of PDF that Adobe Reader will reject early if they are malformed. In this case, the targeted code might never be executed! It's critical to debug and troubleshoot models carefully so that precious iteration time is not wasted.

Document Process/Results
This almost goes without saying, but it's important to keep close tabs on your process and your results. As we brought more people into the process, we found our project wiki to be invaluable. It's now being used by other groups within Adobe as a learning resource to refine their own fuzzing efforts.

Use Results to Improve the SPLC
Each unique crasher or new vulnerability is an opportunity to drive improvements and optimization into the rest of the Secure Product Lifecycle (SPLC). The code level flaw might highlight a gap in security training, provide an opportunity for some high ROI target code reviews in the surrounding code, or suggest a new candidate for the banned API list. Fuzzing is best done not in a vacuum but as an integrated part of a broader security process.

Parting Advice
My last piece of advice:

Approach fuzzing your own product with humility.

Humility allows you to start simply. It also allows you to question a model that isn't producing results. We often found that when a model wasn't producing, it was because of a mistake in the model or because the tool was misconfigured rather than because the code was good. Humility also allows you to choose targets in the unbiased way an attacker will choose them. Finally, humility allows you to recognize that security vulnerabilities can sneak in under even the most watchful eye. Fuzzing is one way we help catch bugs that have survived all the other security processes. Application security should be an ongoing effort on any product, just as it is with Adobe Reader.

Peleus here. Research was recently published that makes several claims regarding Flash content on sites that allow end-user uploads that I would like to address.  The topic raised is not news; it's something that has been understood and discussed by the security community for years. Most importantly, this is not a vulnerability in Adobe Flash Player.  Web servers that choose to accept user-uploaded content also choose to accept the risks that go along with that functionality.  Flash Player's behavior is consistent with other web technologies and the web browser security model. Several web technologies pose the same risk to servers that allow end-user uploads.

To really understand the issue, we need to take a step back and understand the same-origin policy. This policy basically states that two pieces of content hosted on the same domain and loaded by the same protocol trust each other.  Conversely, two pieces of content hosted on different domains do not trust each other and cannot interact. The same-origin policy is enforced by the web browser, and all web content must follow it.
 
The researchers noticed that there are many web applications that allow end-users to upload their own content to the same domain as the web application itself. According to the same-origin policy, this means that the web application trusts the end-user's content in the same way that it trusts its own content. In most real world scenarios, this is not true. The web application does not trust end-user content and does not mean to grant end-user content any privileges on its domain. Quoting Law #4 of Microsoft's 10 Immutable Laws of Security, "If you allow a bad guy to upload programs to your web site, it's not your web site anymore." Microsoft originally published these laws in 2000 and TechNet Magazine revisited them in 2008 where they concluded, "Law 4, at least in spirit, still holds--in spite of some sites that are designed to permit people, bad and good, to upload programs to them."
 
When Flash content (SWF), as well as any other web content, is hosted on the same domain as the surrounding HTML, the same-origin policy defines a trust relationship between the two. Proposals have been made to have Flash content assume there is never any trust and therefore ask permission for each and every request that it makes. This would break all legitimate deployments of Flash content on the web and require all administrators to change their configurations in order to start supporting Flash again. It also would not solve the problem. Flash content is not the only content-type that can execute JavaScript when clicked on by a user. Law #4 still holds. The fundamental problem is that the site is not properly handling the upload of arbitrary active content. Attackers can still attempt to upload other forms of server-side or client-side code in an effort to exploit the server.
 
The web has already widely deployed much better solutions. Sites solve the problem by hosting untrusted content on a different domain. If a site developer wants to host arbitrary uploaded content on a site and does not want SWF content to execute on their site, the SWF can be served with headers, which instruct Flash Player to treat the file as an attachment. Flash Player will acknowledge that request and present an Open/Save/Cancel dialogue to the user rather than render the content. Developers may also choose to limit uploads to only the types of content that they plan to support.  These steps to properly managing user-generated active content uploads may be challenging, but good web security requires vigilance against this type of risk alongside all the other familiar web threats like CSS, CSRF, ...
 
We understand that developers have many challenges when managing rich content, and we are working with the industry in discussing and supporting solutions (such as Content-Security Policies) to minimize the risks and educate the community. Our goal is to enable all of our customers (end-users, developers, and web site owners) with solutions that will protect them from the critical risks they face.