Author Archive: Peleus Uhley

Reflections on Black Hat & DefCon

This year the ASSET security team along with security engineers from several other Adobe teams travelled to Vegas to attend the summer’s largest security conferences – Black Hat and DefCon. The technical talks can typically range from “cool bugs” to “conceptual issues that require long term solutions.” While the bugs are fun, here’s my take on the major underlying themes this year.

One major theme is that our core cryptographic solutions such as RSA and TLS are beginning to show their age. There was more than one talk about attacking TLS and another presentation by iSEC Partners focused on advances related to breaking RSA. The iSEC team made a valid case that we, as an industry, are not prepared for easily deploying alternative cryptographic solutions. Our industry needs to apply the principles of “crypto agility” so that we can deploy alternative solutions in our core security protocols, should the need arise.

Another theme this year was the security issues with embedded systems. Embedded systems development used to be limited to small bits of assembly code on isolated chips. However, advances in disk storage, antenna size, and processors has resulted in more sophisticated applications powering more complex devices. This exposed a larger attack surface to security researchers at Black Hat and DefCon who then found vulnerabilities in medical devicesSIM cardsautomobilesHVAC systemsIP phonesdoor locksiOS chargersSmart TVsnetwork surveillance cameras, and similar dedicated devices. As manufacturing adopts more advanced hardware and software for devices, our industry will need to continue to expand our security education and outreach to these other industries.

In traditional software, OS enforced sandboxes and compiler flags have been making it more difficult to exploit software. However, Kevin Snow and Lucas Davi showed that making additional improvements to address space layout randomization (ASLR), known as “fine-grained ASLR,” will not provide any significant additional levels of security. Therefore, we must rely on kernel enforced security controls and, by logical extension, the kernel itself. Mateusz Jurczyk and Gynvael Coldwind dedicated significant research effort into developing tools to find kernel vulnerabilities in various operating system kernels. In addition, Ling Chuan Lee and Chan Lee Yee went after font vulnerabilities in the Windows kernel. Meanwhile, Microsoft offered to judge live mitigation bypasses of their kernel at their booth. With only a small number of application security presentations, research focus appears to be shifting back toward the kernel this year.

Ethics and the law had an increased focus this year. In addition to the keynote by General Alexander, there were four legal talks at Black Hat and DefCon from the ACLU, EFF and Alex Stamos. Paraphrasing Stamos’ presentation, “The debate over full disclosure or responsible disclosure now seems quaint.” There were no easy answers provided; just more complex questions.

Regardless of the specific reason that drew you to Vegas this year, the only true constant in our field is that we must continue learning. It is much harder these days to be an effective security generalist. The technology, research and ethics of what we do continues to evolve and forces deeper specialization and understanding. The bar required to wander into a random Black Hat talk and understand the presentation continues to rise. Fortunately, walking into a bar at Black Hat and offering a fellow researcher a drink is still a successful alternative method of learning.

Peleus Uhley
Platform Security Strategist

The Evolution of Exploit Sophistication

When we look at the exploits that Adobe patched from February and March of this year, it is clear that today’s zero-day exploits are increasingly more sophisticated. This increase in sophistication is not limited to the skills needed to find and exploit the vulnerability. The code used to exploit the environment is also more robust in terms of code quality and testing. In short, exploit creation today requires the same level of rigor as professional software engineering projects.

Today’s advanced exploits need to be written to work in any target environment. For instance, February’s Reader 0-day supported 10 different versions of Reader with 2 sub-versions dependent on the end-user’s language. In addition, Flash Player CVE-2013-0634 had shell code for Windows XP, Vista, Windows 7, Server 2003, Server 2003 R2, Server 2008 and Server 2008 R2 as well as supporting six versions of Flash Player. Variants of CVE-2013-0634 also supported Firefox and Safari on Mac OS X. An exploit developer would need a robust testing environment to ensure that the exploit would work in that many different environments for each version of Flash Player. The exploit writers even took into account different CPU architectures by including a signed 32-bit payload and a 64-bit payload. This reflects the fact that these exploits are written with professional code quality and stability requirements for distribution across a dynamic target base.

As vendors are increasing software defenses through techniques such as sandboxing, attackers are now combining multiple vulnerabilities from different vendors to achieve their goals.When I look at the reports from Pwn2Own and some of the recent zero-day reports such as CVE-2013-0643, attacks are moving toward combining vulnerabilities from multiple products, some of which are from different vendors. We are moving away from the model of single vulnerability exploits.

This is all a part of the natural evolution of the threat landscape and the commercialization of exploits. This will require an equal evolution on the part of vendors in their software defences. Karthik Raman and I will be discussing this topic, “Security Response in the Age of Mass Customized Attacks,” in more detail at the upcoming Hack in the Box Conference (HITB) Amsterdam next week. Please stop by our talk if you would like to discuss this further.

Peleus Uhley
Platform Security Strategist

Top 10 Web Hacking Techniques of 2012

I was honored that Jeremiah Grossman asked me to serve again on the panel for the Top 10 Web Hacking Techniques of 2012. During the process, I get to take a look at the year’s most interesting research in greater detail than what is normally allowed in my day-to-day schedule. It makes me question what I thought I knew about existing attack techniques and how this year’s top research reflects on the industry.

Ranking the entries is challenging when you are comparing an SSL cryptography bug vs. the server side request forgery attacks vs. attacks against Chrome add-ons. To do so, requires that you read the full research, review the source code that was written and try to determine what impact the research had on the community. I typically start by considering factors such as:

  • Is the research completely new or is it just an incremental improvement of a known issue?
  • How many end-users are impacted?
  • Is this an individual bug affecting one vendor or an industry-wide issue?
  • What does successful exploitation yield (XSS, file access, OS control, etc.)?
  • Is the exploit practical to conduct?
  • What was the depth of research (a white paper, a tool, etc.)?
  • Extra points for style or creativity?

Reviewing the research to this depth then leads to uncovering smaller bits that are interesting in their own right. For instance, the Blended Threats and JavaScript research leveraged HTML5 to create cross-site file upload capabilities. You will find links to tools that back up the research in entries such as Blended Threats, Attacking OData, Bruteforcing PHPSESSID, and the Chrome add-on research. In some cases, the research reminds you about how many of the old tricks can be re-purposed against new technologies.

As an example, in July, 1998 Microsoft issued security bulletin MS98-004 which allowed, “Unauthorized ODBC Data Access with RDS and IIS”. In 1999, new information was published regarding the bug and Bugtraq ID 529 said exploiting the vulnerability could result in, “obtaining access to non-public servers or effectively masking the source of an attack on another network”. Today, we would classify this type of attack as, “Server Side Request Forgery” (SSRF). Modern SSRF attacks can include both old and new attack techniques. For instance, the ERPScan paper discussed how to use the gopher: protocol in an XML External Entity to attack SAP gateways. Over a decade later, we now have a formal taxonomy for classifying these types of bugs. However, the core issue of effectively sandboxing a web application to a finite set of resources is not much easier than it was 15 years ago. If anything, it has become more difficult as the number of supported communication channels between endpoints has increased.

Finally, you begin to think about why these top 10 bugs are critical to the industry. The panel ranked,”Compression Ratio Info-leak Made Easy” (CRIME) in the top slot due to the overall difficulty of the research, the vendor action required to address it, and the importance of SSL to the security ecosystem. As we move more data to the cloud where we will access it via our mobile or tablet devices, we will increasingly rely on SSL to protect it in transmission. Another interesting trend in this year’s top list was the multiple SSRF entries. As I mentioned, it wasn’t a completely new concept but rather a concept that is seeing a renewed interest as more data moves to the cloud. There were three SSRF entries in the Top 15 and the SSRF entry that was ranked number 2 includes research by two separate groups. As our dependence on cloud storage grows, SSRF will become an increasingly used attack vector for reaching all the critical data that lies behind those front end cloud servers.

As everyone reviews the Top 15, I encourage you to find time to read through each entry. The final list represents great information by talented researchers. Also, I would recommend sharing the list with your engineers. When I shared the Top 15 list with our internal Adobe community a few months ago, I can attest that at least one developer recognized a flaw that existed in his system and corrected the issue. Thanks to Jeremiah Grossman for putting this together each year and allowing me to be a part of it.

Peleus Uhley
Platform Security Strategist

Firefox Click-to-Play Helps Protect Our Customers

The Adobe team has worked hard to improve patch adoption by delivering background updaters for Flash Player and Adobe Reader. In addition, we have worked with partners, such as Microsoft and Google, to reduce update fatigue by delivering patches through existing update mechanisms. However, one of the hardest challenges in protecting end users is reaching what is sometimes referred to as the “long tail” in an update graph. These are the users who, for various reasons, have not updated their systems in several months or even years. Reaching these last few end users can be difficult if they have disabled their update mechanisms. Unfortunately, they are also the users who are most likely to be successfully attacked.

Yesterday, Mozilla announced an update to the Firefox click-to-play feature that will warn users when they try to play plugin content with an out-of-date browser plugin. Since Mozilla will now be assisting plugin vendors in reminding these users to update, we will hopefully be able to convert more of them to patched versions. At the same time, Mozilla is helping to protect these users from those who would use older vulnerabilities to infect their systems. We support Mozilla in their efforts to protect and inform our mutual customers.

ColdFusion 10 Provides Powerful New Security Tools

Today marks the release of ColdFusion 10. This release redefines many aspects of the ColdFusion 10 security model and incorporates the principles of the Adobe Secure Product Lifecycle (SPLC). With this release, we’ve worked to improve three major areas: Our goals were to improve patch adoption, improve the default configuration, and to make it easier for developers to create secure ColdFusion applications.

One of the most common reasons for a successful attack against a ColdFusion server is that it doesn’t have the latest security updates. In all fairness, this is not completely the administrator’s fault. Updating a ColdFusion server can be difficult due to the number of manual steps involved. Also, it is easy to miss a security update announcement. With ColdFusion 10, we make both of these steps easier. The ColdFusion 10 administration interface now incorporates a simple “Check For Updates” button. Alternatively, the server can be configured to automatically check for updates and send an email to the administrator once one becomes available.  Finally, the interface allows the developer to apply the patch through a single button click in the administrator interface. These features help make updating the server much more straightforward.

The second major area of improvement focused on making it easier for administrators to securely deploy ColdFusion 10. One of the most attractive characteristics of ColdFusion is that it has always been a simple development environment. Therefore, there were several features that favored making the early phases of development easier by leaving the complicated aspects disabled by default. The cost of this choice was that once developers were ready to deploy to production, they had to review a 35-page lockdown guide to enable and/or configure those more complicated features appropriately. With today’s release, we offer the option of starting the server in a secure-by-default configuration. This greatly simplifies the process of making a server production-ready with a secure configuration.

The last area of improvement focused on providing developers with an increased number of tools for creating secure ColdFusion applications. One example is that we have provided integrated OWASP ESAPI support in the platform. We originally started to include ESAPI in ColdFusion 9 just for our internal needs of addressing cross-site scripting (XSS). Once developers noticed the library in the update, they quickly developed several blogs on how to unofficially start using it in your ColdFusion code. Today’s release formally exposes several aspects of ESAPI through ColdFusion API’s to help developers avoid cross-site scripting vulnerabilities.

We also improved the session management capabilities in ColdFusion–another aspect of making it easier for developers to create ColdFusion applications. We have improved APIs to make it easier to set the HttpOnly and Secure flags on cookies. Session rotation has been improved through new SessionRotate and SessionInvalidate APIs.  To combat cross-site request forgery (CSRF) with active sessions, the ColdFusion team added an API for generating unique tokens for form requests. The team also added support for protecting against clickjacking attacks on active users by adding support for the X-FRAME-OPTIONS header.

ColdFusion 10 is a significant advancement in helping ColdFusion customers improve their secure product lifecycle processes. It is even easier to create secure content, deploy the content on a secure server and manage the server updates once it is deployed. This is only an introduction to the major security enhancements in ColdFusion 10. For more information on all the new security APIs for developers, please see the ColdFusion documentation on Security Enhancements in ColdFusion 10.  ColdFusion administrators should review the Administering Security and the Server Update sections for a complete list of server improvements.

A Basic Distributed Fuzzing Framework for FOE

Last week, CERT released a Python-based file format fuzzer for Windows called Failure Observation Engine (FOE). It is a Windows port of their Linux-based fuzzer, Basic Fuzzing Framework(BFF). CERT provided Adobe with an advanced copy of FOE for internal testing, and we have found it to be very useful. One of the key features of FOE is its simplicity. The configuration file is very straightforward, which makes it easy to introduce to new teams. We have also used the “copy” mode of FOE to help automate triaging large sets of external reports. It is a great tool to have for dumb fuzzing. For this blog, I am going to discuss a simple Python wrapper I created during my initial testing of the tool which helped to coordinate running FOE across multiple machines. This approach allows you to pull seed files from a centralized location. You can also view the status of all of the fuzzing runs and their results from the same location. If you are not interested in writing a distributed fuzzing framework, then you might want to stop reading because the rest of this blog is all about code. :-)

The goal of this distributed fuzzing framework design was to create something simple, lightweight since I was experimenting with a new tool. I set a personal limit of keeping the project to around 1,000 lines of code in order to scope my time investment. That said, I also wanted to build something that I could easily scale later in the event that I liked it enough to invest more time. For the client-side code, I used Python since that was already required for FOE. On the server side, I had a Linux/Apache/MySQL/Perl (LAMP) server. Knowing that everyone has their own preference for server-side authoring, I am only going to describe the server-side architecture rather than providing the Perl source. Nothing in the server-side code is so complicated that a Web developer couldn’t figure out how to do an implementation in the language of their choice from this description. While I designed this for testing the FOE fuzzer, only one file in the entire system is FOE-specific, which makes the infrastructure reusable for other fuzzers. The current name of the main script is “dffiac.py” because I thought of this project as a, “Distributed Fuzzing Framework in a Can”.

For this design, all of the tracking logic is consolidated on the centralized server. The Python script will issue requests for data using simple GETs and POSTs over HTTP. The server will respond to the requests with basic XML. The fuzzing seed files are hosted on the server in a public web server directory from which they can be downloaded. Identified crashes will be uploaded to the server and placed in a public web server directory. Both the client-side and server-side codes are agnostic with regards to the format of the seed files and the targeted application. Therefore, this should be relatively easy to set up in any infrastructure.

 

The database design

In this design, the mySQL server coordinates the runs across all the different machines. You first need a table containing all the files that you want to fuzz. At a bare minimum, it needs a unique primary key (fid), the name of the file and its location on the web server. I currently have a database of more than 60,000 SWF files that are sub-categorized based on type so that I can focus fuzzing to specific types of SWF files. However, name and location will get you started with fuzzing.

 

seed_files

Field Type Description
fid Integer (primary key, autoincrement) The unique File ID for this entry
name VARCHAR The filename
location VARCHAR The relative web directory for the file (e.g. “/fuzzing/files/”)

 

The next thing that you will need is a table to track all of the fuzzing runs. A “run” is defined as one or more servers testing with the same FOE configuration file against a defined set of seed files. There are multiple ways in which you can define the selected seed files for the run. For instance, you may want to use FOE against multiple types of applications. For this scenario, you might have a different seed_files for each file type. To support the need for different seed_files tables, the design of run_records requires that you provide the “table_name” that will be used for this run. Once a seed_files table is selected, it may be necessary to further restrict the run to a subset of files within the seed_files tables. For instance, you may only want to select a subset of files within the given table. Therefore, the design requires that you provide a “type” parameter which denotes the method for selecting files from the seed_files table. The value of type can include values such as “all”, “range” or any other sub-category you want to define. As an example, this particular run may be a “range” type that starts at start_fid and stops at end_fid.

 

run_records

Field Type Description
rid Integer (primary key, autoincrement) The unique ID for this run
name VARCHAR The human readable name for the run
description VARCHAR A description for the run (e.g. config or mutation used, # of iterations, etc.)
type VARCHAR Values can include (all, range, etc)
table_name VARCHAR The name of the seed_files table that will be used for testing
start_fid Integer The first fid from seed_files to be fuzzed in this run
end_fid Integer The last fid from seed_files to be fuzzed in this run
current_fid Integer This tracks the next fid to be tested during the run

 

For every run, you will have multiple servers running FOE. For each server instance, it will be necessary to track the server name, when it started, the current status of the server, and when it last provided an update. The status will include values such as “running” and “complete.”  You can infer whether a machine has died based on whether it has been too long since the timestamp for the last_update field was modified.

 

server_instances

Field Type Description
siid Integer (primary key, autoincrement) The unique server instance ID
server_name VARCHAR The name of the server (e.g hostname + IP address)
status VARCHAR Is it running or has it completed.
start_time timestamp When did this instance start?
last_update timestamp When was the last request from this instance?
rid Integer What run_record is this instance associated with?

 

Lastly, you will need a table to record the results. The script will record the server_instance ID (siid) where the crash was found in case there are issues with reproducing the crash. This will allow a QA to retest on the original machine where the crash occured. It is also necessary to track which run was able to identify the crash. The rid is not recorded because it can already be extrapolated from the siid. According to database normalization rules, redundant information should not be stored in tables. In this design, the script will record a result in fuzz_records regardless of whether a crash was identified.  This allows you to track which files have been tested against which FOE configurations. If a crash is identified, the web server directory where the crash result was stored is also recorded.

 

fuzz_records

Field Type Description
frid Integer (primary key, autoincrement) The unique fuzz record ID
fid Integer The seed_files ID for this entry
siid Integer The server instance ID for this entry
crash Boolean Whether a crash was recorded during this test
location VARCHAR Where the crash result was stored (e.g. /results/run_id/)

 

The config file

You will start the Python script by providing a simple configuration file in the command line: “python dffiac.py dffiac.cfg”. The configuration file is in the same format as the FOE configuration file and contains the following:

 

dffiac.cfg

[foeoptions]
python_location=C:Python26python.exe
config_location=C:FOEconfigsmy_foe_config.cfg

 

[runoptions]
run_id=1
web_server=http://my.internal.server.com
upload_cgi=/fuzzers/crash_uploader.cgi
action_cgi=/fuzzers/action_handler.cgi

 

[logoptions]
log_dir=C:dffiaclogs

 

The foeoptions section tells the script where to find the Python executable and the location of the FOE config script you will use for this run. The runoptions section provides the run id (rid) the database is using to track this run along with the location of the web server, the path to the action_handler.cgi and the path to the CGI that will handle the file uploads. The logoptions allows you to specify where the script will log local information regarding the run. The logs directory needs to exist prior to starting the script. The config_location and run_id are likely the only two elements that will change from run to run.

 

The transaction flow

For this next section, we will review the transactions between the dffiac.py script and the web server. The web server will read in the GET parameters, execute the relevant SQL query and return the results as XML. All but one request is handled by the action_handler defined in the dffiac.cfg config file. The upload of the crash results is handled by the upload_cgi defined in the dffiac.cfg config file.

Once dffiac.py has started and been initialized by the config file, the script will begin sending requests to the server. An “action” parameter informs the action_handler CGI which query to perform. The server will always respond to the Python script with the relevant information for the request in a simple XML format.

 

The first HTTP request from the Python code will be to gather all the information regarding the run_id provided in the config file:

GET /fuzzers/action_handler.cgi?action=getRunInfo&rid=1

 

The web server will then perform this SQL query with the rid that was provided:

select run_type,start_fid,end_fid from run_records where rid = ?

 

The results from the query will be used to return the following XML (assuming the run is defined as the range of fids from 1-25):

<xml>
  <run_type>range</run_type>
  <start_fid>1</start_fid>
  <end_fid>25</end_fid>
</xml>

 

Now that dffiac.py has the information for the run, it will then inform the web server that the run is starting:

GET /fuzzers/action_handler.cgi?action=recordServerStart&rid=1&serverName=server1

 

This HTTP request will result in the following SQL query:

insert into server_instances (server_name,status,start_time,rid) values (?,'running',NOW(),?)

 

The insert_id from this query (siid) becomes the unique identifier for this instance and is returned for use in later queries:

<xml>
  <siid>1</siid>
</xml>

 

Now that this instance has officially registered to contribute to this run, the Python script will begin requesting individual files to test:

GET /fuzzers/action_handler.cgi?action=getNextFid&rid=1&run_type=range

 

The corresponding SQL query will vary depending on how you have defined your run. For this example, we will assume that this is a basic run that will incrementally walk through the file IDs in the seed_files table. To accomplish this, we create an SQL variable called “value” and assign it the current_fid. By recording the value of the current fid and incrementing the “value” in a single statement, we can avoid a race condition when multiple servers are running.

update run_records set current_fid = current_fid + 1 where rid = ? and @value := current_fid;

 

At this point, “@value” is set to 1 which is the fid the Python script will test and the current_fid in the database table has been incremented to 2. The web server can then fetch “@value with the following SQL command:

select @value;

 

Since the process of asking for the next fid will automatically increment the value of current_fid, the value of current_fid will eventually exceed the value of the end_fid in the database table. While it may seem weird, it doesn’t hurt the process. This can be allowed to occur or you can add a little more server-side logic to have the server return -1 as the current_fid to stop the run when end_fid is reached.

 

The “select @value” result will be returned to Python script as the current_fid available for testing:

<xml>
  <current_fid>1</current_fid>
</xml>

 

The Python script will then compare the current_fid with the end_fid that it received earlier to determine whether to stop testing.

 

Once we have the fid of the file that we will test, we can then fetch the information for that specific file:

GET /fuzzers/action_handler.cgi?action=getFileInfo&rid=1&fid=1

 

Using the rid, the web server can query the run_records table to find the table_name that contains the seed files.

select table_name from run_records where rid = ?

 

Assuming the result of that query will be saved as the variable, “$table_name”, the web server can construct the query to retrieve the file name and the directory location that corresponds to the file id:

"select name, location from" . $table_name . "where fid = ?"

 

The web server will return the file name and location with the following XML:

<xml>
  <name>seed.txt</name>
  <location>/fuzzers/files/</location>
</xml>

 

Now, that the location of the seed file is known, it can be downloaded by dffiac.py and saved in the FOE seeds directory. The FOE fuzzer is then started, and dffiac.py waits for FOE to finish testing that seed file. Once FOE testing has completed, the result will need to be recorded by sending the fid and a boolean value indicating whether a crash was identified with that test:

GET /fuzzers/action_handler.cgi?action=recordResult&siid=1&fid=1&crash=1

 

This will result in the following query:

insert into fuzz_records (siid,fid,crash) values (?,?,?)

 

The web server will also record that it has received an update from this fuzzing server instance in the server_instances table to let us know that it is still alive and processing:

update server_instances set lastUpdate = NOW() where siid = ?

 

The result is recorded regardless of success or failure so that you can track which files have been successfully tested with which configs. You could infer this from the run_records, but if a machine dies, a file might be skipped. The server-side code will take the insert_id from the fuzz_records statement (frid) and return the following XML:

<xml>
  <frid>1</frid>
</xml>

 

If there was a crash, the Python script will zip up the crash directory, base64 encode the file and POST it to the upload_cgi identified in the dffiac configuration file. The script will leave the zip file on the fuzzing server if an error is detected during the upload. Along with the zip file, it will send the rid and frid. The rid is used to store files in a web server directory unique to that run. The frid is sent so that the action_handler can update the fuzz_records entry with the location of the uploaded crash file (e.g. “/results/1/zip_file_name.zip”) in the following SQL query:

update fuzz_records set location = ? where frid = ?

 

A successful upload will result in the following XML:

<xml>
  <success>1</success>
</xml>

 

A failed upload can return the description of the error to the client with the following XML:

<xml>
  <error>Replace me with the actual error description</error>
</xml>

 

The dffiac.py script will then continue retrieving new files and testing them with FOE until the end_fid is reached. Then the final call to the web server will record that this fuzzing server instance has completed its run and has stopped:

GET /fuzzers/action_handler.cgi?action=recordRunComplete&siid=1

 

The web server will record the completion with the following SQL query:

update server_instances set status='complete', lastUpdate=NOW() where siid = ?

 

The web server will respond to this last request with the following XML:

<xml>
  <success>1</success>
</xml>

 

The last XML response is currently ignored by the Python script but a more robust implementation could double-check for errors.

 

The Python code

The logic for the distributed fuzzing framework is split into one main file (dffiac.py) and three libraries that are contained in a /libs directory. We’ll start with the three libraries in the /libs directory. The code below is the library that contains the utilities for creating the zip file of the crash result.

 

ZipUtil.py (30 lines)

import zipfile
import os

 

class ZipUtil:

 

#Create a zip file and add everything in path_ref
def createZipFile(self, path_ref, filename):
  zip_file = zipfile.ZipFile(filename, 'w')

 

  #Check to see if path_ref is a file or folder
  if os.path.isfile(path_ref):
    zip_file.write(path_ref)
  else:
    self.addFolder(zip_file, path_ref)

 

  zip_file.close()

 

#Recursively add folder contents to the zip file
def addFolder(self, zip_file, folder):
  for file in os.listdir(folder):

 

    #Get path of child element
    child_path = os.path.join(folder, file)

 

    #Check to see if the child is a file or folder
    if os.path.isfile(child_path):
      zip_file.write(child_path)
    elif os.path.isdir(child_path):
      self.addFolder(zip_file, child_path)

 

The second library will base64 encode the zip file prior to uploading it to the web server via a POST method.  On the server side, you will need to base64 decode the file before writing it to disk.

 

PostHandler.py (77 lines)

import mimetools
import mimetypes
import urllib
import urllib2
import base64

 

class PostHandler(object):

 

  def __init__(self,webServer,uploadCGI):
    self.web_server = webServer
    self.upload_cgi = uploadCGI
    self.form_vars = []
    self.file_attachments = []
    self.mime_boundary = mimetools.choose_boundary()
    return

 

  #Add a form field to the request
  def add_form_vars(self, name, value):
    self.form_vars.append((name, value))
    return

 

  #Get the mimetype for the attachment
  def get_mimetype(self,filename):
    mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
    return(mimetype)

  #Add a base64 encoded file attachment
  def append_file(self, var_name, filename, file_ref, mimetype=None):
    raw = file_ref.read()
    body = base64.standard_b64encode(raw)
    if mimetype is None:
      mimetype = self.get_mimetype(filename)
    self.file_attachments.append((var_name, filename, mimetype, body))

  #Get the body of the request as a string
  def get_request_body(self):
    lines = []
    section_boundary = '--' + self.mime_boundary

 

    # Add the form fields
    for (name, value) in self.form_vars:
      lines.append(section_boundary)
      lines.append('Content-Disposition: form-data; name="%s"' % name)
      lines.append('')
      lines.append(value)

 

    # Add the files to upload
    for var_name, filename, content_type, data in self.file_attachments:
      lines.append(section_boundary)
      lines.append('Content-Disposition: file; name="%s"; filename="%s"' % 
        (var_name, filename))
      lines.append('Content-Type: %s' % content_type)
      lines.append('Content-Transfer-Encoding: Base64')
      lines.append('')
      lines.append(data)

 

    #Add the final boundary
    lines.append('--' + self.mime_boundary + '--')
    lines.append('')

 

    #Combine the list into one long string
    CRLF = 'rn'
    return CRLF.join(lines)
  #Send the final request
  def send_request(self):
    request = urllib2.Request(self.web_server + self.upload_cgi)
    content_type = 'multipart/form-data; boundary=%s' % self.mime_boundary
    request.add_header('Content-type',content_type)

 

    form_data = self.get_request_body()
    request.add_header('Content-length',len(form_data))
    request.add_data(form_data)

 

    result = urllib2.urlopen(request).read()
    return result

 

 

The third library handles the communication between the client and server. It will generate the GET requests and parse the XML responses.

 

actionHandler.py (94 lines)

import urllib
import urllib2
from xml.dom.minidom import parseString

 

class ActionHandler:

 

  #Initialize with the information from the config file
  def __init__(self,options,localLog):
    self.webServer = options['runoptions']['web_server']
    self.uploadCGI = options['runoptions']['upload_cgi']
    self.actionCGI = options['runoptions']['action_cgi']
    localLog.write("Configured web servern")

 

  #Parse the XML for the requested text value
  def getText(self,nodelist):
    rc = []
    for node in nodelist:
      if node.nodeType == node.TEXT_NODE:
        rc.append(node.data)
    return ''.join(rc)

 

  #Make a web request to the server with the provided GET parameters
  def retrieveInfo(self,values):
    url = self.webServer + self.actionCGI
    data = urllib.urlencode(values)
    response = urllib2.urlopen(url,data)
    xml = response.read()
    response.close()
    return(xml)

 

  #Get the information for the rid provided in the config file
  def getRunInfo(self,rid):
    values = {'action':'getRunInfo',
      'rid': rid}
    xml = self.retrieveInfo(values)
    dom = parseString(xml)
    run_type = self.getText(dom.getElementsByTagName("run_type")[0].childNodes)
    start_fid = self.getText(dom.getElementsByTagName("start_fid")[0].childNodes)
    end_fid = self.getText(dom.getElementsByTagName("end_fid")[0].childNodes)
    return (run_type,start_fid,end_fid)

 

  #Record that this server instance is starting a run
  def recordServerStart(self,rid,serverName):
    values = {'action':'recordServerStart',
      'rid': rid,
      'serverName':serverName}
    xml = self.retrieveInfo(values)
    dom = parseString(xml)
    lastrowid = self.getText(dom.getElementsByTagName("siid")[0].childNodes)
    return (lastrowid)

 

  #Record that the server is now complete with its tests
  def recordRunComplete(self,siid):
    values = {'action':'recordRunComplete',
      'siid': siid}
    xml = self.retrieveInfo(values)

 

  #Get the fid for the next file to be fuzzed
  def getNextFid(self,rid,fid,run_type):
    values = {'action':'getNextFid',
      'fid':fid,
      'run_type':run_type,
      'rid':rid}
    xml = self.retrieveInfo(values)
    dom = parseString(xml)
    current_id = self.getText(dom.getElementsByTagName("current_fid")[0].childNodes)
    return current_id

 

  #Get the file name and location for the selected fid
  def getFileInfo(self,rid,fInfo):
    values = {'action':'getFileInfo',
      'rid':rid,
      'fid':fInfo.fid}
    xml = self.retrieveInfo(values)
    dom = parseString(xml)
    fInfo.name = self.getText(dom.getElementsByTagName("name")[0].childNodes)
    fInfo.location = self.getText(dom.getElementsByTagName("location")[0].childNodes)

 

#Record the result from the fuzzing test
  def recordResult(self,siid,fid,result):
    values = {'action':'recordResult',
      'siid':siid,
      'fid':fid,
      'crash':result}
    xml = self.retrieveInfo(values)
    dom = parseString(xml)
    frid = self.getText(dom.getElementsByTagName("frid")[0].childNodes)
    return frid

 

Finally, we get to the main file which is responsible for reading the config file and driving the fuzzing run. This is the only file that is specific to the FOE fuzzer.

 

dffiac.py (177 lines)

import os
import shutil
import socket
import subprocess
import sys
import urllib2
import ConfigParser
import time

 

sys.path.append("libs")

 

from ZipUtil import ZipUtil
from PostHandler import PostHandler
from ActionHandler import ActionHandler

 

#This will track the fid, and location of the file
class FileInfo:
  pass

 

#Convert the options in the config file to lists
def parse_options(config):
  options = {}
  for section in config.sections():
    options[section] = {}
    for (option, value) in config.items(section):
      options[section][option] = value
  return options

#Create a local text file for logging
def openLog(options):
  localLogDir = options['logoptions']['log_dir']
  runName = options['runoptions']['run_id']
  timestamp = int(time.time())
  localLog = open(localLogDir + runName + '_' + str(timestamp) + '.txt', 'w')
  localLog.write("Starting run: " + runName + " at " + str(timestamp) + "n")
  return localLog

 

#Close the local text file log
def closeLog(localLog):
  localLog.write("COMPLETEn")
  localLog.close()

 

#Download the next file to be fuzzed
def getNextFile(fInfo, options, foe_options, localLog):
  u = urllib2.urlopen(options['runoptions']['web_server'] + fInfo.location + fInfo.name)
  localFile = open(foe_options['runoptions']['seedsdir'] + "\" + fInfo.name, 'wb')
  localFile.write(u.read())
  localFile.close()
  localLog.write ('Created file: ' + foe_options['runoptions']['seedsdir'] + "\" + fInfo.name + 'n')

 

#Store the results in a zip file
def createZip(outputDir,filename):
  zipTool = ZipUtil()
  zipTool.toZip(outputDir,filename)
  zipFile = open(filename,'rb')
  return zipFile

 

#Post the zip file to the server
def postZip(options,frid,rid,filename,zipFile):
  form = PostHandler(options['runoptions']['web_server'], options['runoptions']['upload_cgi'])
  form.add_form_vars('frid',frid)
  form.add_form_vars('rid',rid)
  form.append_file('fname',filename,zipFile)
  result = form.send_request()
  return result

 

if __name__ == "__main__":
  if (len(sys.argv) < 2):
    print "usage: %s <runconfig.cfg>" % sys.argv[0]
    exit(1)

 

  #Read the dffiac config file
  configFile = sys.argv[1]
  if not os.path.exists(configFile):
    print "config file doesn't exist: %s" % configFile
    exit(1)
  config = ConfigParser.SafeConfigParser()
  config.read(configFile)

 

  #Read the foe config file
  options = parse_options(config)
  config2 = ConfigParser.SafeConfigParser()
  config2.read (options['foeoptions']['config_location'])
  foe_options = parse_options(config2)

 

  #Set up logging
  localLog = openLog(options)

 

  #Configure the web server
  aHandler = ActionHandler(options, localLog)

 

  #Get the information for this run
  rid = options['runoptions']['run_id']
  (run_type,start_fid,end_fid) = aHandler.getRunInfo(rid)

 

  #Record server start
  hostName = socket.gethostname()
  hostIP = socket.gethostbyname(hostName)
  serverName = hostName + "_" + hostIP
  siid = aHandler.recordServerStart(rid, serverName)
  localLog.write("Starting as server instance: " + siid + "n")

 

  #Get the first file to be processed
  fInfo = FileInfo()
  fInfo.fid = aHandler.getNextFid(rid,start_fid,run_type)
  localLog.flush()

 

  #loop until done
  while (int(fInfo.fid) <= int(end_fid)):
    #Get the location information for the current file
    aHandler.getFileInfo(rid,fInfo)

 

    #Download and store the file
    getNextFile(fInfo,options,foe_options,localLog)

    outputDir = foe_options['runoptions']['outputdir'] + "\" + foe_options['runoptions']['runid']

 

    #Run fuzzer
    exitCode = subprocess.call(options['foeoptions']['python_location'] + " " + options['foeoptions']['foe_location'] + " " + options['foeoptions']['config_location'], shell=True)

 

    #Check for completion of a succesful run
    if exitCode != 0:
      localLog.write("Error running foe on fid " + fInfo.fid + "n")
    else:
      dirList = os.listdir(outputDir)

 

      #Detect whether bugs were found
      if len(dirList) > 2:

 

        #Record the result in fuzz_records
        frid = aHandler.recordResult(siid,fInfo.fid,1)
        localLog.write("Recording frid: " + frid + "n")

 

        #Store the results in a zip file
        filename = frid + "-" + fInfo.name + ".zip"
        file_path = os.getcwd() + filename
        zipFile = createZip(outputDir,file_path)

 

        #Post the zip file back to the server
        result = postZip(options,frid,rid,filename,zipFile)
        zipFile.close()

 

        #Make sure the file got there OK
        if result.find("error") == -1:
          localLog.write("Results successfully uploaded.n")
          os.remove(file_path)
        else:
          localLog.write("There was an error in the upload: " + result + "n")

 

        localLog.write("Found bugs with " + fInfo.fid + "n")
      else:
        #Record no bugs found in the directory
        aHandler.recordResult(siid,fInfo.fid,0)
        localLog.write("No bugs found with " + fInfo.fid + "n")

 

    #The if len(dirlist) check on the results is complete
    #Erase files so that FOE starts clean on the next run
    os.remove(foe_options['runoptions']['seedsdir'] + "\" + fInfo.name)
    shutil.rmtree(outputDir)
    localLog.flush()

 

    #Get the next FID
    fInfo.fid = aHandler.getNextFid(rid,fInfo.fid,run_type)

 

  #The while loop is complete
  #Record this run instance as being complete
  aHandler.recordRunComplete(siid)

 

  #Close the local file log
  closeLog(localLog)

 

This blog is only meant to describe how you can stand up a basic distributed fuzzing framework based on FOE fairly quickly in approximately 1,000 lines of code. The client-side code turned out to be 378 lines, my server-side action_handler CGI was 150 lines and the upload CGI was 72 lines of Perl. That is enough to get the script to run based on information from a database. With the remaining 400 lines, I created a CGI to display the status of my runs and a CGI to generate a run. You will also want to write a script to mirror the dffiac.cfg and FOE configuration file across machines. Over time, I expect that you would make this design more robust for your particular infrastructure and needs. You can also expand this infrastructure for your other fuzzers with some modifications to the main file. What I provide here is just enough to help you get started performing distributed fuzzing with a small amount of coding and the FOE fuzzer.

 

Permission for this blog entry is granted as CCplus, http://www.adobe.com/communities/guidelines/ccplus/commercialcode_plus_permission.html

 

Examples of Community Engagement

Recurity Launches Blitzableiter 1.0 at FIRST & Drs. Venkatakrishnan and Hamlen Awarded National Science Foundation Trustworthy Computing Grant

Recurity Launches Blitzableiter 1.0 at FIRST

Ever since a late-night conversation with Felix ‘FX’ Lindner, Brad Arkin and myself at Black Hat last summer, members of the ASSET and Adobe Flash engineering teams have been assisting researchers from Recurity Labs, the German security research and consultancy company, in their development of Blitzableiter (“Lightning Rod”). This mitigation technology filters malicious Flash (.SWF) files before they can carry out an attack against a vulnerability in the Adobe Flash Player.

Today, Recurity officially launched Blitzableiter v1.0 at the FIRST conference in Vienna (June 12-17, 2011). The Blitzableiter beta has already been used by several companies, including a large social networking site in Europe.

Blitzableiter is a signature-free, open source mitigation technology for enhancing Flash content security that uses complete format normalization instead of scanning. A potentially malicious input file is read, parsed and interpreted, applying strict rules of specification compliance.  If the input file violates those rules, it’s rejected.  After initial parsing, the original input file is discarded completely, and a new file is created based on the information obtained from the original input. Blitzableiter supports automatic modification of AVM1/2 (AS2/3) code in Flash (.SWF files) and during testing has demonstrated the ability to block almost every Flash Player exploit sample observed since 2010. It supports version SWF3 to SWF10. The 1.0 release version can be used client-side with NoScript in Firefox, or integrated with proxy servers or firewalls using an included ICAP server.

Congratulations to the Recurity team on the Blitzableiter launch! For more information on Blitzableiter, visit the Recurity Labs website at http://blitzableiter.recurity.com/.

Drs. Venkatakrishnan and Hamlen Awarded National Science Foundation Trustworthy Computing Grant

Congratulations also to Drs. Venkatakrishnan and Hamlen at UI Chicago and UT Dallas for being awarded a National Science Foundation Trustworthy Computing grant for ‘Securing Web Advertisements.’ Their project will be a combination of Dr. Hamlen’s research in Certified In-lined Reference Monitoring (IRM) system for ActionScript bytecode and Dr. Venkatakrishnan’s research on HTML and JavaScript advertisements to create a unified web advertisement security framework.

Both projects serve as great examples of members of the security community, academia and vendors collaborating to help protect customers from malicious attacks.

Peleus Uhley
Platform Security Strategist

Advancing Flash Player Privacy and Security

Today, Adobe has released Flash Player 10.3, which includes several important new privacy and security features for our customers. Let us provide some background and perspective on how these features came about and what they mean for our customers:

Adobe has been leading a signficant privacy initiative focusing on managing Flash Player Local Shared Objects (LSOs). We have actively participated in industry discussions on the topic and worked closely with Carnegie Mellon University and the Center for Democracy and Technology (CDT) to follow up on the reported misuse of Flash Player LSOs to “respawn” browser cookies—and to make changes in our technology to help address the associated privacy concerns. It was important for Adobe to understand how our development community was using our technology. The Carnegie Mellon University study showed that respawning browser cookies no longer appeared to be an active practice on the sites that were studied.

While these were promising results, we also wanted to further improve our end-users’ ability to control their settings and data. Flash Player 10.3 includes a number of exciting new features designed to give end-users more (and easier) ways to control their privacy:

  • Adobe coordinated with the open-source browser community to develop the ClearSiteData NPAPI. This new API allows the browsers to communicate a user’s desire to wipe user data stored by installed browser plugins. Now, when end-users go into their browser settings to clear their browser history or clear their cookies, they will be able to clear both their browser data as well as their plugin data. This API was designed so that any plugin can participate, and Flash Player is the first plugin to support the new API. Mozilla Firefox 4 already supports the new API today, and the Google Chrome team currently offers browser support for the feature in their dev channel. We expect to have official support across all open source browsers in the near future.
  • In addition to coordinating with the open-source browsers, Adobe also teamed up with Microsoft to provide equivalent functionality within Internet Explorer. With today’s launch, end-users can start taking advantage of this functionality in Internet Explorer 8 and 9. Microsoft even created a demo page, so that end-users can try out the functionality.
  • Another key focus area was to improve the Flash Player Settings Manager itself by making it easier for end-users to manage their Flash Player settings. In January, Emmy Huang, group product manager for Flash Player, announced our native control panel for Flash Player 10.3. Until now, end-users could manage their Flash Player settings by right-clicking on content written for Flash Player and selecting “Global Settings…” or by visiting the online Flash Player Settings Manager. The online version of the Flash Player Settings Manager was not very intuitive for end-users. With Flash Player 10.3, we have created a new native control panel for Windows, Macintosh and Linux desktops that will allow end-users to manage all of the Flash Player settings, including camera, microphone and Local Shared Objects. The control panel can still be found by right-clicking on content written for Flash Player and selecting “Global Settings.” However, starting with Flash Player 10.3, it can now also be found in the Control Panel or System Settings for your operating system. As an example, on Windows operating systems, the new native control panel in Flash Player 10.3 can be found under Control Panel -> Programs.

In addition to these privacy improvements, Flash Player 10.3 includes a new auto-update notification mechanism for the Mac OS platform. In the past, Mac users often had trouble keeping up with Flash Player updates since the Mac OS and Flash Player ship schedules are not in sync. With this new feature, Flash Player will automatically check each week for new updates and notify the user when new updates are available. This feature matches the auto-update notification capability previously implemented on Microsoft Windows.

Please check out Flash Player 10.3, and try out the new control panel. Also, watch for updates in your open-source browsers, since you will soon be receiving the ability to clear plugin data directly from the browser. We want to thank everyone who worked so hard in assisting us with this effort.

Peleus Uhley, Platform Security Strategist
Lindsey Wegrzyn, Sr. Privacy Product Manager

Updated 5/15/2011: Added reference to Mozilla Firefox 4 already offering support for the new ClearSiteData NPAPI.

Flash content and the same-origin policy

Peleus here. Research was recently published that makes several claims regarding Flash content on sites that allow end-user uploads that I would like to address.  The topic raised is not news; it’s something that has been understood and discussed by the security community for years. Most importantly, this is not a vulnerability in Adobe Flash Player.  Web servers that choose to accept user-uploaded content also choose to accept the risks that go along with that functionality.  Flash Player’s behavior is consistent with other web technologies and the web browser security model. Several web technologies pose the same risk to servers that allow end-user uploads.

 

To really understand the issue, we need to take a step back and understand the same-origin policy. This policy basically states that two pieces of content hosted on the same domain and loaded by the same protocol trust each other.  Conversely, two pieces of content hosted on different domains do not trust each other and cannot interact. The same-origin policy is enforced by the web browser, and all web content must follow it.
 
The researchers noticed that there are many web applications that allow end-users to upload their own content to the same domain as the web application itself. According to the same-origin policy, this means that the web application trusts the end-user’s content in the same way that it trusts its own content. In most real world scenarios, this is not true. The web application does not trust end-user content and does not mean to grant end-user content any privileges on its domain. Quoting Law #4 of Microsoft’s
10 Immutable Laws of Security
, “If you allow a bad guy to upload programs to your web site, it’s not your web site anymore.” Microsoft originally published these laws in 2000 and TechNet Magazine revisited them in 2008 where they concluded, “Law 4, at least in spirit, still holds–in spite of some sites that are designed to permit people, bad and good, to upload programs to them.”
 
When Flash content (SWF), as well as any other web content, is hosted on the same domain as the surrounding HTML, the same-origin policy defines a trust relationship between the two. Proposals have been made to have Flash content assume there is never any trust and therefore ask permission for each and every request that it makes. This would break all legitimate deployments of Flash content on the web and require all administrators to change their configurations in order to start supporting Flash again. It also would not solve the problem. Flash content is not the only content-type that can execute JavaScript when clicked on by a user. Law #4 still holds. The fundamental problem is that the site is not properly handling the upload of arbitrary active content. Attackers can still attempt to upload other forms of server-side or client-side code in an effort to exploit the server.
 
The web has already widely deployed much better solutions. Sites solve the problem by hosting untrusted content on a different domain. If a site developer wants to host arbitrary uploaded content on a site and does not want SWF content to execute on their site, the SWF can be served with headers, which instruct Flash Player to treat the file as an attachment. Flash Player will acknowledge that request and present an Open/Save/Cancel dialogue to the user rather than render the content. Developers may also choose to limit uploads to only the types of content that they plan to support.  These steps to properly managing user-generated active content uploads may be challenging, but good web security requires vigilance against this type of risk alongside all the other familiar web threats like CSS, CSRF, …
 
We understand that developers have many challenges when managing rich content, and we are
working with the industry in discussing and supporting solutions (such as Content-Security Policies) to minimize the risks and educate the community. Our goal is to enable all of our customers (end-users, developers, and web site owners) with solutions that will protect them from the critical risks they face.

Securely deploying cross-domain policy files

Peleus here. There has been a recent focus on cross-domain policy deployments in the media, so I thought this would be a good time to remind people of some best practices.  Cross-domain security was a focus of my recent presentation at the Microsoft BlueHat conference. I have also covered this topic within my Creating More Secure SWF Web Applications article.  With more plugin platforms adopting the format and similar controls being added to HTML5, it is becoming increasingly important to ensure the correct deployment of a cross-domain architecture.  Here are five best practices that should be followed to ensure a secure deployment.

 

1. Avoid full wildcard permissions (domain=”*”,  headers=”*”, to-ports=”*”).  There are only a small number of legitimate use cases for full wildcard (*) permissions.  If granting full permission is absolutely necessary, then the best practice is to create a sub-domain on your site whose explicit purpose is to serve cross-domain data.  Another option is to leverage Flash Player’s support of per-directory cross-domain permissions and place the data and the full wildcard cross-domain policy within a sub-directory of the site dedicated for that purpose.  Full wildcards on internal networks can also be dangerous since they can result in external content being granted access to internal resources. A full wildcard should also never applied to the headers attribute of the allow-http-request-headers-from element or the to-ports attribute of the allow-access-from element in production.  Once a wildcard permission has been deployed, it can be very challenging to restrict permissions at a later date because there is no easy way to identify what content depends on that permission.

 

2. Don’t use sub-domain wildcards (*.example.org) with domains that allow user-uploaded content.  This issue has grown as an increasing number of sites on the web support hosting user generated content. Quoting Microsoft’s 10 Immutable Laws of Security, “Law #4: If you allow a bad guy to upload programs to your website, it’s not your website any more.”  Therefore, if you are granting cross-domain permission to *.example.org and upload.example.org hosts end-user content, then you are not trusting just example.org.  You are also trusting all the users who are permitted to upload content to upload.example.org.  An attacker can upload a malicious SWF to upload.example.org, use the *.example.org permission to retrieve sensitive data from your site and then pass that data back to the attacker’s web site.

 

3. Avoid cross-domain permissions on sites that require authentication.  Any data that requires authentication for access probably should not be available to third-party domains.  Flash Player does not provide access to the header of an HTTP response.  Therefore developers may assume that SWF content cannot gain access to the session information stored within the cookie headers.  However, some architectures will add the session information as a parameter onto the end of a URL contained within the response body where an attacker can gain access to it.  Once an attacker has access to the session information of the victim, they can impersonate that user.

 

4. Cross-domain policies require periodic maintenance.  Your web site will grow and change over time and you will need to reevaluate the cross-domain permissions with respect those changes.  If you are granting permissions to domains outside of your control, then keep in contact with that party to ensure that the permissions are still necessary and that they are still used within the same context.  You can limit your risk by periodically removing excess permissions.

 

5. Cross-domain permissions are not the only method for sharing data between domains.  Rather than using the client to bridge two domains, consider using server-side code to make the cross-domain request via a server-to-server channel.  Server-side code can enforce stricter controls on the request and act as a proxy between the client and the second domain.  This has the trade-off of increasing traffic to the server but may allow for a more controlled channel.  Be sure to consider all of your options before determining the best solution.

 

Cross-domain policies are a part of the code that makes up your web application.  Like any other code, they require periodic maintenance and review to ensure security.  Cross-domain policies should follow the same principle of least privilege that you apply to code. Only the minimum permissions required for the architecture should be enabled.  The best architectures are simple and straightforward. Creating a sub-domain whose explicit purpose is to host cross-domain content can prevent unintentional disclosure. Overall, approach cross-domain policies as you would any other code. Following these best practices can help you secure your cross-domain deployment.

 

For further information and best practice guidance, please see: