Posts in Category "Security"

Striving for a Security Culture

Security is a discipline that is forever changing. There are always new threats on the horizon and we are constantly warning people that software is never as secure as we might like it as we cannot predict the future. Be alert! Never let your guard down! Think like an attacker! These are well tread refrains.

The truth is, however, none of us can be a perfect sentry. Yes, there is a lot of security work that can be automated and integrated into the product lifecycle, but there will always be a human element and humans are not perfect. They can get bored, complacent, and rely too much on crutches that have worked for them in the past. Also, there are limits to the amount of information that a person can take in. A focus on security can get lost amongst things like needing to create new features or meeting deadlines.

So how do we make sure that security awareness is not lost? That all the training we spend precious treasure creating doesn’t get lost in the noise?

The answer is to systematically work towards creating a culture of security.

A healthy security culture is one where folks have internalized your security protocols to the extent that desirable security actions are the default. For example: suspicious activity on a server? Immediately report it to the incident response team.

Culture is what you do with the information, education and tools that you have been exposed to. Creating an organizational culture strong in security principles and awareness requires identifying deliberate behaviors, creating opportunities for these behaviors to exist, rewarding them, and measuring the result. This can often be achieved through “gamification”.

So, how did we use this technique effectively? We wanted to foster a culture of open communication about security. We want everyone to be able to join the conversation, but we knew we were going to need to find allies in order to make the conversation authentic and genuine. We created an e-mail distribution list and we started inviting everyone who took our training to join. We also encouraged our security champions to join as watchers and helpers for any issues on the thread.

After a while, it became clear that our black and brown belts (the most advanced trainees) were very active on the list, researching and answering questions that would have cost our dedicated Security Researchers valuable time. We maintain a program where we give spontaneous positive feedback to managers, and the brown and black belt holders are consistently at the top of that list for their participation.

Over the years, the list has grown considerably, it is self-mediated, and remains a thriving community where anyone can ask a question. It is the first place we go to make announcements about our security efforts, where we learn from across the company tips and tricks to help keep things more secure, and where we share the latest security news.

Let’s break down further what we did here:

  1. Declared goal – open communication
  2. Created e-mail list and advertised it to champions and thought leaders
  3. Let people interact without over moderating
  4. Reward participants with positive feedback, engage in their topics, and encourage public praise
  5. Keep track of subscriber growth

Here are a few of the many dividends that have come from this tool to encourage our open security culture:

  • Security issues or potential problems within our products can be more quickly identified
  • We are often made aware of cutting edge industry and academic research that might be useful
  • There is a sense of “community” amongst members of this list
  • It has become easier to spread information to the right people very quickly

As mentioned above, security is an ever moving target. Security culture efforts need to adapt to meet that target. The culture of openness example above is a broad target designed to change a complex set of behaviors. You can use the same strategy to address more simple behaviors – like badge surfing or locking computer screens. Prioritize the behaviors you want to change, then focus on them, reward people for even the smallest of victories and measure the difference in the occurrence of the behavior.

Josh Kebbel-Wyen
Sr. Security Awareness and Training Program Manager

Building a Team of Digital Marketing Security Champions

Two years ago, I joined the Digital Marketing Product Security Team and took on the responsibility of establishing and managing the Secure Product Lifecycle (SPLC) process for Digital Marketing Product Engineering. There are currently eight Digital Marketing Solutions with engineering teams located all over the world.  Many of these solutions came to Adobe by way of acquisition.  I work with differing stacks, languages, company cultures, and time zones.  I knew some of the engineers from having run our 3rd Party Penetration Testing program for three years – however, I was mostly starting the process from scratch. My main goal was to lower security overhead in the product development cycle and leverage existing processes.

I am very passionate about quickly making improved security an integrated part of our product development and leveraging as many existing processes and tools as possible.   In order to promote security knowledge throughout the large Digital Marketing engineering organization.  I created a human “botnet” of security champions. These champions come from positions all over the organization and coordinate with our security team to facilitate ongoing management and enforcement of our SPLC process.

Security, admittedly, has a bit of an “image problem” among development teams.  It is something that developers often think of as this big, scary set of tasks intended to make their jobs more difficult or less enjoyable.  We placed a big emphasis on changing this perception. The Digital Marketing Security team is focused on being a supportive, service organization – a far cry from the perception that we can be a terrible force of nature leaving engineers feeling like they’ve been hit by a truck or would like to be.  Rather than coming in with the metaphorical hammer, we thought, “can we get people to actually enjoy their interactions with our security team?  How can we make this incredibly important, but often dreaded, piece of software development an integral and easier to implement piece of the existing process?”

The first thing I did was to meet with the solution owners and program managers to learn about how these teams develop and deliver software for these SAAS offerings.  Adobe has an incredible program management network, and an existing Service Lifecycle program that I was able to leverage and adapt to help meet requirements of our Secure Product Lifecycle.  I worked with the program managers to figure out how we could best add SPLC steps to their development and release process. I also ensured we had a clear process for adding security requirements and checkpoints to the release process. I worked with solution engineering directors to identify Security Champions on their engineering teams who would work with me to continue to improve our approach to security for the solutions.

A Security Champion is: ‘An advocate of security and the Digital Marketing Security team’s point of contact for the solution.  The champion has a good understanding of the technology, an interest in ensuring better security for their offering, and a strong personal network in the engineering organization.’  Once this human “botnet” of Security Champions was established, the heavy lifting began.  I set key performance indicators (KPIs) for the different elements of the SPLC around security training, threat modeling, static/dynamic analysis and penetration testing. The very first KPI that we focus on, for the purpose of enabling the proper background for having security conversations with the engineers, is technical security training.  Adobe’s corporate secure software engineering team (known as “ASSET”) has created a fantastic training program that focuses on technical security topics and awards certifications in the form of white and green belts, similar to karate training. Each of the program managers have added this training to the new engineer onboarding steps and they and the security champions have helped to develop strong measurements for the other KPIs.

My Security Champions helped increase the pervasiveness of our “security culture” more than I could have imagined when first starting this program.  They are one of the driving forces in helping to further improve security across Adobe’s Digital Marketing solutions.  They have been an amazing force multiplier helping to prioritize security practices in their teams’ design process, roadmap development, and mindset.

About 6 months after kicking off the Security Champions program, Digital Marketing Engineering had grown their base of security knowledge to have over 95% of their engineers white and green belt certified.  We’d also increased the number of threat models, penetration tests, ongoing security projects, and automated security testing. Our metrics against these initiatives have continued to increase and improve. The teams are more proactively involving the Digital Marketing and corporate security teams in their design discussions helping to ensure better security implementations throughout the process.

Messages like this from the teams show it’s working and make it all worth it:


We’re committed to building and maintaining the trust of our Digital Marketing customers by developing and providing them with the most secure software possible – solutions that help meet business demands and allow configurations to help meet their security and compliance needs.  The SPLC and Security Champions program have helped to broaden the security knowledge and awareness of the Digital Marketing engineering teams.  We will continue to raise that bar by continuing to iterate and improve on these programs.

Julia Knecht
Security Analyst, Digital Marketing

Adobe CCF Enables Quicker Adherence to Updated PCI Standards

The e-commerce store has been a PCI level 1 certified merchant for the last few years.  Adobe has significantly reduced its Card Holder Data environment (CDE) scope for this environment by using an external tokenization solution and maintains PAN-free environment by not storing any Primary Account Numbers (PAN) in its internal network. Adobe has implemented its Common Controls Framework (CCF) within the Card Holder Data environment which allows it to use the same set of controls to meet with requirements set forth by Payment Card Industry Data Security Standard PCI DSS V3.1 and many other security/compliance frameworks like ISO27001:2013, SOC2, among others. CCF is a set of approximately 250 controls designed specifically for Adobe’s business and provides the benefit by rationalizing the overlapping requirements across 10 different compliance and security frameworks.

PCI Security Standards Council (PCI SSC) recently released the latest version of the Data Security Standard V3.2. One of the notable changes in the PCI DSS V3.2 is the additional clarification provided around the use of multi-factor authentication for all administrative and remote access to the CDE.

PCI DSS V3.2 reference:

“8.3 Secure all individual non-console administrative access and all remote access to the CDE using multi-factor authentication.”

 By implementing CCF within the CDE, Adobe has already established a baseline control which requires all remote VPN sessions and production environments to be accessed via multi – factor authentication. This baseline control was adopted to meet with the requirements established by the more stringent of the compliance frameworks, hence allowing for Adobe to already be complaint with the clarifications provided in the PCI DSS v3.2 around multi-factor authentication.

Prasant Vadlamudi
Manager, Risk Advisory and Assurance Services (RAAS)

Fingerprinting a Security Team

The central security team in a product development organization plays a vital role in implementing a secure product lifecycle process.  It is the team that drives the central security vision for the organization and works with individual teams on their proactive security needs.   I lead the technical team of proactive security researchers in Adobe. They are all recognized security experts and are able to help the company adapt to the ever changing threat landscape.  Apart from being on top of the latest security issues and potential mitigations that may need to be in place, the security team also faces challenges of constant skill evolution and remaining closely aligned to the business.

This post focuses on the challenges faced by the security team and potential ways to overcome them.

Increase in technologies as a function of time.

A company’s product portfolio is a combination of its existing products, new product launches, and acquisitions intended to help bridge product functionality gaps or expand into new business areas.  Over time, this brings a wide variety of technologies and architectures into the company.  Moreover, the pace of adoption of new technologies is much higher than the pace of retiring older technologies.  Therefore, the central security team needs to keep up with the newer technology stacks and architectures being adopted while also maintaining a manageable state with existing ones. An acquisition can further complicate this due to an influx of new technologies into the development environment in a very short period of time.

Security is not immune to business evolution.

The cloud and mobile space have forced companies to rethink how they should offer products and services to their customers.  Adobe went through a similar transformation from being a company that offers desktop products to one that attempts to strike the right balance between desktop, cloud, and mobile.  A security team needs to also quickly align with such business changes.

Multi-platform comes with a multiplication factor.

When the same product is offered on multiple operating systems, on multiple form factors (such as mobile and desktop), or deployed on multiple infrastructures, security considerations can increase due to the unique qualities of each platform. The central security team needs to be aware of and fluent in these considerations to provide effective proactive advice.

Subject matter expertise has limitations.

Strong subject matter expertise helps security teams’ credibility in imparting sound security advice to teams.  For security sensitive areas, experts in the team are essential to providing much deeper advice.  That said, any one individual cannot be an expert on every security topic.  Expertise is something that needs to be uniformly distributed through a team.

These challenges can be addressed by growing the team organically and through hiring.  Hiring to acquire new skills alone is not the best strategy – the skills required today will soon be outdated tomorrow.  A security team therefore needs to adopt strategies that allow it to constantly evolve and stay current. A few such strategies are discussed below.

T-Shaped skills.

Security researchers in a security team should aim for a T-Shaped skill set.  This allows for a fine balance between breadth and depth in security. The breadth is useful to help cover baseline security reviews.  The depth helps researchers become specific security subject matter experts. Having many subject experts strengthens the overall team’s skills because other team members learn from them and they are also available to provide guidance when there is a requirement in their area of expertise.

Strong Computer Science foundations.

Product security is an extension of engineering work.  Security requires understanding good design patterns, architecture, code, testing strategies, etc. Writing good software requires strong foundations in computer science irrespective of the layer of technology stack one ends up working on. Strong computer science skills can also help make security skills language and platform agnostic.  With strong computer science skills, a security researcher can learn new security concepts once and then apply to different platforms as and when needed.  With such strong fundamentals, the cost of finding out the “how” on new platforms is relatively small.

Hire for your gaps but also focus on ability to learn quickly.

A working product has so many pieces & processes that make it work.  If you can make a mental image of what it takes to make software, you can more clearly see strengths and weaknesses in your security team.  For example, engineering a service requires a good understanding of code (and the languages of choice), frameworks, technology stacks (such as queues, web server, backend database, third party libraries), infrastructure used for deploying, TLS configurations, testing methodologies, the source control system, the overall design and architecture, the REST interface, interconnection with various other services, the tool chain involved – the list is extensive. When hiring, one facet to evaluate in a candidate is whether he or she brings security strengths to the team through passion and past job experience that can fill the team’s existing gaps.  However, it can be even more important to evaluate the candidate’s willingness to learn new skills.  The ability to learn, adapt, and not be held captive to one existing skill set is an important factor to look for in candidates during hiring.  The secondary goal is to add a variety of security skills to the team and try to avoid duplicating the existing the skill set already in the team.

“Skate where the puck’s going, not where it’s been.”

To stay current with the business needs and where engineering teams are headed, it is important for a security team to spend a portion of their time investigating the security implications of newer technologies being adopted by the product teams.  As Wayne Gretzky famously said, “you want to skate where the puck’s going, not where it’s been.” However, security teams need to cover larger ground. You do have to stay current with new technologies being adopted. Older technologies still get used in the company as only some teams may move away from them. So it would be wise not to ignore those older technologies by maintaining expertise in those areas, while aiming to move teams away from those technologies as they become more difficult to effectively secure.  Predicting future areas of investment is difficult.  Security teams can make that task easier by looking at the industry trends and by talking to engineering teams to find out where are they headed.  The managers of a security team also have a responsibility to stay informed about new technologies, as well as future directions their respective companies may go in, in order to invest in newer areas to grow the team.

Go with the flow.

If a business has taken a decision to invest in cloud or mobile or change the way it does business, a security team should be among the first in the company to detect this change and make plans to adapt early.  If the business moves in a certain direction and the security team does not, it can unfortunately label a team as being one that only knows the older technology stack.  Moreover, it is vital for the security team to show alignment with a changing business. It is primarily the responsibility of the security team’s leadership to detect such changes and start planning for them early.

Automate and create time.

If a task is performed multiple times, the security team should evaluate if the task can be automated or if a tool can do it more efficiently.  The time reduced through automation and tooling can help free up time and resources which can then be used to invest in newer areas that are a priority for the security team.

Growing a security team can have many underlying challenges that are not always obvious to an external observer.  The industry’s primary focus is on the new threat landscapes being faced by the business.  A healthy mix of organic growth and hiring will help a security team adapt and evolve continuously to the changes being introduced by factors not in their direct control.  It is the responsibility of both security researchers and the management team to keep learning and to spend time detecting any undercurrents of change in the security space.

Mohit Kalra
Sr. Manager, Secure Software Engineering

Reflections on Pwn2Own

Returning from CanSecWest left me reflecting on how the Pwn2Own competition has evolved over time. A lot has changed in the Pwn2Own competition over the years. The event has grown in attendance, competitors, and complexity, just as the industry has grown.

For the first contest in 2007, no one competed on the first day. Shane McCauley called fellow security researcher Dino Dai Zovi in NYC that night and urged him to compete. Dino was able to write the exploit over night and win the contest on the second day. Visually, the attacks seem no different from year to year. The contestant sits down at the machine, and “seconds” later the calculator (or notepad in this year’s case), pops up on the screen.  However, the preparation for the attacks from 2007 to 2016 are now drastically different, with contestants preparing attacks weeks in advance.

Media coverage around security advisories is often just a run down of how many CVEs a vendor released that month. People often imply from these articles that all the CVEs are easily exploitable and trivially weaponizable. This can lead to false perceptions that exploiting the software is a simple task. In reality, a CVE from a vendor is not a guarantee of exploitability. Even if the CVE can give the attacker the ability to overwrite memory, that is not a guarantee that it can be weaponized. Technologies such as ASLR (Address Space Layout Randomization) weren’t even released for Mac OS X when Dai Zovi competed in 2007. Today’s attackers have to work around defenses such as CFG (Control Flow Guard), Isolated Heaps, and a number of other technologies designed to prevent a crash from becoming an exploit.

In addition, competitors have to deal with the fact that the contest frequently occurs after Patch Tuesday where a vendor’s security improvements could interfere with their attack. Adobe has been aggressively adding mitigations to Flash Player over the last few months. In our release the week before the contest, we added changes to zero memory more often and leveraged the Windows’ low fragmentation heap. Both Adobe and Google found that some of the contestant’s entries overlapped with recent security reports. Part of the reason for the increasing payouts for the winners comes from the fact that the targets for the competition all have active security teams and external communities that are constantly working to improve the platform. As the community and vendors continue to mature their software, bug or mitigation collisions become more of a material risk for competitors. While it may not seem like it on the surface, the attackers are trying to hit moving targets.

The Flash Player updates prior to the contest led to some failed attempts at the competition. The failed attempts were by teams who had already won under different categories. Therefore, the competitors were clearly highly skilled and had already proven themselves to be capable of weaponizing exploits for the target platform. The reality is that what they are attempting to do is not easy and the failures serve to remind us how hard it is to get those wins. When any competition gets to a certain level, even the most skilled players are going to experience some losses.

That said, the contest always has its share of winners. Those wins demonstrate that there is always more that vendors can do in order to improve security. While the individual bugs help, Pwn2Own is truly valuable because it shows how different researchers will try to bypass the existing mitigations to create the fully weaponized exploit. That insight into different attack approaches inspires us as vendors to come up with the next generation of defenses.

Like many pros, the winning contestants always make it look easy. Although, as an industry we are often too quick to lump everything in the same “it is easy because it is completely vulnerable” bucket. Companies like Adobe, Microsoft, and Google are in a constant sprint with attackers. The security industry has progressed from the days of just trying to write clean, well validated code. Today, we are adding in large platform features that serve no other purpose than trying to thwart attackers. These types of features are added at an increasingly frequent basis. The companies who are on the front line of this battle will change and grow over time. It is important for those vying to one day be on the defender’s side of Pwn2Own to understand the current table stakes.

Overall, Pwn2Own is a fun contest to interact with security researchers and to push the industry forward. Beneath the high level pageantry of smoking jackets and large prizes, is a low-level escalation between offensive and defensive strategists. While the visual results from watching in the room seem similar from year to year, the innovation and challenge required to achieve those results increases every year.

Peleus Uhley
Principal Scientist

Adobe @ Women in Cybersecurity 2016

Adobe was a supporter of the Women in Cybersecurity (WiCys) conference again this year. This year’s conference was organized by Tennessee Tech and held in Dallas, Texas. We had a great experience over the three days of the conference which saw women from across industry and academia come together to discuss important security topics and encourage more women to pursue careers in security. Adobe was joined by several major industry peers including Google, Facebook, Cisco, and IBM.

The conference provided a good mix of technical and non-technical sessions. First off was a keynote by Heather Adkins, Director of Information Security at Google. She talked a little bit about how she became the first woman hired in an operational role at Google and also the first woman on Google’s corporate security team. She then discussed her ideas for structuring an incident response organization. Key components of a good incident response organization per Ms. Adkins are: 1) Figure out what happened (forensics), 2) Are we still under attack (Monitoring), 3) Who did it (Threat intelligence), and 4) Restoring the business (Remediation). It was helpful to hear how another large peer organization like Google handles incident response.

One of the workshop sessions we found most interesting was on the topic of “Big Data Analytics for Cyber Security Applications.” It discussed how to leverage big data frameworks in the field of cyber security. The workshop taught us how to create smaller data sets from the huge amount of threat intelligence information received nowadays and process the data sets using tools such a Hadoop or Spark. It also described the varying levels of sanitization and risk associated with using different malware data sets. Data sets in academia are highly sanitized, and low risk, but typically out-of-date. Research data sets are moderately sanitized and have moderate risk associated with download, but are more current. Individual malware collections are the least sanitized, and have the most risk associated with download, but are up-to-date. Adversaries are thus actively modifying their patterns of behavior to avoid detection (polymorphism) – so multiple techniques and tools are needed to adapt our defenses.

We particularly enjoyed the non-technical sessions that included women leaders in the industry talking about their career journey and how they got to where they are today. Particularly impressive was the talk by Shelley Westman, VP of Operations and Strategic Initiatives, from IBM. Shelley talked about the various stages of her career and experiences as a woman at each of those stages. Shelley also discussed the importance of building relationships with male allies who will support opportunities for women in the workplace.

The conference also had some great networking opportunities during the social events hosted by several companies. We got a chance to attend one hosted by Facebook and participate in a “lean-in” circle conversation. We also attended the Cisco event where we had great interactions with their security team and got to learn more about their security and trust organization.

It was a great conference again this year and we’re happy that Adobe continues to support organizations working to make more opportunities available for women in our industry.

Disha Agarwal
Product Security Manager

Jeanette Azevedo
Security Engagement Specialist

Adobe Security Team Members Win Recent CTF Competition

Kriti and Abhiruchi from our corporate security team in Noida, India, were crowned the winners of the recent Winja Capture the Flag (CTF) competition hosted at the NullCon Goa security conference. Twelve (12) teams competed in this year’s contest. We would like to congratulate Kriti and Abhiruchi on their win. Adobe is an ongoing sponsor of the Nullcon conference. This competition was created by women to encourage their peers to enter the field of cybersecurity. It is a complete set of simulated web application security hacking challenges. Each challenge is separated into small tasks that can be solved individually by the competitors on each team. Each team works through the timed two (2) hour duration of the event in an attempt to attack and defend the computers and networks using prescribed tools and network structures.

Adobe is a proud supporter of events and activities encouraging women to pursue careers in cybersecurity. We are also sponsoring the upcoming Women in Cybersecurity conference March 31st to April 2nd in Dallas, Texas. Members of our security team will be there at the conference. If you are attending, please take the time to meet and network with them.

David Lenoe
Director, Product Security

New Security Framework for Amazon Web Services Released

The Center for Internet Security, of which Adobe is a corporate supporter, recently released their “Amazon Web Services Foundations Benchmark.” This document provides prescriptive guidance for configuring security options for a subset of Amazon Web Services with an emphasis on foundational, testable, and architecture agnostic settings. It is designed to provide baseline suggestions for ensuring more secure deployments of applications that utilize Amazon Web Services. Our own Cindy Spiess, web security researcher for our cloud services, is a contributor to the current version of this framework. Adobe is a major user of Amazon Web Services and efforts like this further our goals of educating the broader community on security best practices. You can download the framework document directly from the Center for Internet Security.

Liz McQuarrie
Principal Scientist & Director, Cloud Security Operations

Applying the SANS Cybersecurity Engineering Graduate Certification to Adobe’s Secure Product Lifecycle (part 2 of 2)

In the first part of this series I discussed how the SANS Cybersecurity Engineering Graduate Certificate helped me analyze the Heartbleed vulnerability and how it applied to static code analysis. This resulted in the paper “The Role of Static Analysis in Heartbleed.” Now I’ll focus on what was learned in the final part of the program and how we can apply this to our efforts here at Adobe.

The course on “Hacking Techniques and Incident Response” taught some of the techniques that attackers use to compromise systems. It also provided an overview of  the forensic tools that are used to detect those attacks. In addition to the labs, students were given access to the NetWars platform. NetWars is a simulated cyber range where you can practice offensive and defensive cybersecurity techniques against live systems. Practical hands on experience is key when doing incident response and the platform facilitates this learning. The highlight of the week was being able to use the NetWars system in a competition format solving real-world problems.

The final course on “Advanced Network Intrusion Detection and Analysis” started with “bit bootcamp” and explored the lower levels of the networking stack. We were taught how to decode packets and truly understand all of the network communication that is occurring on a given computing device. The second part of the class was focused on learning how to deploy network utilities like Snort, Bro, and Silk for network analysis. The SANS at Night talks were also very useful as they allowed me to hear about recent vulnerabilities faced by the security industry as a whole and discuss potential solutions.

There were numerous techniques taught in both of these classes and through Netwars that were directly relevant to working on Adobe Photoshop and our related Creative Cloud services. While I was familiar with Charles and BurpSuite to do web session debugging, these classes allowed me to truly understand how to use deeper features of these tools to determine types of traffic being sent across the network. They also helped me understand when and how to carve the network traffic in order to focus on a smaller set of packets which can then be manually inspected in tools like Wireshark. These tools are great for testing for accidental information disclosure as well as proper authorization checks on cookies.

Taking the techniques learned during professional development and applying them directly to a set of problems on the job is the best you can hope for in any training.  The tools and information  from these classes has already helped in our ongoing efforts to make Photoshop a more secure product.

Jeff Sass
Engineering Manager, Photoshop

Better Security Through Automation

Automation Strategies

“Automate all the things!” is a popular meme in the cloud community. Many of today’s talks at security conferences discuss the latest, sophisticated automation tool developed by a particular organization. However, adding “automation” to a project does not magically make things better by itself. Any idea can be automated; including the bad ones. For instance, delivering false positives “at scale” is not going to help your teams. This blog will discuss some of the projects that we are currently working on and the reasoning behind their goals.

Computer science has been focused on automation since its inception. The advent of the cloud only frees our ideas from being resource bound by hardware. However, that doesn’t necessarily mean that automation must take up 100 scalable machines. Sometimes simple automation projects can have large impacts. Within Adobe, we have several types of automation projects underway to help us with security. The goals range from business-level dashboards and compliance projects to low level security testing projects.


Defining goals

One large project that we are currently building is a security automation framework focused on security assertions. When you run a traditional web security scanner against a site, it will try to tell you everything about everything on the site. In order to do that effectively, you have to do a lot of pre-configuration (authentication, excluded directories, etc.). Working with Mohit Kalra, the Sr, Security Manager for the ASSET security team, we experimented with the idea of security assertions. Basically, could a scanner answer one true/false question about the site with a high degree of accuracy? Then we would ask that one simple question across all of our properties in order to get a meaningful measurement.

For instance, let’s compare the following two possible automation goals for combating XSS:

(a) Traditional automation: Give me the location of every XSS vulnerability for this site.

(b) Security assertion: Does the site return a Content Security Policy (CSP) header?

A web application testing tool like ZAP can be used to automate either goal. Both of these tests can be conducted across all of your properties for testing at scale. Which goal you choose will decide the direction of your project:

Effort to implement:

(a) Potentially requires effort towards tuning and configuration with a robust scanner in order to get solid results. There is a potential risk to the tested environment (excessive DB entries, high traffic, etc.)

(b) A straight forward measurement with a simple scanner or script. There is a low risk to the tested environment.

Summarizing the result for management:

(a) This approach provides a complex measurement of risk that can involve several variables (reflected vs. persistent, potential value of the site, cookie strategy, etc.). The risk that is measured is a point-in-time assessment since new XSS bugs might be introduced later with new code.

(b) This approach provides a simple measurement of best practice adoption across the organization. A risk measurement can be inferred but it is not absolute. If CSP adoption is already high, then more fine grained tests targeting individual rules will be necessary. However, if CSP adoption is still in the early stages, then just measuring who has started the adoption process can be useful.

Developer interpretation of the result:

(a) Development teams will think in terms of immediate bugs filed.

(b) Development teams will focus on the long term goal of defining a basic CSP.

Both (a) and (b) have merits depending on the needs of the organization. The traditional strategy (a) can give you very specific data about how prevalent XSS bugs are across the organization. However, tuning the tools to effectively find and report all that data is a significant time investment. The security assertion strategy (b) focuses more on long term XSS mitigations by measuring CSP adoption within the organization. The test is simpler to implement with less risk to the target environments. Tackling smaller automation projects has the added value of providing experience that may be necessary when designing larger automation projects.

Which goal is a higher priority will depend on your organization’s current needs. We found that, in playing with the true/false approach of security assertions, we focused more of our energy on what data was necessary versus just what data was possible. In addition, since security assertions are assumed to be simple tests, we focused more of our design efforts on perfecting the architecture of scalable testing environment rather than the idiosyncrasies of the tools that the environment would be running. Many automation projects try to achieve depth and breadth at the same time by running complex tools at scale. We decided to take an intermediate step by using security assertions to focus on breadth first and then to layer on depth as we proceed.


Focused automation within continual deployment

Creating automation environments to scan entire organizations can be a long term project. Smaller automation projects can often provide quick wins and valuable experience on building automation. For instance, continuous build systems are often a single chokepoint through which a large portion of your cloud must pass before deployment. Many of today’s continuous build environments allow for extensions that can be used to automate processes.

As an example, PCI requires that code check-ins are reviewed. Verifying this process is followed consistently requires significant human labor. One of our Creative Cloud security champions, Jed Glazner, developed a Jenkins plugin which can verify each check-in was reviewed. The plugin monitors the specified branch and ensures that all commits belong to a pull request, and that the pull requests were not self merged. This allows for daily, automatic verification of the process for compliance.

Jed worked on a similar project where he created a Maven plug-in that lists all third-party Java libraries and their versions within the application. The plugin would then upload that information to our third-party library tracker so that we can immediately identify libraries that need updates. Since the plug-in was integrated into the Maven build system, the data provided to the third-party library tracker was always based on the latest nightly build and it was always a complete list.


Your organization will eventually build, buy or borrow a large scale automation tool that scales out the enumeration of immediate risk issues. However, before you jump head first into trying to build a robust scanning environment from scratch, be sure to first identify what core questions you need the tools to answer in order to support your program. You might find that starting with smaller automation tasks that track long term objectives or operational best practices can be just as useful to an organization. Deploying these smaller projects can also provide experience that can help your plans for larger automation projects.


Peleus Uhley
Principal Scientist