Author Archive: rashah

Automating Credential Management

Every enterprise maintains a set of privileged accounts for a variety of use cases. They are essential to creating new builds, configuring application and database servers, and accessing various parts of the infrastructure at run-time. These privileged accounts and passwords can be extremely powerful weapons in the hands of attackers, because they open access to critical systems and the sensitive information that resides on them. Moreover, stealing credentials is often seen as a way for cybercriminals to hide in plain sight since it appears as a legitimate access to the system.

In order to support the scaling of our product development we need to ensure that our environments remain secure while they grow to meet the increasing demands of our customers. For us, the best way to achieve this is by enforcing security at each layer, and relying on automation to maintain security controls regardless of scaling needs. Tooling is an important part of enabling this automation, and password management solutions come to our aid here.  Using a common tool for credential management is one method Adobe uses to help secure our environment. Proper password management helps make deployments more flexible.  We ensure that the access key and API key needed to authenticate to the backup database is not stored on the application server. As a defense-in-depth mechanism, we store the keys in a password manager and pull them at run time when the backup script on the server is executed.  This way we can have the keys in one central location rather than being scattered on individual machines when we scale our application servers. Rotating these credentials becomes easier and we can easily confirm that there are no cached credentials or misconfigured machines in the environment.  We can also maintain a changeable whitelist of the application servers that need to access the password manager, preventing access to the credentials from any IP address that we do not trust.

If an attacker were able to access build machines they could create malicious binaries that would appear to be signed by a legitimate source. This could enable the hacker to distribute malware to unsuspecting victims very easily. We use two major functions of commercially available password managers to help secure our build environment.  We leverage the credential management solution in order to avoid having credentials on any of our build servers. The goal here is similar to the use-case above where we want to keep all keys off the servers, only retrieving them at run-time.  In order to support this, we’ve had to build an extensive library for the client-side components that need to pull credentials.  This library allows us to provision new virtual machines constantly with a secure configuration and a robust communication channel with the credential manager.  Adapting tooling in this way to suit our needs has been a recurring theme in our effort to find solutions to deployment challenges.

Our build environment also uses the remote access functionality provided by password managers, which allows users to open a remote session to the target machine using the password manager as a proxy.  We ensure that this is the only mechanism in which engineers can access machines, and we maintain video recordings of the actions executed on the target machine. This gives us a clear audit trail of who accessed the machine, what they did, and when they logged out.  Also, since we initiate the remote session, none of the users or admins need to know what the actual passwords are since the password manager handles the authentication to the machine.  This prevents passwords from being written down and shared – it also becomes seamless to change them as needed.

Credential management has become a challenge primarily because of the sheer number of passwords and keys out there. Given some of our use-cases we’ve found commercially available password management tools can help make deployments easier in the long-term.  Adobe is a large organization with unique products that have very different platforms – having a central location for dealing with password management can help solve some of the challenges that we face as a services company.  As we look to expand each service, we will continue to adapt our usage of tools like these so that we can help keep our infrastructure safe and provide a more secure experience to all our customers.

Pranjal Jumde and Rajat Shah
ASSET Security Researchers

Adobe is a Sponsor for the Nation’s Largest Student Cyber Security Competition

ASSET team members Karthik Raman, Bronwen Matthews and I recently attended the NYU Poly CSAW IX Cyber Security competition  in Brooklyn, New York. The annual event first took place in 2003 and has since grown from a small, local cyber security competition to a worldwide event. This year, more than 10,000 students from high school to Ph.D level registered to compete in a total of seven CSAW challenges.

Karthik and I contributed four Web challenges to the “Capture the Flag” competition, which were designed to be similar to real-world scenarios hackers face. The challenges were related to commonly found bugs, but required the hacker to deduce the nature of the bug without much feedback from the website. The students responded with a pragmatic approach to the problems, and the competition was won by a team from Carnegie Mellon University. There was also an embedded systems challenge, a forensics challenge and an applied research competition.

Adobe sponsored the “Security Awareness” video challenge, open to high school and college students worldwide. The contest challenged students to develop a consumer-friendly educational video on an important security topic with the theme: “Securing Every Device, Everywhere.” Adobe provided access to the free version of Adobe Creative Cloud for all participants, enabling them to use our latest video production tools. Guest judges from the security teams at Adobe, Microsoft, VMWare, Facebook, and the NSA selected the final winners. The first place winner of the challenge this year was Ethan Bain of the Illinois Mathematics and Science Academy in Aurora, Illinois. You can watch his winning video here.

The first day of the event focused on mobile security, with presentations from Dan Guido, Vincenzo Iozzo and Dino Dai Zovi from Trail of Bits, as well as Mike Arpaia.  Other presenters included:  Collin Mulliner of Northeastern University, Jon Oberheide of DUO Security, and Chris Rohlf of LeafSR.

Ryan Naraine from Kaspersky Lab moderated an interesting panel discussion entitled: “If a Cybercriminal is Determined to Hack You, Can You Do Anything About it?” Panelists included representatives from Kaspersky, Harvard University IT and NYU Poly.

The high school students competed in a challenging, live security quiz, sponsored by DHS. (We played along in the audience. Let’s just say we got most of the answers right.)

It was a fun couple of days. We met some excellent students doing interesting and important work in security. It is reassuring to know that the next wave of security researchers coming out of some of our high schools and colleges are way ahead of the game in cyber security.

Rajat Shah
Security Researcher
Adobe Secure Software Engineering Team (ASSET)