Adobe Installation and Licensing

Tonight’s post is written by the Sr. Engineering Manager leading the development team for the deployment (installer/updater) and provisioning (licensing) technology, Nagendra Bangalore. Nagendra is a long-time installer guru at Adobe who became responsible for the installer technology for Creative Suite 4. Having successfully addressed many of the goals for Creative Suite 4 installers, namely fixing most of the retail customer workflows, Nagendra now has responsibility for the update technology as well as the provisioning technology for future versions of the Creative Suite. –Eric Wilde

A slight bit of history and maybe a bit of overlap to what Eric wrote in his previous blog; so please bear with me on this one. Note that this article focuses on retail installers. Enterprise deployment considerations are significantly different and will be handled in future posts.

A long time ago when Adobe was not releasing the Creative Suites, life in installer land was very pretty and relatively straightforward. Each team that released a product had a Windows & Mac engineer releasing native installers. The installers were industry standard and worked well and then the Creative Suite came along and changed everything.

Lets take a look at how the first Creative Suite was accomplished.

In the first Creative Suite, all individual products were using InstallShield or MindVision to create non-MSI installers on Windows and Vise Packages on Mac. A team was put together and wrote installers for the Creative Suite that assembled and installed the individual installers in silent mode. At that time there were many engineers each working on individual applications. These engineers working on the individual product installers and the Creative Suite installers ensured six products (Win & Mac) were assembled into a single Creative Suite installer (Win & Mac). The efficiency in original Creative Suite installer development was two engineers per individual product.

It worked great for the Creative Suite as it was custom assembled; but, for every change required by the Creative Suite each product’s installer engineers had to change their code to accommodate the requirements of the Creative Suite. This became a coordination nightmare. We had a staggering number of install order incompatibilities, and we routinely had individual product installers stomping on each other. There was a lot of testing that went into Creative Suite installers that didn't go into general co-existence issues, and in fact a lot of the fixes in suite installers depended on the particular order of installation of the products in the suite. It was, however, still manageable even though the teams working on the individual product installers were in three different geographies (San Jose, Seattle, and Hamburg). One thing was clear going in to Creative Suite 2, we had solve the issue of file sets that were common across many individual products in the Creative Suite.

A look at how the Creative Suite 2 was done.

As the team started on Creative Suite 2, it was decided to standardize on MSI (InstallShield) and Apple Packages for Mac. Standard templates were created, all of the Creative Suite requirements for the installer functionality were moved to scripts and DLLs, (written once but used by everyone) and the Creative Suite Installer team worked on another custom installer to bring them all together.

Since we had a very obvious issue of files being stomped all over the place in the first Creative Suite, we solved the issue by introducing the common files installer (aka shared component installers.) See the note below on MSI and shared compoqnents. Another problem became the co-ordination among installer engineering to pick-up the right version of the common code implemented in DLLs and scripts. Any mismatch meant some Creative Suite functionality was broken. The overwhelming complexity brought in by Creative Suite 2 was a major indicator that for Creative Suite 3 installers would have to use a different technology.

A look at how the Creative Suite 3 was done.

As the team formed in Creative Suite 3 to look at this problem, they had a bigger problem at hand. The acquisition of Macromedia brought many very different products into the Creative Suite. These products plus the integration of the video and audio products in CS3 resulted in much more complexity from the sheer size of the Creative Suite. No longer could we enjoy the luxury of having 1 installer per product. They had to solve this by coming up with a solution that created installers that shared code, could be used across six different Creative Suite offerings, create installers that could simply plug into the Creative Suite architecture, solve package management problems, implement new functionality and requirements, all while keeping the overall installers easy to create.

Phew, there's a bit of history on how the Creative Suites were accomplished. Now lets talk a bit about the technology itself,

Folks always asked me, "Why do you have to make it so complex!! Microsoft Office is a suite of applications and they have MSI-based installers. Why can't Adobe do this?"

Okay, Let’s investigate MSI a bit.

MSI is valuable installation technology for delivering individual products and it does a great job handling all of the installation needs (performance can be a bit more desirable, but hey, if it is reliable then performance can take a back seat for now). But where it breaks down is handling a suite of applications as well as shared components that are independently updatable.

Before we go any further, let me take a brief moment to define what I mean by "shared component" or, in Adobe nomenclature, “Shared Technology Installer”. Many of the Adobe products use certain functionality that is shared between more than one product. These functionalities are called shared components. Shared components should have their own installers and should be managed in such a way that install/uninstall of a particular software that uses the shared component does not break any other software using that shared component. This involves reference counting of that shared component on the end-user’s machine in some way. Examples of shared components are Adobe Camera Raw, Adobe Extension Manager, Adobe Bridge, Dynamic Link, CMaps, TypeSupport and the list goes on. Organizationally, these components are owned by different teams who deliver the shared component to each product team. Shared components are delivered as standalone installers for integration purposes.

By now an MSI expert is thinking “Why installers? Why doesn't Adobe release MSM for the teams to integrate?” Good Question.

Here is why we could not release MSM for shared components:

a) Integration of MSM into an MSI installer means literal addition of these files into the product's MSI and Cab file. If two such products are in a Creative Suite that effectively means you just doubled the space of the media and also means you increased the size of the electronic software download. Now imagine what the Master Collection would look like with all of those duplicates, many of which are in four or more products.
b) Once the MSM's become part of the MSI then those files are not part of the main installers. Once this happen, creating a patch/release/upgrade of the shared component would be a nightmare and in some cases impossible due to the constraints imposed by MSI on upgrades and patches.
c) Now imagine you are on the verge of shipping the Creative Suite. There is a bug in the shared component and a fix is released. If the fix is in an MSM file then every product that included the MSM must rebuild and recertify. This can very readily lead to a slip of the shipment date.

I am not saying that releasing the shared components as installers is the only answer to this; but, it did solve some of our major requirements:

a) Since we released shared components as installers, these installers could now be plugged in to the product installer. The integration became easy. When a Creative Suite was created we needed to carry one copy of the shared component installer.
b) Creation of patches for these shared components is not a problem because they are treated as a product itself.

Once we decided to go with installers it introduced a host of other issues that we had to solve:

a) Reference counting (one more reason why MSI’s can’t be used for the Creative Suites or individual products in the case of Adobe products) - Since MSI only does component ref-count we had to now keep track of the installer ref-count. No one does this currently. We had to come up with a way to handle this.
b) Versioning issues – We now had to make sure that the installers didn’t install the wrong or older version of a shared component.

Apple Packages and Creative Suite Installers

When we first started investigating what installer technology to use for Mac Installers, it was obvious that we should be using Apple PKG and MPKG formats.
In Creative Suite 2, all the individual product installers did use PKG and MPKG formats. As we dug in to Apple PackageMaker we encountered the following issues that made us look at other options.

a) No script can be executed before the UI starts (so that we can check for OS, conflicting process, incompatible versions of application installation, check for pre-release versions, etc).
b) We could not show a serial number dialog as part of the installer workflow.
c) Decision from a script directly affects the selection of items in the UI.
d) Add custom UI and custom messages and insert them into the different workflows, (install, re-install and uninstall workflows)
e) Selection/De-Selection of one package in the options screen must affect other items and prevent users from making an incompatible selection.
f) Prompt for disk-swap when an installer switches from one media to another.
g) Ease of installer creation when it comes to shared technologies, individual products and their integration into Creative Suites.

Here is a simple table that illustrates where MSI and Apple PKG format do or do not meet our needs.

In short, our determination was that the native platform installers would not allow us to get the reliability, flexibility and engineering efficiencies of shared component install, package management, and the UI flexibility we needed for our suite installers. We realize that the lack of native installers for our products has raised questions for all of our customers, and presents real problems for enterprise deployment situations. As to the questions: we hope this post has answered many of them, and we look forward to your follow-ups.

We also realize that these factors affect enterprise customers the most. We are working to make this much better. In future blog posts we will address this issue and talk to you all about the options we are thinking about to solve the pain felt by our enterprise customers and we sincerely look forward to your feedback.

Thanks,
Nagendra Bangalore
Sr, Engineering Manager
Nagendra@adobe.com

Posted by Eric Wilde at 11:44 PM | Permalink | Comments (30)

Note: Our current plan is to post a new entry here every week and then follow up that post many times per day in the comments section. This week’s post is the last of the “Introductory” posts. Next week’s topic is scheduled to be why native OS installers (MSI/PKG) don’t satisfy our needs. I’m really looking forward to next week’s discussion.

The Creative Suite has struggled for a long time with its installers. There have been complaints about installation failures or inability to deploy in an enterprise environment since CS1. In CS3 Adobe moved to its own installer technology. In a future post we’ll go through the details of why we use our own installer technology, including why MSI/PKG formats are unacceptable. For this post, I’d like to talk a little bit about what happened to the installers over the last four Creative Suite releases and the opportunities currently available to us.

CS1 was really a collection of traditional Adobe applications each with its own unique install requirements. The applications grew over literally decades into the mature Adobe applications we sell today. As such, the install requirements were diverse and sometimes not totally rational. The shared technologies in CS1 were minimal. The primary shared technology that made the aggregate applications into a “suite” was the Version Cue feature set. Given the diverse install requirements and limited shared components, CS1 continued to have each application create its own unique installer and aggregated these into a single suite installer.

CS2 was quite similar to CS1 in that the number of applications didn’t dramatically change. The install requirements of the applications were relatively constant compared to CS1. The number of shared components grew only slightly, with the introduction of the Bridge application. Even this limited set of shared components and only-slightly changing product line stretched the abilities of the platform install technologies. Creating the CS2 installers was a very costly and time consuming process. Something needed to change.

With CS3 and the inclusion of former Macromedia applications, it was very clear that the platform install technologies would not satisfy the business requirements for Creative Suite installers. So Adobe created its own install technology. For CS3, the installer was all new code. It was introduced to provide the kind of package management and flexibility needed by Adobe’s increasingly diverse install requirements. The move to our own installers was extremely challenging both for Adobe and Adobe’s customers. We’re really not in the business of creating installers and would prefer not to do so; but, given the need to rapidly integrate the former Macromedia applications with all their different installer requirements, the now significantly larger set of shared components and the experience of creating CS2 installers, it was clear that we could no longer use the platform technologies.

As I said, the move to our own installers was extremely painful for Adobe and Adobe’s customers. One could easily argue that it was a step backward. Certainly for our enterprise customers it set up really difficult road blocks to deploying the Creative Suite. So when we started work for CS4 we put lots of focus on the install technology to try to “get it right”, at least for the retail customers. The personnel also changed so that the technology was now managed by the same group responsible for the Creative Suite itself (me) – essentially, we wanted the technology team and its management to have skin in the game.

Our goal for CS4 was to make the installers work for the retail customer. It was a difficult choice because we knew the enterprise customers would continue to run across the same road blocks as with CS3. Throughout the CS3 cycle we gathered input from customer care about what problems our retail users were having and tried to ensure the CS4 installers addressed those problems. We focused on solidifying the install technology so that it worked as it was supposed to work. We operated within the limitations of the existing technology to provide a UI as stable as we could.

Lots of hard work later, we shipped CS4 and did see dramatic improvements in the installer for retail customers. Customer care calls about failed installs dropped significantly. There’s no doubt that problems still exist. The enterprise customers are still left in the lurch. The UI still has lots of room for improvement. But for retail customers, it was definitely a step in the right direction.

Now we have some opportunities ahead of us. The personnel changes that happened for CS4 allowed us to train the engineers on the technology and the problems our customers faced for installation. So the engineers can now make architectural changes to the install technology that can eliminate some of the road blocks for enterprise customers. We also have the opportunity to migrate the UI technology away from DHTML (used for CS3 and CS4) into an embedded Flash interpreter. This will allow us to have much more flexibility and stability in the UI; which will, in turn, allow us to use a better experience design. We continue to receive feedback from our customers about the problems they face when installing and will continue to tackle the most egregious problems that surface, thereby making the installers yet more stable and drive down customer care calls even further. I’m sure it still won’t be perfect; but, it had better be much better than CS3 and CS4!

We’d like to make further progress on the aforementioned architectural changes before going into the details on this blog. I’d hate to advertise a change and then have to renege on that promise a month or two later. As we do make progress we’ll advertise the changes being made so that you’re all well aware of what to expect in the next version before it ships. There may also be an opportunity to provide some changes for CS4 prior to shipping the next version, at least for changes that don’t involve massive rework or break United States accounting rules.

So that’s where we are today. We continue to gather information from World Wide Customer Care (tech support), strategic partners and users such as readers of this blog. With this information we can look at what is failing and see how we might be able to fix it. Some of those fixes can come as an update or workaround in CS4. Some of those fixes will have to come in future versions of the Creative Suite.

The comments from the first post were very encouraging. You all seem to approach this with a spirit of cooperation and are educating us on exactly what is causing you problems. As we come up with solutions and verify that those solutions are actually feasible we’ll try to let you know our plans. That way you’ll know what to expect in the future and can hopefully confirm that our plans really will satisfy most of your needs – or correct us where we run astray.

Thanks,
--Eric Wilde
Sr. Engineering Manger
ewilde@adobe.com

Posted by Eric Wilde at 9:06 PM | Permalink | Comments (26)

Welcome!

Let me start by introducing myself. I manage the engineering team responsible for the Creative Suite and its installer technology for CS4 (plus a few other odds and ends). For future versions of the Creative Suite I’ll also be responsible for the licensing and update technologies. I’ve been working at Adobe for over sixteen years as an engineer and, later, as an engineering manager. In the past I’ve done hardware design, core PostScript implementations, Acrobat Distiller, Acrobat plugins and various pieces of technology shared across Adobe products.

This blog is intended as a two-way communication about the “out of box experience” that users of Adobe software have when they purchase our products. Through posts on this blog we intend to communicate what is happening with the deployment (installer/updater) and provisioning (licensing) aspects of Adobe software. The scope includes everything from the time of purchase through download, installation, licensing and later updates of Adobe applications. Along the way we will provide some historical perspective for why things are this way today and where we’re going tomorrow.

In addition, we want to hear back from the Adobe community about your out of box experience. We’re under no delusion that the world of installers, updaters and licensing is just peachy. Some of you experience problems installing or updating Adobe software, particularly in enterprise environments. Likewise we have a few problems with the licensing technology for some customers. We want to hear about the problems you face and, hopefully, help to solve them. In addition, the information we get from you all through this blog will help us prioritize your needs so that we address the most important issues first, so please do respond in comments to the posts. We’ll do our best to keep up with you.

Most of the posts on this blog will be from the OOBE (“Out of Box Experience”) group for the Creative Suite products. Folks from various roles within the group will post, from engineering and program management through product management and customer care. We hope to bring in some individuals at Adobe working on similar aspects of non-Creative Suite products for guest posts as well.

Lastly, I want to thank a number of individuals in the Adobe user community for suggesting that the installer and licensing group have their own blog. It is through constructive and pointed feedback from such users that we’ve been able to address some of the more painful issues our users face. Hopefully this blog serves its purpose to educate both sides about what is happening so that Adobe’s installation and licensing experience continues to improve.

Thanks,
--Eric Wilde
Sr. Engineering Manager
ewilde@adobe.com

Posted by Eric Wilde at 10:25 PM | Permalink | Comments (125)