Two! Four! Six! Eight! Numbers we appreciate!

I write every once in a while about Flash Player statistics, which I hope is interesting and useful for you. As product managers for Flash Player, Justin Everett-Church and I are responsible for managing and interpreting the data related to Flash Player. That data includes the penetration study, as well as player download and installation metrics. So, forgive me for bringing up old news, but I was out on vacation when the Firefox 3 Download Day happened. I was (quite happily) checked out enough to not really hear anything about it until I got back in the office this month. I understand a few Adobe folks blogged about how our Flash Player numbers were also world-record worthy, and a lot of different download numbers often show up in blogs and press without any context of time period. Is the number of daily downloads a useful metric? What do these numbers really say?

Don’t get me wrong — I love data, and download numbers ARE interesting. But, as with any data, you need to figure out what the numbers tell you and why they are important. Adobe tends to talk about the install rate of Flash Player quite often, although it may not be the right number for people to focus on since it doesn’t give you context, and you have to take our word for it. For Firefox, the download number is interesting because they wanted to beat their numbers for the previous release, and to generate buzz for the current release by submitting it for a world record — which was verified by a third party.

It’s interesting to note that for Adobe, the number that is quoted is an “install” and not a “download” number. We’ve never said how many downloads happen a day because it is a ridiculously large number AND we know that it’s not that useful metric because those successful downloads don’t all turn into successful installations. In July 2008, successful downloads averaged about 33 million per day, and successful installs averaged around 18 million per day. That seems like a big drop, but consider that ActiveX was about 80% of our installs that month and when you visit a page that triggers the ActiveX install experience the installer is downloaded to the machine before the security warning dialog appears. The user might say “no thanks” to the security warning dialog, and refreshing the page or visiting another page that requires a newer version may download the installer again.

How do we monitor Flash Player download and installation numbers?

We have an internal dashboard application to track the general “health” of Flash Player downloads and installs, and it can chart the data daily, weekly, monthly, etc. It uses XML feeds of our server log data from Akamai, our current CDN. The dashboard is useful in helping us 1) to understand our traffic so we can try to optimize things like install success rate, and catch problems with our installers or CDN delivery in a matter of days, and 2) get a sense of where our penetration might be in the penetration study in the next wave. The penetration study is only run once a quarter, and it’s hard to wait three months to know where we’re at.

For “Player downloads” we count the attempted and successful downloads for all the player installers we post to the CDN (based on the related HTTP status codes), such as “swflash.cab” and “install_flash_player.exe” (note these numbers aren’t unique.) We also count the number of installs, which is the sum of requests for a small text file that a newly installed player requests the first time it is launched in the browser. That number is used as an estimate – we’ve had releases where the text file request wasn’t implemented or working on certain platforms, and since it is only requested once there are a number of reasons why it might not make it all the way to the server. The install number wasn’t ever intended to be a marketing point, although it was an exciting number for us to talk about when we realized that our install average was 8 million a day shortly after Flash Player 9 first launched. The daily average has gone up since then, most likely due to increasing penetration of broadband and improvements at the CDN. And it can spike depending on whether there is a hot new site that is sweeping the web, or dip if we’ve got a CDN or installer problem.

The point is, I know it can be confusing to hear about numbers like 8, 12, 14 or 25 million daily installs when there isn’t any context in which to interpret the meaning. We don’t yet have an internal “standard” for the install number that we are quoting as a company, such as “average daily installs for the past month” or “average daily installs since the last release”, etc. Not that it really matters. The intent behind stating these VERY BIG numbers is to say something about demand. That is a big reason why we have the penetration study, and have been tracking it since Flash Player 3. I’ve talked in the past about the general aspects of the penetration data we post, and how it may or may not apply when you get down to your particular audience. As with any statistics, you should understand the methodology behind the numbers. But the good thing about that number is it is something you or another third party like Forrester, can independently test or verify — which makes it the more interesting and important number for Flash Player.

6 Responses to Two! Four! Six! Eight! Numbers we appreciate!

  1. Scott Barnes says:

    Interesting read Emmy! :)I’m actually glad someone with a bit of ownership stepped forward and owned this problem from Adobe, so kudos for that! :)Question, I’ve seen your staffers throw the “world-wide” message around whenever we’re in the room etc. What we don’t see is what this number looks like between Consumer (i.e. Web) and behind the firewall. In that the Penetration Study doesn’t seem to clarify the behind the firewall for instance?So, I guess, the question is out of the Fortune 1000 companies, what would be the penetration here and could you clarify the penetration studies methodology in how this is applied?thanks! :)-Scott BarnesMicrosoft.

  2. emmy says:

    Hi Scott,Yes, that is a question that comes up often. From the methodology, “Respondents must be Adults 18+ and have access to an Internet connected computer.The majority of respondents answer from their home computer; however, responses from respondents answering using a work computer are not excluded. A question is included in the survey to track whether the respondent is answering on a home computer or a work computer.”What that means is — while we do know whether someone answered from their work or home computer because the question was asked, the study/sampling isn’t weighted in a way where we could really say “x% of work users have Flash Player.” We don’t ask them if they are at one of the Fortune 1000 companies, and even if we did it wouldn’t be valid to say a percentage based on the way the sampling is done.So the answer to your direct question is that we don’t specifically test the penetration within Fortune 1000 companies in the Millward Brown study. Maybe the Fortune 1000 wants to volunteer that information? ;-). But, you could check the Forrester report to see what their methodology was for enterprises. They surveyed 50K enterprise users. From the article I linked to above:”Forrester’s study examined the browser as well as the desktop environments of the 50,000 users, spread out among 2,300 companies.””Both Flash and Java were nearly ubiquitous. Flash Player version 9 was on 97 per cent of desktops, while Java was on 99.9 per cent of them.”The report is here, but does cost money: http://www.forrester.com/Research/Document/Excerpt/0,7211,45898,00.htmlcheers,Emmy

  3. John Dowdell says:

    For what it’s worth on corporate support levels, back when Chris Swenson was at Macromedia and processing the NPD/MediaMetrix consumer audits, there were a few quarters when corporate accounts were separated from home accounts.The overall penetration rates were essentially the same for business and home, even though corporate decisions are usually “lumpier” than decentralized individual decisions.Whether a particular worker’s computer supports rich-media depends on their particular intranet, and Chris’s testing could have benefited from corroboration, but overall, the penetration rate in enterprise has seemed to match general consumer norms, as that more recent Forrester report confirms.jd/adobe

  4. Scott Barnes says:

    Hi Emmy,I’ve read that study, it doesn’t specify the regions in which the 50,000 derive from and it was more of a pulse check to see how Vista was comparing.The point is, I see the “world-wide” messaging from your folks, and I can’t but help think it’s misleading, in that Flash may have world-wide consumer adoption as per the methodology states, but as to what happens behind the firewall, depending on how liberal folks are in their respective companies is as you may have put it, anyones guess.Another point, is that you specified 8 million per day are installing Flash on average, which to me would be more of a bad thing than good. As if folks have 98% penetration right, than installing 8 million still per day, indicates that there doesn’t seem to be parity with the versioning of Flash and folks are still in the same boat as products like Silverlight “Want to see x content, click here to install”Which furthermore, indicates that folks are willing to install plugins, provided the content offers reward?I also noted this from the same research paper:”Flash is everywhere but don’t overdo it. Flash Player adoption in the enterprise is high, but don’t redevelop an entire user interface with it — business users don’t want to hunt for navigation nor do they crave excitement. By all means, use it for video streaming, but if you need to animate an HTML element or do some asynchronous magic, use a JavaScript library.”Which leaves me thinking that Enterprise / Corporate folks consider Flash to be really a video presentation tool only based off the Researchers findings. Given Flash maybe everywhere, it’s basically not being treated as a Line of Business tool in the enterprise, despite its penetration implication? (Which will be an interesting contrast for Adobe AIR to play in as well).The point is, I think there is a lot of misconception from Adobe on what these statistics mean, and everything points to “yes you have majority, but what are you doing with the majority” being unanswered? Thus it appears to be a “Your stats are bigger than mine” debate, which holds no merit in the end given the above?So, both parties (Silerlight and Flash) are likely to face the same battles and plug-in fatigue on the end user is something we clearly both are constantly having to address and both parties are clearly not immune to this, despite what the above statistics state?-Scott BarnesMicrosoft.

  5. James Ward says:

    Does the Forrester report detail the version / vendor of Java that has 99.9 percent? I’d suspect a high percentage of that 99.9 percent is the MS VM which is very old and known to be incompatible with Sun’s Java.-James

  6. Jeff says:

    You could improve your penetration numbers by releasing a native BSD Flash9/10 player. There are an awful lot of FreeBSD, PCBSD, DesktopBSD,etc. users scratching their heads as to why Solaris which is mainly used for servers, not desktops, has a version and we don’t. The interim solution of using FF3 on Wine to get Flash9 capability isn’t very stable.This is why the web should not adopt proprietary products unless they truly are ubiquitous, IMHO.