The Engineering Behind Flash Player 10.1

Today represents an exciting milestone for the Flash Platform as we officially launch the public beta release of Flash Player 10.1 on Android-based devices. The Beta release is now available and as soon as Google releases their update to the Android OS, code-named “FroYo”, users will be able to get the new Flash Player installed over the air from the Android Market. The public beta release is your opportunity to try out our dramatically updated browser-based runtime with a deep focus on performance, battery life, memory management, user experience on mobile devices, and other mobile-specific features. With the pervasive use of Flash Player content on the web today, users will be able to experience the full extent of the web that Flash enables including video, games, animations, Rich Internet Apps, data presentations and visualizations, commerce, music, and so much more. It’s great to see that our extensive technical collaboration on this with our partners is now becoming a reality. To give you some perspective on the magnitude of this release, let’s look at some of the performance improvements and optimizations we made for Flash Player 10.1.

Power and Battery Optimizations

Mobile platforms are carefully tuned to deliver a broad range of capabilities while minimizing battery consumption. Flash Player 10.1 was developed to seamlessly integrate into this environment. To deliver on this objective, we added a new capability called instance management to intelligently load and play back Flash content only after it comes within view on the web page. This capability also allows us to work in conjunction with the browser to ensure the web page is loaded as quickly as possible.

A related capability we added is called pause and resume. Flash Player will automatically pause the content that is running when the browser is hidden from view or the current tab is placed in the background. Not only does this conserve power, but it allows the content to instantly hibernate should an important trigger event occur, such as an incoming phone call. Once the user returns to their browser session, the HTML page and the Flash content are returned to their exact point prior to the trigger event.

Another example of a power-saving capability we have developed is called timer throttling. Flash Player makes use of timers to control the speed of content execution. When the screen display dims, Flash Player utilizes this signal to automatically throttle back the speed at which the content is executing while the device is in this power-saving mode. In the case where the content is playing back audio or video, Flash Player will not throttle it back to ensure a seamless media playback experience.

These new capabilities in Flash Player 10.1 offer some insight into the priority we placed on conserving power and staying true to the goal of maximizing battery life on mobile devices.

Maximizing Performance

The architecture of mobile platforms is very different from desktops, laptops, and netbooks. To take advantage of these highly integrated hardware environments, we took a very comprehensive look at how Flash Player uses the CPU, GPU, memory, and storage. The Flash Player team, with engineering cooperation across our Open Screen Project partners, meticulously optimized the machine instructions used in our virtual machine, rendering engine, and media codecs to run efficiently on mobile hardware.

In the case of the ActionScript 3 virtual machine, we updated our just-in-time compiler to emit native ARM instructions to maximize the execution speed of applications on these platforms. With a significant portion of the Flash content on the web having been written in ActionScript 1 and 2, we also invested in increasing the execution speed of this content.

In addition, Flash Player will take advantage of the purpose-built hardware on the mobile device including the media decoders to optimize the playback of content such as H.264 video and AAC audio streams. This specialized hardware can often perform a specific task more efficiently than the CPU. Not only does this result in higher fidelity playback, but it can yield a nice power savings.

The investments we made in execution speed will be apparent in a broad range of content, from Flash applications to games to video, and everything in between. We took an extensive look at the performance characteristics of Flash Player in many different scenarios and drove considerable improvements in the execution speed. As a result of this work, most Flash content on the web today will execute more efficiently with no changes required to receive these benefits.

Conserving Memory

Mobile platforms have considerably less memory than most personal computers and Flash Player 10.1 includes numerous optimizations to minimize the amount of memory required for content and applications. We have added automatic compression of media in memory to matchthe typically smaller screen size and color depth of a mobile device. We have also enhanced the memory garbage collection system to work more effectively, particularly in low memory situations. We made changes to more aggressively release temporary buffers and media caches for images and audio data. These changes have translated into some dramatic improvements. In some cases, you’ll see content that now automatically consumes 50% less memory with Flash Player 10.1 when compared to our previous release.

Another important enhancement we made relates to streaming media. In some cases, content such as video is buffered in memory to allow the user to quickly seek forward or backward. In the case of mobile platforms, we added a new circular buffering capability that allows Flash Player to constrain the size of the media buffer and recycle this memory as the content plays. This new capability strikes a balance between allowing the user to navigate within sequential media, like video, and the amount of memory that content can consume.

The Flash Player team also did a lot of development around low memory support, as this is a common situation on mobile devices. To protect against content that requires too much memory for the device, we added a new system that detects when memory is running low so defensive actions can be taken to prevent a crash. In this case, the Flash Player now has a set of heuristics to shut down content when resources are running low or depleted and gracefully exit the content. This work will also benefit personal computer users that have many tabs open in their browser such that their PC is operating in a low memory situation.

Usability and Interactivity

The last topic I’ll cover is usability. In this case, it’s critically important that Flash content on a smart phone both behave consistently with the HTML page and enable rich interactivity with media in Flash. For example, as a web page can be zoomed using a multi-touch gesture by the user, the Flash content needs to enable that to happen seamlessly. If the user rotates the phone to switch into landscape mode, the Flash content responds as you’d expect. If the user double-taps to zoom in or out, the Flash content responds in a consistent, intuitive way. We have enhanced this interaction to enable a new smart-zoom capability, bringing zoomed Flash content to the best width/height dimension for the device to enable easy viewing of video and other content in Flash. We have also added the ability for content to specify that it should automatically go into full screen mode on first interaction in order to enable easy full screen interactivity with games. This new mode is indicated to the user with a message about how to exit full screen mode on their particular device.
The Flash Player team added a number of new capabilities that are essential to smart phones. Text fields in Flash will automatically take advantage of virtual or soft keyboards for text entry when a physical input device is not available. In this case, the text field is intelligently repositioned on the screen to facilitate text input.

In addition, Flash Player exposes some of the unique capabilities of mobile platforms, including access to the accelerometer. This will allow the Flash development community to create content that leverages some of the unique capabilities. We’ve got more work to do here, but we’re off to a good start and look forward to exposing new capabilities in the future, such as geolocation support.

These examples demonstrate the importance of ensuring that Flash content on a mobile device offers a predictable and intuitive experience to the end-user. To help ensure that we got the experience right, we conducted multiple rounds of usability testing to try out new ideas and improve the experience based on the feedback we received. Our goal was to ensure that existing content simply works as expected and ultimately allow smart phone users to experience the full glory of the web.

Looking Ahead

As we look forward, it’s important to note that this is just the start of our work to bring the full Flash Player to mobile devices. We will continue to improve on this effort as we support a broad range of smart phones, tablets, televisions, and other devices. Of course, a number of these optimizations and improvements will also directly benefit Flash Player on desktop and netbook platforms. At the same time, we’ve seen content publishers like Sony Pictures, Nickelodeon, Miniclip, Warner Brothers, Kongregate, South Park Studios and others already optimizing their sites to deliver the best possible experience within the context of smaller screens. Developers, who are looking for tips about optimizing Flash content for mobile devices should check out this document.

Smartphones in market today that will be able to run Flash Player 10.1 include the Nexus One, DROID by Motorola, HTC Evo, HTC Incredible, HTC Desire, Samsung Galaxy S, Motorola Milestone and others, but will depend on FroYo availability for a given device. Upgrade mechanisms and timing will vary by device and device manufacturer. To make it as easy as possible to install Flash Player, we are enabling over-the-air downloads which include content-initiated downloads, system software updates, and on-device app catalogs. Over time, a number of new phones are planned to ship with Flash Player 10.1 preinstalled. We’re thrilled to take the first step today and can’t wait to see how millions of Flash developers will innovate for mobile devices.

I hope you’re as excited about the future and the opportunities ahead as we are.

Paul Betlem
Sr. Director, Flash Player Engineering
PS: If you are a developer and want to start developing apps for Android today, check out this post from the AIR team and read all about the AIR SDK developer release we made available for Android today.

30 Responses to The Engineering Behind Flash Player 10.1

  1. Tim says:

    I was looking for this for such a long time: flash on a smart phone. Now, I’m sure android will conquer the mobile phone market. Now, I am wondering what the release date of FroYo for Milestone is.

  2. ben farrell says:

    The power saving features sound great, and I’m glad you folks are working hard to get this into mobile. I’m wondering though if this would screw up any existing applications like sound. If the Flash player pauses while I scroll down my MySpace page to look at a profile for example while I still hear the song I was playing? Or is this type of thing only used to pause the framerate?
    Even if it did just pause the framerate, I’ve heard many times over to use an enterframe instead of excessive timer usage to not cause player hiccups. If you use an enterframe as a timer/heartbeat, does that get taken away from us if our content is off screen?
    Thanks for the hard work, whatever the answer is!

  3. Paul Puri says:

    Thank you. The full web experience. Magical ;^)

  4. JTtheGeek says:

    Congratulations Adobe, I have never seen such a well written writeup with such emphasis on user experience above all else from the flash player team. This release sounds amazing, and I hope the incredible emphasis on user experience remains tantamount over adding new features. As much as flash 10.1 is getting all the press right now, I’m feeling that Air 2 is where it is really at. Finally the ability to build cross platform mobile applications in the greatest platform in history. Well done.

  5. J T says:

    When will you release it for WebOS?

  6. Nick says:

    Fantastic! I cant wait to be able to view all the Flash enabled games and web sites on my mobile device.
    Its a shame iFad users wont be able to experience the full web on their phones or mobile devices.

  7. Rob McKeown says:

    I am counting the days until my AT&T (iPhone) contract to be up

  8. Matthew Fabb says:

    ben farrell, look up Flash Player Engineer, Tinic Uro’s blog for more details on how Flash Player 10.1 works in pausing content. However, the short answer is that content with audio or video playing will not be paused.

  9. RazorX says:

    HTML5 doesn’t stand a chance against this level of engineering and innovation. The mobile world will never be the same. Truly awesome work!

  10. Cardin says:

    I’m impressed at the effort you guys have put into maximizing Flash performance on handhelds. Adobe FTW!!!

  11. JohnB says:

    FroYo only? WHY!!?

  12. PG says:

    Will you consider releasing the Flash player for Android outside the Market?
    While Android is an open source system, Market isn’t part of that. It would benefit us hobbyist developers using Android on unofficial hardware if we could still use Flash.

  13. Paul Betlem says:

    FroYo contains the required updates to Android that allow the browser to support the Flash Player. Unfortunately, previous version of Android do not have these mandatory changes and as a result, cannot host the Flash Player.

  14. doh says:

    With your recent announcement of support for WebM will Vorbis become a first class audio codec in Flash alongside AAC and MP3?

  15. Mimmo says:

    It is a long time I hear this: Flash is coming on mobile. Ok, fine.
    When is the final release?

  16. HELPS TO KNOW MUCH ABOUT FLASH PLAYER

  17. Paul Vlasveld says:

    I just read a review at http://www.Forbes.com that does not give a good review about Flash:
    Flash For Smartphones Is Finally Here, And It Is Terrible
    How would you respond to this review?

  18. Paul Betlem says:

    The article is one person’s opinion that doesn’t match the vast majority of positive feedback we’ve seen with the Beta. Unfortunately, the author doesn’t seem to have tried Flash Player on Android, but is only providing their interpretation of someone else’s perspectives.
    That said, a Beta release is all about listening to and responding to constructive feedback from the community and with that in mind, there is value in all feedback, both positive and negative.

  19. Paul Betlem says:

    I’m not aware of any plans to do so, but this is an interesting suggestion. I’ll explore it further.

  20. Paul Betlem says:

    Thanks! We’ll also share details on the related work we did for netbooks/latops/desktops shortly.

  21. Paul Betlem says:

    Unfortunately, I cannot provide the timing on behalf of Palm. Their public statement is here:
    http://investor.palm.com/releasedetail.cfm?releaseid=436243

  22. D'Sjonaun says:

    But can Adobe say that they have everything done on their part as far as Flash 10.1 for WebOS, because it was shown that Flash was working flawlessly on WebOS in the beginning of the year.

  23. ben says:

    Hi Adobe.
    It seems as if apple gets more and more momentum on the adobe flash plattform.
    iam a 6 years full time flash / flex developer and get my living out of your technology.
    It will not matter, how superior or imature the flash player will behave on different plattforms, as long as the user experience on an iphone/ipad is way ahead of the user experience on an android / fp10.1 built.
    today i spoke with 3 persons who threw their android/fp 10.1 phone out of the window because neither they nor their girlfriends could handle it.
    bottomline, they will by an iphone and push the momentum towards apple.

  24. trev says:

    I have a palm pre and have been waiting for flash 10.1 since October 2009. When are we going to get it? Have you guys heard of the World Cup, its a football competition with fanatical fans who would love to watch the footy on their mobile. This would be one in the eye for iphone. As it is all the delays are a huge own goal, pull ur finger out. You have a tech advantage use it or sad to say you will lose it.

  25. Palmdoc says:

    Yup – still waiting for the webOS version Adobe! C’mon!

  26. bennyT says:

    Can’t wait! Keep up the good work Adobe! Its time all us Flash developers showed Apple what we can do – let the fun begin.

  27. Doug says:

    Great news about 10.1! Are there any new mms.cfg settings? I would like to toggle (disable/enable) H.264 hardware decoding as part of a test suite.

  28. lengsenior says:

    I’m sorry, but there’s something I REALLY REALLY don’t understand. Why did you let the general CEO of Adobe talk with the press about Flash?
    Adobe has a lot of different products (flash, but also photoshop, after effects, illustrator, etc.) And I don’t think a general CEO is the best person to talk with the press about something that’s not-the-easiest-thing in a specific development proces in a specific product, because he isn’t an expert on it.
    And also doesn’t really seem to know much about the matter… And COULDN’T convince the public why the claims that Flash isn’t ‘technically’ capable to run on mobile devices. OR EVEN TRY TO DO THAT!

  29. Hanamel says:

    While I definitely appreciate the thought behind the pause & resume feature and how it conserves energy and CPU cycles, I would like to have the ability to toggle it on and off, at least for systems connected to a power outlet.
    For example, I might want some lengthy animation in a flash game to run in the background while I do something else.The inability to toggle this feature harms my multitasking capability.

  30. kim says:

    For about 2 weeks am trying to download Adobe Flash Player 10.1. Everytime it goes to the download and install page it shows its done but it doesnt move from there and it is still not downloaded.