The Importance of Scaling Images on Mobile Devices

When we launched Adobe AIR for Android, I created an application called AIRBench to help us measure the performance of AIR on all the different Android devices out there. Not only did AIRBench tell us which phones run AIR the best, but it also brought to my attention the importance of scaling images before displaying them on mobile devices.

One test we saw consistently fail on Droid devices (only the first Droid — none of the others) was the CameraUI test. The CameraUI test simply lets users take a picture, then places the image on the display list. It’s very simple and straightforward, however we saw numerous failures. After looking into it, we discovered that the issue isn’t actually a bug in AIR, but rather it’s the result of the application running out of memory.

The test uses the Spark Image component to display the image. When the code hands the Image component the file URL of picture that was just taken, two important things happen:

  1. The image is decoded into a bitmap which means it uses much more memory than the compressed JPEG version.
  2. The image is visually scaled down to fit on the screen. I say "visually scaled" because it only looks to be smaller, but the entire uncompressed bitmap is still in memory.

Because the Droid has a pretty high resolution camera (5 megapixels), but a relatively small amount of RAM (256 MB as opposed to the 512 MB of the Droid 2), it’s not hard to get an application to run out of memory by displaying uncompressed bitmaps if there are other applications running at the same time.

Although I’ve never seen this happen on any other device (since most other Android phones that AIR supports have more than 256MB of RAM), it’s still a good idea to really scale your images before displaying them. By "really scale", I mean the following:

  1. Figure out the width and height that gets the image small enough to fit on the screen, but also maintains its aspect ratio.
  2. Draw the data from the Loader to a new BitmapData object and using a Matrix object to actually scale the image down (meaning most of the data is removed).

I did some tests and found that by properly scaling an image before putting it on the display list (in this case, handing it to the Spark Image component), you can easily display images from either the CameraUI or the CameraRoll on the Droid with no issues whatsoever. A little bit of profiling revealed that scaling images properly uses only a few hundred kilobytes as opposed to between 20 and 30 megabytes (depending on the image size).

The sample code below illustrates how to do the following:

  1. Use either the CameraUI (if it’s supported), or a native file browser (for desktop testing) to let the user take a picture or choose an image.
  2. Read the bytes of the image into a ByteArray.
  3. Load the bytes into a Loader.
  4. Figure out the correct scale factor which will allow the image to fit on the screen, but still maintain its aspect ratio.
  5. Create a new BitmapData object with the correct dimensions.
  6. Draw the image into the BitmapData object and scale it down to the correct size.

Ok, enough talk. Here’s the code:

<?xml version="1.0" encoding="utf-8"?>
<s:View xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" title="Home">
  <fx:Script>
    <![CDATA[

      private const MAX_SIZE:uint = 400;

      private function onPickImage(e:MouseEvent):void
      {
        // If we're on a mobile device
        if (CameraUI.isSupported)
        {
          var cameraUI:CameraUI = new CameraUI();
          cameraUI.addEventListener(MediaEvent.COMPLETE, onCameraUIComplete);
          cameraUI.launch(MediaType.IMAGE);
        }
        else // If we're on the desktop (for testing)
        {
          var desktop:File = File.desktopDirectory;
          desktop.addEventListener(Event.SELECT, onFileSelected);
          desktop.browseForOpen("Pick a Big Image", [new FileFilter("Images", "*.jpg;*.jpeg;")]);
        }
      }

      private function onCameraUIComplete(e:MediaEvent):void
      {
        var cameraUI:CameraUI = e.target as CameraUI;
        cameraUI.removeEventListener(MediaEvent.COMPLETE, onCameraUIComplete);
        this.readFile(e.data.file);
      }

      private function onFileSelected(e:Event):void
      {
        var imageFile:File = e.target as File;
        imageFile.removeEventListener(Event.SELECT, onFileSelected);
        this.readFile(imageFile);
      }
  
      private function readFile(imageFile:File):void
      {
        // The two lines of code below are tempting because it's so easy,
        // but if you don't scale the image before passing it to the Image
        // component, your application will use far more memory than it needs
        // to. The code below the "return" is the better approach.
  
        /*
        this.image.source = imageFile.url;
        return;
        */

        var fs:FileStream = new FileStream();
        fs.open(imageFile, FileMode.READ);
        var imageBytes:ByteArray = new ByteArray();
        fs.readBytes(imageBytes);
        this.scaleImage(imageBytes);
      }
  
      private function scaleImage(imageBytes:ByteArray):void
      {
        var loader:Loader = new Loader();
        loader.contentLoaderInfo.addEventListener(Event.COMPLETE, onLoaderComplete);
        loader.loadBytes(imageBytes);
      }
  
      private function onLoaderComplete(e:Event):void
      {
        var loaderInfo:LoaderInfo = e.target as LoaderInfo;
        loaderInfo.removeEventListener(Event.COMPLETE, onLoaderComplete);
        var scaleFactor:Number = 1;
        if (loaderInfo.width > loaderInfo.height && loaderInfo.width > MAX_SIZE)
        {
          scaleFactor = MAX_SIZE / loaderInfo.width;
        }
        if (loaderInfo.height > loaderInfo.width && loaderInfo.height > MAX_SIZE)
        {
          scaleFactor = MAX_SIZE / loaderInfo.height;
        }
        var scaleMatrix:Matrix = new Matrix();
        var bitmapData:BitmapData = new BitmapData(loaderInfo.width * scaleFactor, loaderInfo.height * scaleFactor);
        scaleMatrix.scale(scaleFactor, scaleFactor);
        bitmapData.draw(loaderInfo.loader, scaleMatrix, null, null, null, true);
        // Now the image is really scaled!
        this.image.source = bitmapData;
      }
    ]]>
  </fx:Script>
  <s:Image id="image" width="400" height="400" top="50" horizontalCenter="0"/>
  <s:Button label="Get an Image" click="onPickImage(event);" bottom="5" left="20" right="20"/>
</s:View>