Using AIRBench to Test AIR Across Android Phones

I recently uploaded an application called AIRBench to the Android Market which is designed to compare Adobe AIR performance across devices. We test AIR across as many different devices with as many different configurations as we can, of course, but there are so many different permutations out there now that we decided to release an app to help us understand how different devices compare to each other.

AIRBench performs two types of tests: capability and performance. Capability tests just ensure that a particular API works, and performance tests determine how well your device performs a particular task. Below is a breakdown of all the tests included in the first version of AIRBench:

Capability Tests

  1. Camera: calls Camera.getCamera() and displays the video feed.
  2. CameraUIImageTest: Makes sure you can take a picture using the CameraUI class.
  3. CameraUIVideoTest: Makes sure you can record video with the CameraUI class.
  4. CameraRollAddTest: Adds an image to the camera roll.
  5. CameraRollBrowseTest: Tests the ability for the user to browse the camera roll and choose an image.
  6. GeolocationTest: Makes sure geolocation data can discovered.
  7. AccelerometerTest: Makes sure the Accelerometer APIs work.
  8. MultitouchTest: Has the user place as many fingers on the screen as Multitouch.maxTouchPoints reports, and makes sure those two numbers are equal.
  9. MicrophoneTest: Makes sure sound can be detected from the microphone. (Watch out for interference on this one.)
  10. StageWebViewTest: Loads an HTML page into a StageWebView.

Performance Tests

  1. XMLTest: Parses a pretty big XML file (74K) and reports the time. (It’s my Google Reader feed copied several times.)
  2. MemoryAllocationTest: Creates a ByteArray and sets it length to 8MB, then reports the amount of time it took.
  3. FileAccessTest: Creates a file, writes 1MB of data, reads the file, then deletes the file. Reports the total time.
  4. SQLTest: Creates a new database, creates a table with 5 columns, inserts 500 rows of data, runs a select statement, deletes all the data, and closes the connection. Reports the total time.
  5. StringTest: Runs 9 different string operations on a very big string and reports the total time.
  6. ComputationTest: Hashes a 49K image using SHA256 (to test bitwise operations), then calculates sunrise and sunset times for 10 different locations (Adobe offices) 100 times (to test math operations).
  7. RenderingTest: Animates a square, changing the rotation, scale, and alpha at 60 frames per second. Measures the FPS after a few seconds and reports the number of frames dropped. (Note that this version of the test does not use hardware acceleration — that will be tested in a future test.)

Tests are versioned, so I’ll release new and better tests as we add new capabilities to the runtime, and as I have time to make improvements. Results can be submitted to Adobe (completely anonymously), and once I get back from MAX, I’ll write a series of reports for everyone to check out at airbenchmark.com.

If you want to help contribute data to the AIRBench database, just search for AIRBench in the Android Market, run through the series of tests, and submit the results. AIRBench serves a second purpose by providing a simple demonstration of how all the device-specific APIs work, so if you’re interested in doing some AIR mobile development, check out the code.