How to provide video rotation information to ActionScript as metadata

fp_air

By Jason Lee

Prior to Flash Player and AIR version 20, the Flash runtime does not correctly process the matrix field values in the Track Header Box and the Movie Header Box, as defined in the ISO specification ISO/IEC 14496-12:2008. As a result, in some instances, the user experiences an incorrectly resized and rotated video when the video is played back with Flash. The problem is particularly pronounced when playing videos that are recorded in portrait mode with old Android or iOS devices. This happens because these devices save video in landscape mode along with a matrix representing rotation transformation, when the users record the video in portrait mode.

Flash Player and AIR version 20 introduce a feature whereby video rotation meta-data is provided through ActionScript. This feature mitigates the problem discussed above by providing the video matrix information to ActionScript as a part of the meta-data object of the onMetaData event, an event handler of the NetStream object. This allows the ActionScript developers to render the video in the intended rotation matrix. It should be noted that the Flash Player engine does not automatically apply the video matrix information to the rendered video output, so as to avoid any resizing or rotation that is not intended by the ActionScript developer.

video-rotation-problem

The ActionScript code example below shows how to apply the rotation matrix to the Video object.

Note: The example assumes that the video is recorded along a matrix representing 90° clockwise rotation.

var video:Video = new Video();
 addChild(video);
 var nc:NetConnection = new NetConnection();
 nc.connect(null);
 var ns:NetStream = new NetStream(nc);
 ns.client = {};
 ns.client.onMetaData = ns_onMetaData;
 ns.client.onCuePoint = ns_onCuePoint;
 video.attachNetStream(ns);
 ns.play("Demo.mp4");
 
function ns_onMetaData(info:Object):void {
 trace("metaData");
 video.x = 0,
 video.y = 0;
 video.width = info.width;
 video.height = info.height;
 varmatrix:Matrix = video.transform.matrix; // Read old transform matrix of video object.
 varoldSize:Point = new Point(video.width, video.height);
 varnewSize:Point = new Point(Math.abs(info.matrix.transformPoint(oldSize).x), Math.abs(info.matrix.transformPoint(oldSize).y));
 matrix.translate(-(oldSize.x / 2), -(oldSize.y / 2));
 if (info.hasOwnProperty("trackHeaderBoxMatrix"))
 matrix.concat(info.trackHeaderBoxMatrix[0]); // Apply the matrix of the Track Header Box of Video Track provided by this feature to video object.
 // trackHeaderBoxMatrix is an array of MatrixObject because there can be one or more video tracks.
 if (info.hasOwnProperty("movieHeaderBoxMatrix"))
 matrix.concat(info.movieHeaderBoxMatrix); // Apply the matrix of the Movie Header Box provided by this feature to video object.
 matrix.translate((newSize.x / 2), (newSize.y / 2));
 video.transform.matrix = matrix; // Set new matrix to transform matrix of Video object.
 }
 function ns_onCuePoint(item:Object):void {
 trace("cuePoint");
 }

The feature works for NetStream objects using progressive download, which allows Flash runtime to download and directly parse a video file. However, this feature does not handle NetStream using RTMP or appendBytes, in which case, the matrix properties provided by this feature are not available. This feature provides the matrices for all file formats — MP4, M4V, F4V, and 3GPP in accordance with ISO specification ISO/IEC 14496-12:2008, when using progressive download. For all other file formats, such as FLV, the matrix properties provided by this feature are not available.

The matrix information provided by this feature can only be applied to an instance of the Video class. Therefore, with StageVideo, which uses GPU acceleration, for example, the matrix information is not applicable.

This feature introduces the following new properties of the object representing video meta-data, a parameter of onMetaData event handler:

  • Object.movieHeaderBoxMatrix: flash.geom.Matrix
  • Object.trackHeaderBoxMatrix: An array of flash.geom.MatrixtrackHeaderBoxMatrix is an array of MatrixObject because there can be one or more video tracks in a single video file.

If you are a Flash content developer and are having trouble getting a video to display in correct orientation, please try this new ActionScript functionality for handling video rotation meta-data.

Comments are closed.