This post is part of an on-going series on video mea­sure­ment tips, tricks and best practices.

The first step of any new mea­sure­ment project is to iden­tify the “whats”, “whys”, and “hows”.  Video is no dif­fer­ent.  In this post we’ll exam­ine the “what” and “why” of video ana­lyt­ics. The “how” deserves its own post, or many posts.

Cre­at­ing the per­fect plan for video mea­sure­ment starts with under­stand­ing the for­mat (“what”) and meta­data (“why”) for your video con­tent.  Once you’ve iden­ti­fied your cadence and data points, then clearly doc­u­ment­ing the video require­ments can turn video mea­sure­ment from a cap­size to a smooth sail.

Within this post, when I refer to site, I really mean wher­ever your video is play­ing, which could be a tra­di­tional web­site, a mobile app, or even a game con­sole.  Cre­at­ing a video mea­sure­ment plan is plat­form agnos­tic.  Keep­ing a con­sis­tent video strat­egy across all your plat­forms is a best prac­tice.  Also, when I refer to con­tent, I’m talk­ing about indi­vid­ual video assets that are played via a video player.  In this post we aren’t going to worry about asset for­mats or video player types; those top­ics are firmly part of the “how” discussion.

Video on Demand (VOD) is the term used to cover any videos that are not live and that start at the begin­ning of the video when the user presses play.  For instance, replay­ing sail­ing races from yes­ter­day or high­lights from ear­lier in the day would be deliv­ered by VOD.  Com­pare that to Live Stream videos, which dis­play video con­tent that has a sin­gle start time, just like broad­cast TV. Users may jump into a live stream at any point dur­ing play­back.  Live cov­er­age of the America’s Cup, a pre­miere sail­ing race, would uti­lize stream­ing video to broad­cast the two week event.

What:  Mile­stones and Frequency

Video mea­sure­ment is a bal­anc­ing act between data gran­u­lar­ity and server call fre­quency.  You want to send enough calls to accu­rately cap­ture play­back but not more than you need from a cost per­spec­tive.  Until cer­tain tech­nol­ogy lim­i­ta­tions, such as browser close behav­ior, are changed, we are going to need to con­tinue to worry about gran­u­lar­ity vs. fre­quency.  If you don’t send track­ing calls fre­quently enough, you could lose data for users who leave the video before your next track­ing call is sent.  Just like in sail­ing rac­ing, the goal is to go as fast as pos­si­ble with­out tip­ping over.

The biggest deci­sions in video track­ing con­tinue to be the type of mile­stones used and the fre­quency of calls sent.  The types of video mile­stones are per­cent­age com­plete and time elapsed.  Per­cent­age mile­stones are cal­cu­lated based on the total dura­tion of the video.  By rely­ing on per­cent­age com­plete met­rics, the ana­lyst can stan­dard­ize video con­sump­tion across videos of dif­fer­ent lengths.  Gen­er­ally, per­cent­age com­plete fre­quen­cies are used for VOD because the video dura­tion is a known value.  Time elapsed mile­stones are based on sec­onds of video con­sumed by the user and pro­vide stan­dard­ized mile­stones for video con­sump­tion with­out know­ing the total video length or the user start­ing point.  Time elapsed mile­stones are the stan­dard for live stream­ing video because video dura­tion is not known, and nei­ther is user start­ing point.

Regard­less of the type of mile­stone used, I always rec­om­mend send­ing a call when the video starts to play and another call when the video is com­plete or the user leaves.  Every video, no mat­ter what for­mat, should send a start call.  The start call is the most impor­tant video track­ing call you can send because it cap­tures the ini­tial request and user inten­tion.  Video com­plete calls and what con­sti­tutes a com­plete (100% or 98% or…) is a lit­tle less clear cut.  I rec­om­mend send­ing a 100% com­plete call, as it is your last chance to col­lect any video usage data.

The fre­quency of calls not only indi­cates how often the track­ing calls will be sent but also deter­mines the gran­u­lar­ity of the video reports.  To iden­tify the fre­quency of calls that will work best for your con­tent, start by exam­in­ing the types of video on your site.  Are they long-form videos like com­plete cov­er­age of yesterday’s sail­ing matches, or are they short-form videos like 30 sec­ond clips of rac­ing highlights?

Short-Form VOD videos are more likely to be watched start to fin­ish and they are also, by def­i­n­i­tion, short.  This means you can scale back on your fre­quency of calls and still achieve a high accu­racy.  I rec­om­mend track­ing start and com­plete mile­stones and throw­ing in an optional mid-point call depend­ing on the com­mon con­sump­tion pat­terns for your con­tent as well as the length.  For exam­ple, if your rac­ing high­lights videos aver­age one minute in length and the stand­ings are announced within in the first 30 sec­onds, then I’d advo­cate for a 50% call, since many users will close the video before reach­ing the end. On the other hand, if your videos aver­age 25 sec­onds, then a start and com­plete call will be sufficient.

Long-Form VOD videos tend to have a higher incom­plete rate and may also have more scrub­bing and paus­ing, mak­ing the fre­quency of calls that much more impor­tant.  I like to start with quar­tile track­ing (0%, 25%, 50%, 75%, 100%) for long-form videos.  From there it is not uncom­mon to add addi­tional calls towards the start and end of the video (0%, 10%, 25%, 50%, 75%, 90%, 100%).  This will give you a lit­tle more accu­racy for those users who leave the video shortly after start­ing or those who leave before the cred­its roll.  If your VOD is very long, like replay­ing an entire day of sail­ing races, then increas­ing the per­cent­age calls to every 10% will help main­tain a higher accuracy.

Live Stream video mea­sure­ment fol­lows the assump­tion that users either stop by for a quick fix or stay and watch until the end.  Based on this the­ory it is believed that users are more likely to drop off soon after they start watch­ing and that the longer they watch the less likely they are to leave. Due to this behav­ior pat­tern, I rec­om­mend front-loading the fre­quency of calls, which means send­ing more calls at the begin­ning of the user expe­ri­ence and less calls as the user con­tin­ues to watch the video.    A com­mon cadence is to send calls at start, 30 sec­onds, 1 minute, 2 min­utes, 3 min­utes, 4 min­utes, 5 min­utes, then every 5 min­utes until the con­tent stops or the user leaves.  Some­times this gets spread out fur­ther to every ten min­utes or even every thirty min­utes for long live events. By send­ing more calls at the begin­ning of the America’s Cup live cov­er­age, it will be eas­ier for the ana­lysts to deter­mine the aver­age time spent dur­ing each visit, assum­ing that most users watch less than five min­utes of the action (which is not uncommon).

Why: Meta­data and Context

Once you’ve decided on the mile­stone type and fre­quency of the calls, the next step is to deter­mine why you are track­ing the videos and which data points will best meet your needs.  Video data falls into two areas, meta­data and con­text of play­back data.  Meta­data is data about the video con­tent and is com­monly drawn from the video asset man­age­ment sys­tem.  Con­text of play­back is data about where and when the video was viewed.  The con­text of play­back data may be found within the video player, the site con­tent man­age­ment sys­tem, or the appli­ca­tion dis­play­ing the video.

A bare bones video imple­men­ta­tion will include video title and/or video id.  Exam­ples of addi­tional meta­data on video assets include video title, show name, tal­ent, sea­son, con­tent tags, and other data about the asset that is crit­i­cal to your busi­ness. There are a vari­ety of ways to bring this data in, either at play­back, or later via data clas­si­fi­ca­tion.  For plan­ning pur­poses, sim­ply iden­tify all the impor­tant data points and include them in your doc­u­men­ta­tion.  To go back to the sail­ing regatta, every sail­ing video is tagged with the sailors fea­tured in the video.  To report on the most pop­u­lar sailors watched, I must include the list of the ath­letes’ names as a key meta­data element.

Con­text of play­back data can be exten­sive because it comes from mul­ti­ple lev­els of device, appli­ca­tion, and site. Care­fully iden­tify why you need the data before request­ing it.  Request­ing too much data can make imple­men­ta­tion costly and dif­fi­cult, with­out adding value to the analy­sis.  Com­mon con­text of play­back data includes the par­ent page on which the video was played, appli­ca­tion name, player name, site sec­tion, cam­paign, and other ele­ments that pro­vide more infor­ma­tion on how the video was found and from which expe­ri­ence it was watched.  Con­text of play­back data may also include track­ing social share links and other ele­ments pre­sented within the video player. On the America’s Cup site, the same video can be dis­played on mul­ti­ple pages. To accu­rately report on which pages a given video was watched, I need to cap­ture the par­ent page name.

Cre­at­ing a Mea­sure­ment Map

After iden­ti­fy­ing the “what” and “why” of your video mea­sure­ment, you will have a long list of data ele­ments.  Now it is time to doc­u­ment your plan. The eas­i­est way I’ve found for com­mu­ni­cat­ing with devel­op­ers or fel­low ana­lysts, while also cre­at­ing a doc­u­ment that can be used for QA, is to lay out a sim­ple spread­sheet.  This spread­sheet has affec­tion­ately been called a “magic decoder ring” and it does just that, by pro­vid­ing a map between the data sent on each call and the vari­able names within Adobe Analytics.

In the first col­umn of the sheet, list out each of fre­quency points within the video where a call will be sent.  Every col­umn to the right should con­tain one data point or vari­able to be cap­tured.  Feel free to put in the vari­able val­ues here includ­ing props, evars, and con­text data.  As you read across, each row will show all the data to be col­lected at each point within the video play­back.  Don’t for­get to include paus­ing, scrub­bing, and any social shar­ing links if you require track­ing calls to be sent.

The fol­low­ing is a sam­ple for the sail­ing videos, replay­ing the entire race using long-form VOD, show­ing daily race high­lights using short-form VOD as well as the live player dis­play­ing the regatta in real-time. Note the addi­tion of the par­ent page con­text and the ath­lete name meta­data.  More tech­ni­cal details about these vari­ables and other columns in this exam­ple will be part of the next blog post on how to implement.

With the above deci­sions made and your plan laid out, all that is left is imple­men­ta­tion (“how”) and analy­sis (gold). In my next post I will review the basics of video implementation.

0 comments