Recently I have spent a lot of time dis­cussing with cus­tomers the impor­tance of import­ing his­tor­i­cal web ana­lyt­ics data when they move from one ana­lyt­ics plat­form to another.

Over­whelm­ingly, I hear mar­keters say that they must be able to import all data when they make the switch, and over­whelm­ingly, they get wide-eyed with anx­i­ety when I tell them this: I am not a big fan of import­ing his­tor­i­cal data. I believe it is only of value in some very spe­cific circumstances.

Don’t get me wrong. When it comes to his­tor­i­cal offline data, or data from an email cam­paign or from an ERP sys­tem, that infor­ma­tion is of immense value in terms of ana­lyt­ics. But when it comes to com­par­ing the gen­eral data in one ana­lyt­ics sys­tem to another, I believe it’s a fool’s errand: it wastes time and takes a tremen­dous amount of effort for very lit­tle payout.

There are many rea­sons for this, and my friend Avinash recently expressed his own con­cerns on import­ing his­tor­i­cal data on his Occam’s Razor blog – many of which I agree with wholeheartedly.

That said, here are some of my own con­cerns and thoughts on what to import when you switch ana­lyt­ics vendors:

-> Con­cern #1. Incon­sis­tency between platforms

Not all plat­forms count met­rics the same way. Take unique vis­i­tors: there are def­i­n­i­tions in the indus­try for what a “unique vis­i­tor” really means, but none of them are stan­dard­ized. Thus, there’s no sense of com­pli­ance, and each ana­lyt­ics pack­age tracks in a slightly (or hugely) dif­fer­ent way.

There are dif­fer­ences in how ana­lyt­ics pack­ages han­dle browser types, how they han­dle wire­less, how they han­dle cook­ies… and these are just dif­fer­ences between two tag-based ana­lyt­ics sys­tems. If you’re mov­ing from an ana­lyt­ics sys­tem based on tags to one based on log-files, there are even greater dif­fer­ences because of the dif­fer­ences in how the sys­tems process traffic.

The bot­tom line is, it’s almost impos­si­ble to try to com­pare these things.

-> Con­cern #2. The imple­men­ta­tion itself

When a com­pany deploys a new ana­lyt­ics sys­tem, changes are always made. You may put your tags at the top of the page rather than at the bot­tom. You go from a sta­tic HTML page to one that has flash. You deploy a vari­ety of small, seem­ingly incon­se­quen­tial changes.

But those changes mean that in addi­tion to com­par­ing data from one sys­tem to another, you’re look­ing at two dif­fer­ent implementations.

It’s like com­par­ing apples to aardvarks.

The com­pa­ra­bil­ity of your data is affected, and it can lead to mis­in­formed decisions.

So what can you track?

When com­pa­nies switch to Omni­ture ana­lyt­ics and they want to import his­tor­i­cal data, I sug­gest that they focus on their KPIs. Key per­for­mance indi­ca­tors are expres­sions of effec­tive­ness, of how effi­cient a web­site is at achiev­ing a spe­cific busi­ness goal. So, while your under­ly­ing data may shift, the *rela­tion­ship* between the KPIs, between one sys­tem and another, should not be dra­mat­i­cally different.

So, if you used one ana­lyt­ics plat­form for two years and your con­ver­sion rate was 2 per­cent, and you deploy a new one and your con­ver­sion rate is 4 per­cent, then, assum­ing that you haven’t changed any­thing in the site itself, the rela­tion­ship between those aver­ages should remain consistent.

How, then, do you han­dle data when migrating?

-> Step 1. Com­par­i­son in Excel

One of the first things you should do, before you try to import a bunch of data, is slow down and do a very sim­ple com­par­i­son on a daily basis in Excel. Look at the dif­fer­ence between your sys­tems, and develop an under­stand­ing of what the rela­tion­ship actu­ally looks like.

What often hap­pens is peo­ple look at the rela­tion­ship and say, “Our page views are down 20 percent!”

Don’t fret. Instead, relax and look at the rela­tion­ship between the two. Are page views down 20 per­cent con­sis­tently, across the board? If not, there might have been some­thing else going on dur­ing the time the rela­tion­ship changed — per­haps there were hol­i­days, an elec­tion, or the Super Bowl that could have biased your num­bers. Or maybe there was a pro­mo­tion that mar­ket­ing was run­ning that could have affected the spe­cific period dur­ing the switch.

If, for exam­ple, in the week up to tran­si­tion, your mar­ket­ing depart­ment launched a free ship­ping pro­mo­tion — and con­ver­sion num­bers went up — then switched to the new plat­form at the same time that the pro­mo­tion ended, you would have not only an inher­ent dif­fer­ence in mea­sure­ment, but a change in con­ver­sions due to the con­clu­sion of that campaign.

Bear­ing in mind the out­side ele­ments, try to dis­cover what the ratio looks like between the data from the two systems.

-> Step 2. Aug­ment your data with notes on exter­nal events, or “event notification”

Some ana­lyt­ics pack­ages, such as Omniture’s Site­Cat­a­lyst, allow you to beef up your data with notes to inform those look­ing at the data of what was hap­pen­ing exter­nally at the time the data was taken. For exam­ple, you can go into Site­Cat­a­lyst and make a note that on Jan. 5, you switched ana­lyt­ics plat­forms. When look­ing at the data in the dash­boards, the note will show up to help you under­stand any changes in the data.

-> Step 3. Don’t let man­age­ment look at the dif­fer­ence in data with­out giv­ing them a heads-up

It’s very impor­tant to inform man­age­ment that the num­bers are not going to match. Set up their expec­ta­tions that the num­bers will be dif­fer­ent, and tell them how you plan to rec­on­cile the dif­fer­ence. Oth­er­wise, there will be dis­gruntle­ment and heartache.

Also com­mu­ni­cate with line-of-business man­agers and other key stakeholders.

Only at this point should you think about import­ing his­tor­i­cal data into your system.

If you do decide to import KPI data, it’s really not that com­pli­cated, but it’s of lit­tle value if peo­ple don’t under­stand the dif­fer­ences between the two sys­tems. Make sure man­age­ment under­stands the ram­i­fi­ca­tions, and remem­ber to do an event noti­fi­ca­tion to indi­cate when the change in the sys­tem occurred.

If you’d like to dis­cuss his­tor­i­cal data migra­tion or have some thoughts on the sub­ject, please don’t hes­i­tate to leave us a comment!

0 comments