One of the many rea­sons cus­tomers choose Omni­ture is our real-time report­ing capa­bil­i­ties to pro­vide that rapid insight needed to make deci­sions in a fast-paced online busi­ness envi­ron­ment.   Our engi­neer­ing teams work hard to ensure we con­sis­tently deliver this capa­bil­ity, but there are rare occa­sions where latency has pre­sented chal­lenges for some cus­tomers.  It’s impor­tant to note with any latency issue that no cus­tomer data is lost and com­plete data integrity is main­tained.  While we’ve found that recent latency issues have only affected a minor­ity of cus­tomers, it brought to light some other poten­tial con­tribut­ing fac­tors that you can con­trol such as using too many server-intensive fea­tures within a report suite.  These are things that you can do some­thing about, so in this post I will explain steps you can take to help min­i­mize data latency.

A Lit­tle His­tory…
As I men­tioned, latency can be harder to recover from if you’ve enabled cer­tain Site­Cat­a­lyst fea­tures.  Some Site­Cat­a­lyst fea­tures do not have much of an impact and oth­ers do.  Unfor­tu­nately, many clients do not know which fea­tures are resource inten­sive and which are not.  In the old, old days (when I used Site­Cat­a­lyst ver­sion 9), if you wanted to turn on a fea­ture, such as a 2-item Traf­fic Data Cor­re­la­tion, you would need to con­tact Omni­ture and pay for it in an ad-hoc man­ner.  In fact, almost every­thing you wanted to do meant a phone call, an adden­dum and an account man­ager.  Cus­tomers com­plained that it took too much time and that Omni­ture was “nickel and dim­ing” its cus­tomers so Omni­ture moved to the Admin­is­tra­tion Con­sole model where cus­tomers were empow­ered to turn many things on and off as desired.  In fact, many cus­tomers ask if ALL func­tions can be added to the Admin­is­tra­tion Con­sole, but doing this could allow cus­tomers to enable things that could pro­duce latency.

Finally, before dig­ging into the details, let me be clear that it is our inten­tion for Omni­ture cus­tomers to use all of the fea­tures Site­Cat­a­lyst pro­vides.  While the items below may cause you to think about using fea­tures I have pre­vi­ously raved about on this blog, keep in mind that I am pro­vid­ing this infor­ma­tion only so that you under­stand what taxes the sys­tem the most.  If you have a good busi­ness case for using any of these fea­tures, please do so.  We often find that Omni­ture cus­tomers have enabled fea­tures that are both resource-intensive and not actu­ally being used, caus­ing a lose-lose sit­u­a­tion. When I was a cus­tomer, I would review my imple­men­ta­tion a few times a year and find out if any of my users were actively using the fea­tures that were enabled and take action accordingly…

#1 — Full Sub­re­la­tions
While Con­ver­sion Vari­able Sub­re­la­tions are extremely pow­er­ful, it is impor­tant that you plan ahead and think about what eVars for which you need full sub­re­la­tions since they cre­ate an enor­mous amount of over­head.  If you think about it, if you have 10 eVars enabled and then turn on full sub­re­la­tions for one eVar, you are instruct­ing Site­Cat­a­lyst to cre­ate nine new data cubes, all of which have to be updated every time a Suc­cess Event takes place.  For those of you that have access to Omni­ture Dis­cover, I would encour­age you to lever­age this tool for advanced vari­able break­downs instead since it is built for this capa­bil­ity.  Some other solu­tions to this are to con­cate­nate val­ues to one eVar and use Clas­si­fi­ca­tions to split them out or to use a Traf­fic Data Cor­re­la­tion which is much less resource intensive.

#2 — Unused Con­ver­sion Vari­ables
Did you know that even if you are not pass­ing any data to a Con­ver­sion Vari­able (eVar) it is increas­ing your pro­cess­ing load?  This is due to the fact that every time a Suc­cess Event takes place, it is asso­ci­ated to every enabled eVar, even if the value asso­ci­ated with it is “None.”  There­fore, when I see clients that have 20 eVars enabled, but are only actively using 2–3 of them I beg them to dis­able the eVars that are not being used so they can improve performance.

#3 — Mul­ti­ple Suc­cess Events on a Page
Another item that can impact latency is the set­ting of mul­ti­ple Suc­cess Events on a page.  If you set two Suc­cess Events on a page, you are essen­tially dou­bling all of the data pro­cess­ing men­tioned above on those pages.

#4 — Traf­fic Spikes
While it is not pos­si­ble to antic­i­pate every traf­fic spike your site will have, there are many cases in which you know that you will receive more traf­fic such as hol­i­day peri­ods or via sea­son­al­ity.  Site­Cat­a­lyst pro­vides a way for you to tell Omni­ture when you expect a traf­fic spike so we can be pre­pared for it by adding addi­tional resources.  This noti­fi­ca­tion sys­tem can be found in the Admin­is­tra­tion Console:

#5 — Par­tic­i­pa­tion
A few weeks ago, I explained Par­tic­i­pa­tion in a post.  While Par­tic­i­pa­tion is very pow­er­ful, you prob­a­bly do not need to have Par­tic­i­pa­tion enabled for every Suc­cess Event.  For exam­ple, if you have a five step con­ver­sion fun­nel and set a Suc­cess Event for each step, you could prob­a­bly be ok with enabling Par­tic­i­pa­tion on the first and last step with­out los­ing much insight.  Also, once you use Par­tic­i­pa­tion to learn about what pages lead to your site Suc­cess Events, it may be ok to dis­able Par­tic­i­pa­tion and re-enable it later in the year or after a site/page redesign since the data may not change dras­ti­cally from month to month.

#6 — Vis­its & Vis­i­tors in Con­ver­sion Reports
A fre­quent request from cus­tomers is to allow Vis­its and Unique Vis­i­tors in Con­ver­sion reports so they can be used in Cal­cu­lated met­rics.  I often solve this prob­lem by pulling data into the Excel­Client where I can com­pare Traf­fic Vis­its for a day to Con­ver­sion Suc­cess Events for the same date.  How­ever, if this func­tion­al­ity is crit­i­cal to your busi­ness I sug­gest you talk to your Account Man­ager or Omni­ture Consulting.

#7 — Com­plex VISTA Rules
When writ­ing VISTA rules, some­times there is a ten­dency to throw every­thing in (includ­ing the kitchen sink!).  The more com­plex you make VISTA rules, the slower data may process.

#8 — Unique Val­ues
There are some Traf­fic Vari­ables (sProps) and Con­ver­sion Vari­ables (eVars) that end up hav­ing very few unique val­ues and oth­ers that have mil­lions of unique val­ues.  For exam­ple, an eVar cap­tur­ing the user’s birth year will have a rel­a­tively finite num­ber of val­ues, whereas an eVar cap­tur­ing unique User ID’s for a site like Face­book, would have mil­lions of unique val­ues.  While this does not impact pro­cess­ing speed, it can impact report­ing speed.  For this rea­son, you need to think about how many unique val­ues you expect a vari­able to have when set­ting up Con­ver­sion Vari­able Sub­re­la­tions and Traf­fic Data Cor­re­la­tions.  Many times, cus­tomers will use a VISTA rule to “blank out” vari­ables in the Site­Cat­a­lyst inter­face that have over 500,000 unique val­ues per month and lever­age the DataWare­house for these data points.

#9 — Prod­ucts Vari­able
In a past post I men­tioned dif­fer­ent ways that you could use the Prod­ucts Vari­able in Site­Cat­a­lyst.  Since the Prod­ucts Vari­able is cur­rently one of the only Con­ver­sion Vari­ables that allows you to pass in mul­ti­ple val­ues, I have seen some clients be very cre­ative with its use.  How­ever, pass­ing in more than ten items to the Prod­ucts vari­able can cause prob­lems so if you have a need for stor­ing many items in the prod­ucts vari­able, I sug­gest you speak to your Account Man­ager or Omni­ture Con­sult­ing.

#10 — Event Seri­al­iza­tion
In a recent post I explained how you can use Event Seri­al­iza­tion to de-duplicate Suc­cess Events.  How­ever, if you have mil­lions of Suc­cess Events tak­ing place on your site, using Event Seri­al­iza­tion can become a resource issue.  It is best used for Suc­cess Events that are crit­i­cal to your site, not every Suc­cess Event.

So there is your top 10 list of cul­prits.  Omni­ture is con­stantly look­ing at what it can do to improve per­for­mance and hopes that many of the items above will be less resource inten­sive in the future, please keep these items in mind when talk­ing to your Account Man­ager or ClientCare.

Have a ques­tion about any­thing related to Omni­ture Site­Cat­a­lyst?  Is there some­thing on your web­site that you would like to report on, but don’t know how?  Do you have any tips or best prac­tices you want to share?  If so, please leave a com­ment here or send me an e-mail at insidesitecatalyst@​omniture.​com and I will do my best to answer it right here on the blog so every­one can learn! (Don’t worry — I won’t use your name or com­pany name!).  If you are on Twit­ter, you can fol­low me at http://​twit​ter​.com/​O​m​n​i​_​man.

Learn more about Omni­ture Consulting
Learn more about Omni­ture University
11 comments
Adam Greco
Adam Greco

Rob, I have passed your suggestion to Product Management. Thanks! Adam

Rob Patton
Rob Patton

It would be a great improvement if SiteCatalyst could display a latency warning that is promintently visible whenever the servers arent caught-up to nominal near-raltime refreshes. This would prevent users from accidetnally reporting incomplete data to their colleagues. Similar alerts would be helpful if daily/weekly data extracts and other time-sensitive processes become delayed.

Sushant Ajmani
Sushant Ajmani

Here are couple of more contributing factors towards Latency: - SAINT Classification - Though this is among the best features of SiteCatalyst but, it can seriously impact the reports performance if you go beyond 10 Classification Attributes. Last year, we setup 25 Classification Attributes on the request of our client who was very pushy, and it causes train wreck performance. Some of the reports took more than 40 mins to load. So; if you are using classification to classify your Products, Campaigns, Content and Store IDs, please keep the # of attributes under 10 otherwise, you will suffer sever performance issues and often times, your site will show latency issues. Custom Link Tracking - This is often used with AJAX Components where, the engagement is more important than the actual page view activity. Most of the retailers these days, spend a decent amount of money creating these fancy widgets and wants to track each and every click and mouseover activity, and if you are triggering Custom Link Tracking code at a regular interval, you are not only impacting the performance of the widget, but might also result in latency. I don't have good data to prove but, looking forward for that. CMS Content on the site - Most of the Brands and Retailers have lots of content on their Home, Category and Shop Pages which takes a lot of time to load and if your Omniture code is at the extreme footer then, it might impact your Image Request response time. Recently, I observed 74sec response time which was heart breaking as compare to a benchmark number of 100-180msec Packet Header Size - There are lot of retailers these days uses more than 1 analytics solution on their site, and the best combination you would find on the internet is Omniture and Google Analytics. Both allows the First Party Cookie setup as per the industry best practices but often times, the technology folks doesn't keep an eye on the packet header which contains a lot of information which goes beyond the threshold of Omniture and they split up the package in to 2 which causes another set of performance issues and slower response time. Problem caused by an Unknown Client on the Server - If your site is on a same server where 30 other clients of Omniture resides, and one of them is facing a serious latency issue then, you might also become a victim of that.

Adam Greco
Adam Greco

Wei - You can run Omniture's DiscoverOnPremise in-house, but no SiteCatalyst. Adam

Wei
Wei

Hi can anybody tell me whether Omniture customers can host this web analytics tool in-house? Are any clients doing so? What operating platform can it be run on and what database is need? Thanks.

Adam Greco
Adam Greco

Sandy, First, I apologize that you are having latency problems. I have asked OmnitureCare to follow-up with you immediately to see if there is anything else that can be done. Having been a client myself, I understand how frustrating latency can be and I assure you that we are doing everything in our power to address the issue. That being said, the purpose of this post was simply to let clients know about features that are more server-intensive than others to be sure that clients double-check that they are actively using them. We find many clients have our most server-intensive features enabled, but are not using them. The feedback we received at Omniture Summit was that customers did not know which SiteCatalyst features took up more server resources than others and this post was aimed at communicating this so Omniture can be as transparent as possible. Thanks! Adam

Sandy Miller
Sandy Miller

Hi I do not regard your article as helpful. Our Omniture reporting has been suffering greatly from severe latency - to the point taht I am consideing employing secondary analytics since Omniture gives nowhere near real-time data. 'Normal' service is pretty close to real-time but we do not seem to receive 'normal' service very often or for any sustained period of time. I cant check my sales for example, or any campaigns data simply because Omniture is so slow. This is, I would suggest, the second major outage in almost as many months - we suffered greatly in January. And now, for the last week, data is constantly delayed roughly 8 hours. Useless for proactive management of my ecommerce enterprise. Not that 'normal' was ever that good either. Sorry, but we have had nothing but problems with Omniture from the start. The fact I cant see any of todays data renders your system redundant. Any comments Omniture?? You can email me. Oh, and I have been communicating these issues constantly to our partner www.e-inbusiness.co.uk sandy.miller@ewm.co.uk

Adam Greco
Adam Greco

Sal - Any situations like you are describing below should be reported to ClientCare immediately...Thanks! Adam

Adam Greco
Adam Greco

Dan - I know this is a delicate issue, but if you look at my disclaimer, the only purpose of this post is to make people aware of what taxes the system the most, not to dissuade you from using them if you have a valid business objective. Many customers enable things that they don't actively use and then wonder why things are slow..I hope that makes sense... Adam

Sal
Sal

I think that the main issue with the latency is not as much the latency, but that instead of reporting latency issues, Omniture reports just return incorrect values. We have regularly had to go back and change reports and dashboards, as we only later learned that there were server latency issues. With the latency being unpredictable, is there any good solution for us to avoid utilizing bad data?

Dan Roden
Dan Roden

I get what you are saying about processing, but from a customer's view, having been "sold" all of these rich features as solutions to our RFP's and other WA requirements, you are basically saying that we should expect latency if we use anything that is required for high level analytics measures to answer enterprise level questions.