One of the many reasons customers choose Omniture is our real-time reporting capabilities to provide that rapid insight needed to make decisions in a fast-paced online business environment.   Our engineering teams work hard to ensure we consistently deliver this capability, but there are rare occasions where latency has presented challenges for some customers.  It’s important to note with any latency issue that no customer data is lost and complete data integrity is maintained.  While we’ve found that recent latency issues have only affected a minority of customers, it brought to light some other potential contributing factors that you can control such as using too many server-intensive features within a report suite.  These are things that you can do something about, so in this post I will explain steps you can take to help minimize data latency.

A Little History…
As I mentioned, latency can be harder to recover from if you’ve enabled certain SiteCatalyst features.  Some SiteCatalyst features do not have much of an impact and others do.  Unfortunately, many clients do not know which features are resource intensive and which are not.  In the old, old days (when I used SiteCatalyst version 9), if you wanted to turn on a feature, such as a 2-item Traffic Data Correlation, you would need to contact Omniture and pay for it in an ad-hoc manner.  In fact, almost everything you wanted to do meant a phone call, an addendum and an account manager.  Customers complained that it took too much time and that Omniture was “nickel and diming” its customers so Omniture moved to the Administration Console model where customers were empowered to turn many things on and off as desired.  In fact, many customers ask if ALL functions can be added to the Administration Console, but doing this could allow customers to enable things that could produce latency.

Finally, before digging into the details, let me be clear that it is our intention for Omniture customers to use all of the features SiteCatalyst provides.  While the items below may cause you to think about using features I have previously raved about on this blog, keep in mind that I am providing this information only so that you understand what taxes the system the most.  If you have a good business case for using any of these features, please do so.  We often find that Omniture customers have enabled features that are both resource-intensive and not actually being used, causing a lose-lose situation. When I was a customer, I would review my implementation a few times a year and find out if any of my users were actively using the features that were enabled and take action accordingly…

#1 – Full Subrelations
While Conversion Variable Subrelations are extremely powerful, it is important that you plan ahead and think about what eVars for which you need full subrelations since they create an enormous amount of overhead.  If you think about it, if you have 10 eVars enabled and then turn on full subrelations for one eVar, you are instructing SiteCatalyst to create nine new data cubes, all of which have to be updated every time a Success Event takes place.  For those of you that have access to Omniture Discover, I would encourage you to leverage this tool for advanced variable breakdowns instead since it is built for this capability.  Some other solutions to this are to concatenate values to one eVar and use Classifications to split them out or to use a Traffic Data Correlation which is much less resource intensive.

#2 – Unused Conversion Variables
Did you know that even if you are not passing any data to a Conversion Variable (eVar) it is increasing your processing load?  This is due to the fact that every time a Success Event takes place, it is associated to every enabled eVar, even if the value associated with it is “None.”  Therefore, when I see clients that have 20 eVars enabled, but are only actively using 2-3 of them I beg them to disable the eVars that are not being used so they can improve performance.

#3 – Multiple Success Events on a Page
Another item that can impact latency is the setting of multiple Success Events on a page.  If you set two Success Events on a page, you are essentially doubling all of the data processing mentioned above on those pages.

#4 – Traffic Spikes
While it is not possible to anticipate every traffic spike your site will have, there are many cases in which you know that you will receive more traffic such as holiday periods or via seasonality.  SiteCatalyst provides a way for you to tell Omniture when you expect a traffic spike so we can be prepared for it by adding additional resources.  This notification system can be found in the Administration Console:

#5 – Participation
A few weeks ago, I explained Participation in a post.  While Participation is very powerful, you probably do not need to have Participation enabled for every Success Event.  For example, if you have a five step conversion funnel and set a Success Event for each step, you could probably be ok with enabling Participation on the first and last step without losing much insight.  Also, once you use Participation to learn about what pages lead to your site Success Events, it may be ok to disable Participation and re-enable it later in the year or after a site/page redesign since the data may not change drastically from month to month.

#6 – Visits & Visitors in Conversion Reports
A frequent request from customers is to allow Visits and Unique Visitors in Conversion reports so they can be used in Calculated metrics.  I often solve this problem by pulling data into the ExcelClient where I can compare Traffic Visits for a day to Conversion Success Events for the same date.  However, if this functionality is critical to your business I suggest you talk to your Account Manager or Omniture Consulting.

#7 – Complex VISTA Rules
When writing VISTA rules, sometimes there is a tendency to throw everything in (including the kitchen sink!).  The more complex you make VISTA rules, the slower data may process.

#8 – Unique Values
There are some Traffic Variables (sProps) and Conversion Variables (eVars) that end up having very few unique values and others that have millions of unique values.  For example, an eVar capturing the user’s birth year will have a relatively finite number of values, whereas an eVar capturing unique User ID’s for a site like Facebook, would have millions of unique values.  While this does not impact processing speed, it can impact reporting speed.  For this reason, you need to think about how many unique values you expect a variable to have when setting up Conversion Variable Subrelations and Traffic Data Correlations.  Many times, customers will use a VISTA rule to “blank out” variables in the SiteCatalyst interface that have over 500,000 unique values per month and leverage the DataWarehouse for these data points.

#9 – Products Variable
In a past post I mentioned different ways that you could use the Products Variable in SiteCatalyst.  Since the Products Variable is currently one of the only Conversion Variables that allows you to pass in multiple values, I have seen some clients be very creative with its use.  However, passing in more than ten items to the Products variable can cause problems so if you have a need for storing many items in the products variable, I suggest you speak to your Account Manager or Omniture Consulting.

#10 – Event Serialization
In a recent post I explained how you can use Event Serialization to de-duplicate Success Events.  However, if you have millions of Success Events taking place on your site, using Event Serialization can become a resource issue.  It is best used for Success Events that are critical to your site, not every Success Event.

So there is your top 10 list of culprits.  Omniture is constantly looking at what it can do to improve performance and hopes that many of the items above will be less resource intensive in the future, please keep these items in mind when talking to your Account Manager or ClientCare.

Have a question about anything related to Omniture SiteCatalyst?  Is there something on your website that you would like to report on, but don’t know how?  Do you have any tips or best practices you want to share?  If so, please leave a comment here or send me an e-mail at insidesitecatalyst@omniture.com and I will do my best to answer it right here on the blog so everyone can learn! (Don’t worry – I won’t use your name or company name!).  If you are on Twitter, you can follow me at http://twitter.com/Omni_man.

Learn more about Omniture Consulting
Learn more about Omniture University
11 comments
Adam Greco
Adam Greco

Rob, I have passed your suggestion to Product Management. Thanks! Adam

Rob Patton
Rob Patton

It would be a great improvement if SiteCatalyst could display a latency warning that is promintently visible whenever the servers arent caught-up to nominal near-raltime refreshes. This would prevent users from accidetnally reporting incomplete data to their colleagues. Similar alerts would be helpful if daily/weekly data extracts and other time-sensitive processes become delayed.

Sushant Ajmani
Sushant Ajmani

Here are couple of more contributing factors towards Latency: - SAINT Classification - Though this is among the best features of SiteCatalyst but, it can seriously impact the reports performance if you go beyond 10 Classification Attributes. Last year, we setup 25 Classification Attributes on the request of our client who was very pushy, and it causes train wreck performance. Some of the reports took more than 40 mins to load. So; if you are using classification to classify your Products, Campaigns, Content and Store IDs, please keep the # of attributes under 10 otherwise, you will suffer sever performance issues and often times, your site will show latency issues. Custom Link Tracking - This is often used with AJAX Components where, the engagement is more important than the actual page view activity. Most of the retailers these days, spend a decent amount of money creating these fancy widgets and wants to track each and every click and mouseover activity, and if you are triggering Custom Link Tracking code at a regular interval, you are not only impacting the performance of the widget, but might also result in latency. I don't have good data to prove but, looking forward for that. CMS Content on the site - Most of the Brands and Retailers have lots of content on their Home, Category and Shop Pages which takes a lot of time to load and if your Omniture code is at the extreme footer then, it might impact your Image Request response time. Recently, I observed 74sec response time which was heart breaking as compare to a benchmark number of 100-180msec Packet Header Size - There are lot of retailers these days uses more than 1 analytics solution on their site, and the best combination you would find on the internet is Omniture and Google Analytics. Both allows the First Party Cookie setup as per the industry best practices but often times, the technology folks doesn't keep an eye on the packet header which contains a lot of information which goes beyond the threshold of Omniture and they split up the package in to 2 which causes another set of performance issues and slower response time. Problem caused by an Unknown Client on the Server - If your site is on a same server where 30 other clients of Omniture resides, and one of them is facing a serious latency issue then, you might also become a victim of that.

Adam Greco
Adam Greco

Wei - You can run Omniture's DiscoverOnPremise in-house, but no SiteCatalyst. Adam

Wei
Wei

Hi can anybody tell me whether Omniture customers can host this web analytics tool in-house? Are any clients doing so? What operating platform can it be run on and what database is need? Thanks.

Adam Greco
Adam Greco

Sandy, First, I apologize that you are having latency problems. I have asked OmnitureCare to follow-up with you immediately to see if there is anything else that can be done. Having been a client myself, I understand how frustrating latency can be and I assure you that we are doing everything in our power to address the issue. That being said, the purpose of this post was simply to let clients know about features that are more server-intensive than others to be sure that clients double-check that they are actively using them. We find many clients have our most server-intensive features enabled, but are not using them. The feedback we received at Omniture Summit was that customers did not know which SiteCatalyst features took up more server resources than others and this post was aimed at communicating this so Omniture can be as transparent as possible. Thanks! Adam

Sandy Miller
Sandy Miller

Hi I do not regard your article as helpful. Our Omniture reporting has been suffering greatly from severe latency - to the point taht I am consideing employing secondary analytics since Omniture gives nowhere near real-time data. 'Normal' service is pretty close to real-time but we do not seem to receive 'normal' service very often or for any sustained period of time. I cant check my sales for example, or any campaigns data simply because Omniture is so slow. This is, I would suggest, the second major outage in almost as many months - we suffered greatly in January. And now, for the last week, data is constantly delayed roughly 8 hours. Useless for proactive management of my ecommerce enterprise. Not that 'normal' was ever that good either. Sorry, but we have had nothing but problems with Omniture from the start. The fact I cant see any of todays data renders your system redundant. Any comments Omniture?? You can email me. Oh, and I have been communicating these issues constantly to our partner www.e-inbusiness.co.uk sandy.miller@ewm.co.uk

Adam Greco
Adam Greco

Sal - Any situations like you are describing below should be reported to ClientCare immediately...Thanks! Adam

Adam Greco
Adam Greco

Dan - I know this is a delicate issue, but if you look at my disclaimer, the only purpose of this post is to make people aware of what taxes the system the most, not to dissuade you from using them if you have a valid business objective. Many customers enable things that they don't actively use and then wonder why things are slow..I hope that makes sense... Adam

Sal
Sal

I think that the main issue with the latency is not as much the latency, but that instead of reporting latency issues, Omniture reports just return incorrect values. We have regularly had to go back and change reports and dashboards, as we only later learned that there were server latency issues. With the latency being unpredictable, is there any good solution for us to avoid utilizing bad data?

Dan Roden
Dan Roden

I get what you are saying about processing, but from a customer's view, having been "sold" all of these rich features as solutions to our RFP's and other WA requirements, you are basically saying that we should expect latency if we use anything that is required for high level analytics measures to answer enterprise level questions.