Blog Post:One of the many reasons customers choose Omniture is our real-time reporting capabilities to provide that rapid insight needed to make decisions in a fast-paced online business environment.   Our engineering teams work hard to ensure we consistently deliver this capability, but there are rare occasions where latency has presented challenges for some customers.  It's important to note with any latency issue that no customer data is lost and complete data integrity is maintained.  While we've found that recent latency issues have only affected a minority of customers, it brought to light some other potential contributing factors that you can control such as using too many server-intensive features within a report suite.  These are things that you can do something about, so in this post I will explain steps you can take to help minimize data latency. A Little History... As I mentioned, latency can be harder to recover from if you've enabled certain SiteCatalyst features.  Some SiteCatalyst features do not have much of an impact and others do.  Unfortunately, many clients do not know which features are resource intensive and which are not.  In the old, old days (when I used SiteCatalyst version 9), if you wanted to turn on a feature, such as a 2-item Traffic Data Correlation, you would need to contact Omniture and pay for it in an ad-hoc manner.  In fact, almost everything you wanted to do meant a phone call, an addendum and an account manager.  Customers complained that it took too much time and that Omniture was "nickel and diming" its customers so Omniture moved to the Administration Console model where customers were empowered to turn many things on and off as desired.  In fact, many customers ask if ALL functions can be added to the Administration Console, but doing this could allow customers to enable things that could produce latency. Finally, before digging into the details, let me be clear that it is our intention for Omniture customers to use all of the features SiteCatalyst provides.  While the items below may cause you to think about using features I have previously raved about on this blog, keep in mind that I am providing this information only so that you understand what taxes the system the most.  If you have a good business case for using any of these features, please do so.  We often find that Omniture customers have enabled features that are both resource-intensive and not actually being used, causing a lose-lose situation. When I was a customer, I would review my implementation a few times a year and find out if any of my users were actively using the features that were enabled and take action accordingly... #1 - Full Subrelations While Conversion Variable Subrelations are extremely powerful, it is important that you plan ahead and think about what eVars for which you need full subrelations since they create an enormous amount of overhead.  If you think about it, if you have 10 eVars enabled and then turn on full subrelations for one eVar, you are instructing SiteCatalyst to create nine new data cubes, all of which have to be updated every time a Success Event takes place.  For those of you that have access to Omniture Discover, I would encourage you to leverage this tool for advanced variable breakdowns instead since it is built for this capability.  Some other solutions to this are to concatenate values to one eVar and use Classifications to split them out or to use a Traffic Data Correlation which is much less resource intensive. #2 - Unused Conversion Variables Did you know that even if you are not passing any data to a Conversion Variable (eVar) it is increasing your processing load?  This is due to the fact that every time a Success Event takes place, it is associated to every enabled eVar, even if the value associated with it is "None."  Therefore, when I see clients that have 20 eVars enabled, but are only actively using 2-3 of them I beg them to disable the eVars that are not being used so they can improve performance. #3 - Multiple Success Events on a Page Another item that can impact latency is the setting of multiple Success Events on a page.  If you set two Success Events on a page, you are essentially doubling all of the data processing mentioned above on those pages. #4 - Traffic Spikes While it is not possible to anticipate every traffic spike your site will have, there are many cases in which you know that you will receive more traffic such as holiday periods or via seasonality.  SiteCatalyst provides a way for you to tell Omniture when you expect a traffic spike so we can be prepared for it by adding additional resources.  This notification system can be found in the Administration Console: #5 - Participation A few weeks ago, I explained Participation in a post.  While Participation is very powerful, you probably do not need to have Participation enabled for every Success Event.  For example, if you have a five step conversion funnel and set a Success Event for each step, you could probably be ok with enabling Participation on the first and last step without losing much insight.  Also, once you use Participation to learn about what pages lead to your site Success Events, it may be ok to disable Participation and re-enable it later in the year or after a site/page redesign since the data may not change drastically from month to month. #6 - Visits & Visitors in Conversion Reports A frequent request from customers is to allow Visits and Unique Visitors in Conversion reports so they can be used in Calculated metrics.  I often solve this problem by pulling data into the ExcelClient where I can compare Traffic Visits for a day to Conversion Success Events for the same date.  However, if this functionality is critical to your business I suggest you talk to your Account Manager or Omniture Consulting. #7 - Complex VISTA Rules When writing VISTA rules, sometimes there is a tendency to throw everything in (including the kitchen sink!).  The more complex you make VISTA rules, the slower data may process. #8 - Unique Values There are some Traffic Variables (sProps) and Conversion Variables (eVars) that end up having very few unique values and others that have millions of unique values.  For example, an eVar capturing the user's birth year will have a relatively finite number of values, whereas an eVar capturing unique User ID's for a site like Facebook, would have millions of unique values.  While this does not impact processing speed, it can impact reporting speed.  For this reason, you need to think about how many unique values you expect a variable to have when setting up Conversion Variable Subrelations and Traffic Data Correlations.  Many times, customers will use a VISTA rule to "blank out" variables in the SiteCatalyst interface that have over 500,000 unique values per month and leverage the DataWarehouse for these data points. #9 - Products Variable In a past post I mentioned different ways that you could use the Products Variable in SiteCatalyst.  Since the Products Variable is currently one of the only Conversion Variables that allows you to pass in multiple values, I have seen some clients be very creative with its use.  However, passing in more than ten items to the Products variable can cause problems so if you have a need for storing many items in the products variable, I suggest you speak to your Account Manager or Omniture Consulting. #10 - Event Serialization In a recent post I explained how you can use Event Serialization to de-duplicate Success Events.  However, if you have millions of Success Events taking place on your site, using Event Serialization can become a resource issue.  It is best used for Success Events that are critical to your site, not every Success Event. So there is your top 10 list of culprits.  Omniture is constantly looking at what it can do to improve performance and hopes that many of the items above will be less resource intensive in the future, please keep these items in mind when talking to your Account Manager or ClientCare. Have a question about anything related to Omniture SiteCatalyst?  Is there something on your website that you would like to report on, but don't know how?  Do you have any tips or best practices you want to share?  If so, please leave a comment here or send me an e-mail at insidesitecatalyst@omniture.com and I will do my best to answer it right here on the blog so everyone can learn! (Don't worry - I won't use your name or company name!).  If you are on Twitter, you can follow me at http://twitter.com/Omni_man.
Learn more about Omniture Consulting
Learn more about Omniture University
 
 
 
Author: Date Created:March 15, 2009 Date Published: Headline:Top Ten Tips to Minimize Data Latency [Inside Omniture SiteCatalyst] Social Counts: Keywords: Publisher:Adobe Image:https://blogs.adobe.com/digitalmarketing/wp-content/uploads/no-image/no-image.jpg

One of the many reasons customers choose Omniture is our real-time reporting capabilities to provide that rapid insight needed to make decisions in a fast-paced online business environment.   Our engineering teams work hard to ensure we consistently deliver this capability, but there are rare occasions where latency has presented challenges for some customers.  It’s important to note with any latency issue that no customer data is lost and complete data integrity is maintained.  While we’ve found that recent latency issues have only affected a minority of customers, it brought to light some other potential contributing factors that you can control such as using too many server-intensive features within a report suite.  These are things that you can do something about, so in this post I will explain steps you can take to help minimize data latency.

A Little History…
As I mentioned, latency can be harder to recover from if you’ve enabled certain SiteCatalyst features.  Some SiteCatalyst features do not have much of an impact and others do.  Unfortunately, many clients do not know which features are resource intensive and which are not.  In the old, old days (when I used SiteCatalyst version 9), if you wanted to turn on a feature, such as a 2-item Traffic Data Correlation, you would need to contact Omniture and pay for it in an ad-hoc manner.  In fact, almost everything you wanted to do meant a phone call, an addendum and an account manager.  Customers complained that it took too much time and that Omniture was “nickel and diming” its customers so Omniture moved to the Administration Console model where customers were empowered to turn many things on and off as desired.  In fact, many customers ask if ALL functions can be added to the Administration Console, but doing this could allow customers to enable things that could produce latency.

Finally, before digging into the details, let me be clear that it is our intention for Omniture customers to use all of the features SiteCatalyst provides.  While the items below may cause you to think about using features I have previously raved about on this blog, keep in mind that I am providing this information only so that you understand what taxes the system the most.  If you have a good business case for using any of these features, please do so.  We often find that Omniture customers have enabled features that are both resource-intensive and not actually being used, causing a lose-lose situation. When I was a customer, I would review my implementation a few times a year and find out if any of my users were actively using the features that were enabled and take action accordingly…

#1 – Full Subrelations
While Conversion Variable Subrelations are extremely powerful, it is important that you plan ahead and think about what eVars for which you need full subrelations since they create an enormous amount of overhead.  If you think about it, if you have 10 eVars enabled and then turn on full subrelations for one eVar, you are instructing SiteCatalyst to create nine new data cubes, all of which have to be updated every time a Success Event takes place.  For those of you that have access to Omniture Discover, I would encourage you to leverage this tool for advanced variable breakdowns instead since it is built for this capability.  Some other solutions to this are to concatenate values to one eVar and use Classifications to split them out or to use a Traffic Data Correlation which is much less resource intensive.

#2 – Unused Conversion Variables
Did you know that even if you are not passing any data to a Conversion Variable (eVar) it is increasing your processing load?  This is due to the fact that every time a Success Event takes place, it is associated to every enabled eVar, even if the value associated with it is “None.”  Therefore, when I see clients that have 20 eVars enabled, but are only actively using 2-3 of them I beg them to disable the eVars that are not being used so they can improve performance.

#3 – Multiple Success Events on a Page
Another item that can impact latency is the setting of multiple Success Events on a page.  If you set two Success Events on a page, you are essentially doubling all of the data processing mentioned above on those pages.

#4 – Traffic Spikes
While it is not possible to anticipate every traffic spike your site will have, there are many cases in which you know that you will receive more traffic such as holiday periods or via seasonality.  SiteCatalyst provides a way for you to tell Omniture when you expect a traffic spike so we can be prepared for it by adding additional resources.  This notification system can be found in the Administration Console:

#5 – Participation
A few weeks ago, I explained Participation in a post.  While Participation is very powerful, you probably do not need to have Participation enabled for every Success Event.  For example, if you have a five step conversion funnel and set a Success Event for each step, you could probably be ok with enabling Participation on the first and last step without losing much insight.  Also, once you use Participation to learn about what pages lead to your site Success Events, it may be ok to disable Participation and re-enable it later in the year or after a site/page redesign since the data may not change drastically from month to month.

#6 – Visits & Visitors in Conversion Reports
A frequent request from customers is to allow Visits and Unique Visitors in Conversion reports so they can be used in Calculated metrics.  I often solve this problem by pulling data into the ExcelClient where I can compare Traffic Visits for a day to Conversion Success Events for the same date.  However, if this functionality is critical to your business I suggest you talk to your Account Manager or Omniture Consulting.

#7 – Complex VISTA Rules
When writing VISTA rules, sometimes there is a tendency to throw everything in (including the kitchen sink!).  The more complex you make VISTA rules, the slower data may process.

#8 – Unique Values
There are some Traffic Variables (sProps) and Conversion Variables (eVars) that end up having very few unique values and others that have millions of unique values.  For example, an eVar capturing the user’s birth year will have a relatively finite number of values, whereas an eVar capturing unique User ID’s for a site like Facebook, would have millions of unique values.  While this does not impact processing speed, it can impact reporting speed.  For this reason, you need to think about how many unique values you expect a variable to have when setting up Conversion Variable Subrelations and Traffic Data Correlations.  Many times, customers will use a VISTA rule to “blank out” variables in the SiteCatalyst interface that have over 500,000 unique values per month and leverage the DataWarehouse for these data points.

#9 – Products Variable
In a past post I mentioned different ways that you could use the Products Variable in SiteCatalyst.  Since the Products Variable is currently one of the only Conversion Variables that allows you to pass in multiple values, I have seen some clients be very creative with its use.  However, passing in more than ten items to the Products variable can cause problems so if you have a need for storing many items in the products variable, I suggest you speak to your Account Manager or Omniture Consulting.

#10 – Event Serialization
In a recent post I explained how you can use Event Serialization to de-duplicate Success Events.  However, if you have millions of Success Events taking place on your site, using Event Serialization can become a resource issue.  It is best used for Success Events that are critical to your site, not every Success Event.

So there is your top 10 list of culprits.  Omniture is constantly looking at what it can do to improve performance and hopes that many of the items above will be less resource intensive in the future, please keep these items in mind when talking to your Account Manager or ClientCare.

Have a question about anything related to Omniture SiteCatalyst?  Is there something on your website that you would like to report on, but don’t know how?  Do you have any tips or best practices you want to share?  If so, please leave a comment here or send me an e-mail at insidesitecatalyst@omniture.com and I will do my best to answer it right here on the blog so everyone can learn! (Don’t worry – I won’t use your name or company name!).  If you are on Twitter, you can follow me at http://twitter.com/Omni_man.

Learn more about Omniture Consulting
Learn more about Omniture University