While more and more business is taking place on the web each day, we online marketers cannot forget about successes and/or conversions that take place beyond the website.  While Omniture SiteCatalyst can capture all of your online success, there are times when you need to couple this with business metrics that take place offline or in other channels.  To do this, Omniture SiteCatalyst provides a Data Sources feature.  In this post I will briefly explain what Data Sources is and how it can be used.

Understanding Data Sources
So what is Data Sources?  Data Sources is a mechanism to manually import additional metrics and metric dimensions into SiteCatalyst.  The primary reason that customers choose to import this data is to compare online metrics to these non-website metrics.  The following are some examples of situations in which you might want to use Data Sources to import data:

  1. Import historical traffic data from log files or a previous vendor
  2. Import offline success metrics unrelated to the website or taking place after a visit to the website
  3. Import metrics from a separate channel such as e-mail or call center so you can compare them to online metrics and create calculated metrics
  4. Import metrics from other products through Omniture Genesis such as e-mail or CRM metrics (most Genesis integrations use Data Sources behind the scenes)

In all of these cases, clients are looking to supplement their online data with related data so they can get a more complete picture of their business.  I find a good way to understand Data Sources is through a practical example.  Let’s say that an Omniture customer decides to start an e-mail marketing program.  They select a vendor and the vendor happens to be part of Omniture’s Genesis network.  As part of the Genesis integration, the client begins receiving some new metrics in SiteCatalyst via Data Sources including E-mail Sends, E-mail Opens, etc…  Each of these metrics appears alongside existing online Success Events.  Therefore, the client can view key e-mail statistics from within SiteCatalyst without having to look two places.  Finally, the client can create a calculated metric that divides Submitted Online Leads by E-mail Sends and many others.  All of this is accomplished through Data Sources.

How Do I Enable Data Sources?
While I won’t go through the detailed steps required to set-up Data Sources, the following is a brief overview of the process of setting up a Data Source:

  1. Most Data Source imports use Incrementor Success Events so you must create these ahead of time through the Administration Console.  You will need one Success Event for every metric that you plan to import
  2. Create any desired Conversion Variables (eVars).  In addition to importing metrics, you can optionally choose to import eVar values associated with those metrics.  For example, if you track online activity with a Campaign Tracking Code, and have Campaign Tracking Codes for the offline metrics, you can import the metrics with Campaign Tracking Codes.  This will allow you to view both online and offline metrics in Campaign reports.  If you do not import Data Sources metrics with an associated eVar value, you will not be able to view Data Source metrics broken down by eVars, but rather, will only be able to see total metrics (please re-read as this confuses many clients).
  3. Go through the Data Sources wizard which will help you to create a Data Sources file import template and FTP site.  As you go through the wizard you will need to know the Success Events and eVars for which you want to import data.  Please note that the wizard only allows you to select a few Success Event Metrics and a few eVar dimensions which can be confusing, but rest assured that you can add more metrics and eVars directly to the template afterwards (Omniture Consulting or ClientCare can show you how to do this).
  4. Once you have the template and the FTP site, you are ready to import data.  Most clients will do the first few uploads manually and then find a way to automate the FTP upload process on a daily or weekly basis.

Important Things To Know About Data Sources
The following are some important things to know about Data Sources:

  1. Once you import Data Sources data into SiteCatalyst, it cannot be undone or removed.  For this reason, you need to be very careful and make sure you have done thorough testing and have a good process in place for automated data uploads.  Always test within a development report suite!
  2. All data imported via Data Sources is tied to a date so it does not matter when it is imported.  Obviously, the sooner it is imported, the more accurate your overall SiteCatalyst data set will be
  3. You cannot have more than 50 Megabytes of data in your Data Sources FTP account at a time (not 50MB per file, but in total across all files) or it will become “locked” so it is recommended that you feed the data to SiteCatalyst accordingly
  4. Each row of data imported via Data Sources is charged ($$) as an image request (“hit”) at your contracted rate
  5. Some types of Data Source imports allow you to see the imported data in SiteCatalyst only, while others allow you to see imported data in SiteCatalyst, DataWarehouse, ASI and Discover.  For this reason it is important to be sure you select the correct Data Source Type (for more information read the Data Sources user manual found in Knowledge Base article # 527)

Real-World Example
In this week’s real-world example, we will look at a scenario for one of Greco Inc.’s electronics subsidiaries.  In this situation, Greco Inc. is doing its best to sell printers online, but at the time that the sale is completed, it is not able to accurately determine the exact shipping cost, which can vary based upon a few post-sale factors.  In addition, Greco Inc. would like to add the Cost of Good Sold to SiteCatalyst so it can create some more realistic calculated metrics for judging Campaign success and other analyses.

To accomplish these two goals, Greco Inc. sets up two new Incrementor Success Events (COGS & Shipping Costs) and then determines how it would like to break down these new offline metrics.  For its online sales, Greco Inc. does most of its analysis by Tracking Code, Order Number, Brand, Product Type and Coupon (if used).  It already has eVars created for these and regularly looks at online metrics such as Orders, Units and Revenue by these dimensions.  After going through the Data Sources wizard, Greco Inc. has an FTP account set-up and the following Data Sources import template (shown here with one sample line of offline data):

Once all testing is done, Greco Inc. can begin importing the offline data via Data Sources and can then see a SiteCatalyst report that includes both online and offline data:

There is a great deal more to learn about Data Sources including different types (Integration, Full Processing, etc…), but these are more advanced topics to be saved for a future post.  If you are interested in digging into Data Sources, Omniture provides a special user manual on the topic (Knowledge Base article # 527) which I suggest you check out.

 

Have a question about anything related to Omniture SiteCatalyst?  Is there something on your website that you would like to report on, but don’t know how?  Do you have any tips or best practices you want to share?  If so, please leave a comment here or send me an e-mail at insidesitecatalyst@omniture.com and I will do my best to answer it right here on the blog so everyone can learn! (Don’t worry – I won’t use your name or company name!).  If you are on Twitter, you can follow me at http://twitter.com/Omni_man.

Learn more about Omniture Consulting
Learn more about Omniture University
9 comments
Daniel Lum
Daniel Lum

I'm curious about the integration of data through site catalyst. Are there templates or data structure examples that I can use to assemble my off-line data into the best format for integration with my online files? Seeing as I cannot undo an upload I'd like to know the import will be exactly how I need it structured so I can work with it.

Moira
Moira

I uploaded a row using the Data Source yesterday, this row refer to 14th November 2010. I used the template TransactionID for being sure to have the data also in Discover BUT in Discover this data is not visible. After a long debate with the ClientCare, I have finally understood that for seeing the data in Discover I have to upload the data with in 48 hrs of the day I want it associated to. This is due to the fact Discover does not reprocess historical data, but only the last 48 hours. So, in my opinion this is a big issue.

Abhilash
Abhilash

Thanks Ben. Much appreciated for a detailed response!

Ben Gaines
Ben Gaines

Hi Abhilash. Based on your explanation, it sounds like you're basically trying to capture COGS (Cost of Goods Sold) information in SiteCatalyst. This is a fairly common need, and either of the two methods you mentioned would work. 1. Using SAINT, you would set up a numeric classification and then set the price per unit in that classification column, with the various product IDs being the key values. COGS would then be available in Products reports as a metric. 2. Data Sources might be a bit more complex because you would have to upload (periodically) the aggregate COGS for each product each day. Certainly possible, though. There is also a third option—certainly the easiest of the three to implement: 3. Engineering Services offers a COGS VISTA rule which allows you to upload a list of product IDs and unit costs so that SiteCatalyst can apply the unit costs in a custom event automatically on each purchase.

Abhilash
Abhilash

Hi Ben, I have a list of products on the page and I don't want to display or pass the price of them over the web. Each product has a unique Id. The options I am looking for are 1. Create a SAINT classification with Product ID and Price, and pass the product ID each time when my product is viewed and have Site Catalyst to generate a report for revenue per product. 2. Use Data Source - I am not sure how this works. But the ideal way I would like to see this working is to send list of product Id's and price to Omniture as a daily feed and pass the product ID each time when my product is viewed Can you please suggest the best approach? (I would prefer Option 1 if that can work) Thanks much Abhilash

Sid
Sid

Excellent! That makes complete sense. Thanks for your help!

Ben Gaines
Ben Gaines

Sid: Great question. Negative values are not supported in Data Sources uploads. What we recommend instead is to upload reconciliation data into an available custom event using Data Sources, and then to subtract that event from the affected metric. For example, [Adjusted Revenue] = [Revenue] - [Billing Reconciliation] Here, you'd be uploading data into the "Billing Reconciliation" event using Data Sources and then subtracting it from Revenue to get the number you're looking for. I hope this helps!

Sid
Sid

I'm just looking around at Data Sources info, and wondering about the import of "negative" data. All the documentation talks about each import being additive on the last if you were to make a mistake and reimport. Could you reimport a negative value to subtract the incorrect data? Say it was spend data after a billing reconciliation?