While more and more busi­ness is tak­ing place on the web each day, we online mar­keters can­not for­get about suc­cesses and/or con­ver­sions that take place beyond the web­site.  While Omni­ture Site­Cat­a­lyst can cap­ture all of your online suc­cess, there are times when you need to cou­ple this with busi­ness met­rics that take place offline or in other chan­nels.  To do this, Omni­ture Site­Cat­a­lyst pro­vides a Data Sources fea­ture.  In this post I will briefly explain what Data Sources is and how it can be used.

Under­stand­ing Data Sources
So what is Data Sources?  Data Sources is a mech­a­nism to man­u­ally import addi­tional met­rics and met­ric dimen­sions into Site­Cat­a­lyst.  The pri­mary rea­son that cus­tomers choose to import this data is to com­pare online met­rics to these non-website met­rics.  The fol­low­ing are some exam­ples of sit­u­a­tions in which you might want to use Data Sources to import data:

  1. Import his­tor­i­cal traf­fic data from log files or a pre­vi­ous vendor
  2. Import offline suc­cess met­rics unre­lated to the web­site or tak­ing place after a visit to the website
  3. Import met­rics from a sep­a­rate chan­nel such as e-mail or call cen­ter so you can com­pare them to online met­rics and cre­ate cal­cu­lated metrics
  4. Import met­rics from other prod­ucts through Omni­ture Gen­e­sis such as e-mail or CRM met­rics (most Gen­e­sis inte­gra­tions use Data Sources behind the scenes)

In all of these cases, clients are look­ing to sup­ple­ment their online data with related data so they can get a more com­plete pic­ture of their busi­ness.  I find a good way to under­stand Data Sources is through a prac­ti­cal exam­ple.  Let’s say that an Omni­ture cus­tomer decides to start an e-mail mar­ket­ing pro­gram.  They select a ven­dor and the ven­dor hap­pens to be part of Omniture’s Gen­e­sis net­work.  As part of the Gen­e­sis inte­gra­tion, the client begins receiv­ing some new met­rics in Site­Cat­a­lyst via Data Sources includ­ing E-mail Sends, E-mail Opens, etc…  Each of these met­rics appears along­side exist­ing online Suc­cess Events.  There­fore, the client can view key e-mail sta­tis­tics from within Site­Cat­a­lyst with­out hav­ing to look two places.  Finally, the client can cre­ate a cal­cu­lated met­ric that divides Sub­mit­ted Online Leads by E-mail Sends and many oth­ers.  All of this is accom­plished through Data Sources.

How Do I Enable Data Sources?
While I won’t go through the detailed steps required to set-up Data Sources, the fol­low­ing is a brief overview of the process of set­ting up a Data Source:

  1. Most Data Source imports use Incre­men­tor Suc­cess Events so you must cre­ate these ahead of time through the Admin­is­tra­tion Con­sole.  You will need one Suc­cess Event for every met­ric that you plan to import
  2. Cre­ate any desired Con­ver­sion Vari­ables (eVars).  In addi­tion to import­ing met­rics, you can option­ally choose to import eVar val­ues asso­ci­ated with those met­rics.  For exam­ple, if you track online activ­ity with a Cam­paign Track­ing Code, and have Cam­paign Track­ing Codes for the offline met­rics, you can import the met­rics with Cam­paign Track­ing Codes.  This will allow you to view both online and offline met­rics in Cam­paign reports.  If you do not import Data Sources met­rics with an asso­ci­ated eVar value, you will not be able to view Data Source met­rics bro­ken down by eVars, but rather, will only be able to see total met­rics (please re-read as this con­fuses many clients).
  3. Go through the Data Sources wiz­ard which will help you to cre­ate a Data Sources file import tem­plate and FTP site.  As you go through the wiz­ard you will need to know the Suc­cess Events and eVars for which you want to import data.  Please note that the wiz­ard only allows you to select a few Suc­cess Event Met­rics and a few eVar dimen­sions which can be con­fus­ing, but rest assured that you can add more met­rics and eVars directly to the tem­plate after­wards (Omni­ture Con­sult­ing or Client­Care can show you how to do this).
  4. Once you have the tem­plate and the FTP site, you are ready to import data.  Most clients will do the first few uploads man­u­ally and then find a way to auto­mate the FTP upload process on a daily or weekly basis.

Impor­tant Things To Know About Data Sources
The fol­low­ing are some impor­tant things to know about Data Sources:

  1. Once you import Data Sources data into Site­Cat­a­lyst, it can­not be undone or removed.  For this rea­son, you need to be very care­ful and make sure you have done thor­ough test­ing and have a good process in place for auto­mated data uploads.  Always test within a devel­op­ment report suite!
  2. All data imported via Data Sources is tied to a date so it does not mat­ter when it is imported.  Obvi­ously, the sooner it is imported, the more accu­rate your over­all Site­Cat­a­lyst data set will be
  3. You can­not have more than 50 Megabytes of data in your Data Sources FTP account at a time (not 50MB per file, but in total across all files) or it will become “locked” so it is rec­om­mended that you feed the data to Site­Cat­a­lyst accordingly
  4. Each row of data imported via Data Sources is charged ($$) as an image request (“hit”) at your con­tracted rate
  5. Some types of Data Source imports allow you to see the imported data in Site­Cat­a­lyst only, while oth­ers allow you to see imported data in Site­Cat­a­lyst, DataWare­house, ASI and Dis­cover.  For this rea­son it is impor­tant to be sure you select the cor­rect Data Source Type (for more infor­ma­tion read the Data Sources user man­ual found in Knowl­edge Base arti­cle # 527)

Real-World Exam­ple
In this week’s real-world exam­ple, we will look at a sce­nario for one of Greco Inc.‘s elec­tron­ics sub­sidiaries.  In this situation, Greco Inc. is doing its best to sell print­ers online, but at the time that the sale is com­pleted, it is not able to accu­rately deter­mine the exact ship­ping cost, which can vary based upon a few post-sale fac­tors.  In addi­tion, Greco Inc. would like to add the Cost of Good Sold to Site­Cat­a­lyst so it can cre­ate some more real­is­tic cal­cu­lated met­rics for judg­ing Cam­paign suc­cess and other analyses.

To accom­plish these two goals, Greco Inc. sets up two new Incre­men­tor Suc­cess Events (COGS & Ship­ping Costs) and then deter­mines how it would like to break down these new offline met­rics.  For its online sales, Greco Inc. does most of its analy­sis by Track­ing Code, Order Num­ber, Brand, Prod­uct Type and Coupon (if used).  It already has eVars cre­ated for these and reg­u­larly looks at online met­rics such as Orders, Units and Rev­enue by these dimen­sions.  After going through the Data Sources wiz­ard, Greco Inc. has an FTP account set-up and the fol­low­ing Data Sources import tem­plate (shown here with one sam­ple line of offline data):

Once all test­ing is done, Greco Inc. can begin import­ing the offline data via Data Sources and can then see a Site­Cat­a­lyst report that includes both online and offline data:

There is a great deal more to learn about Data Sources includ­ing dif­fer­ent types (Inte­gra­tion, Full Pro­cess­ing, etc…), but these are more advanced top­ics to be saved for a future post.  If you are inter­ested in dig­ging into Data Sources, Omni­ture pro­vides a spe­cial user man­ual on the topic (Knowl­edge Base arti­cle # 527) which I sug­gest you check out.


Have a ques­tion about any­thing related to Omni­ture Site­Cat­a­lyst?  Is there some­thing on your web­site that you would like to report on, but don’t know how?  Do you have any tips or best prac­tices you want to share?  If so, please leave a com­ment here or send me an e-mail at insidesitecatalyst@​omniture.​com and I will do my best to answer it right here on the blog so every­one can learn! (Don’t worry — I won’t use your name or com­pany name!).  If you are on Twit­ter, you can fol­low me at http://​twit​ter​.com/​O​m​n​i​_​man.

Learn more about Omni­ture Consulting
Learn more about Omni­ture University
Daniel Lum
Daniel Lum

I'm curious about the integration of data through site catalyst. Are there templates or data structure examples that I can use to assemble my off-line data into the best format for integration with my online files? Seeing as I cannot undo an upload I'd like to know the import will be exactly how I need it structured so I can work with it.


I uploaded a row using the Data Source yesterday, this row refer to 14th November 2010. I used the template TransactionID for being sure to have the data also in Discover BUT in Discover this data is not visible. After a long debate with the ClientCare, I have finally understood that for seeing the data in Discover I have to upload the data with in 48 hrs of the day I want it associated to. This is due to the fact Discover does not reprocess historical data, but only the last 48 hours. So, in my opinion this is a big issue.


Thanks Ben. Much appreciated for a detailed response!

Ben Gaines
Ben Gaines

Hi Abhilash. Based on your explanation, it sounds like you're basically trying to capture COGS (Cost of Goods Sold) information in SiteCatalyst. This is a fairly common need, and either of the two methods you mentioned would work. 1. Using SAINT, you would set up a numeric classification and then set the price per unit in that classification column, with the various product IDs being the key values. COGS would then be available in Products reports as a metric. 2. Data Sources might be a bit more complex because you would have to upload (periodically) the aggregate COGS for each product each day. Certainly possible, though. There is also a third option—certainly the easiest of the three to implement: 3. Engineering Services offers a COGS VISTA rule which allows you to upload a list of product IDs and unit costs so that SiteCatalyst can apply the unit costs in a custom event automatically on each purchase.


Hi Ben, I have a list of products on the page and I don't want to display or pass the price of them over the web. Each product has a unique Id. The options I am looking for are 1. Create a SAINT classification with Product ID and Price, and pass the product ID each time when my product is viewed and have Site Catalyst to generate a report for revenue per product. 2. Use Data Source - I am not sure how this works. But the ideal way I would like to see this working is to send list of product Id's and price to Omniture as a daily feed and pass the product ID each time when my product is viewed Can you please suggest the best approach? (I would prefer Option 1 if that can work) Thanks much Abhilash


Excellent! That makes complete sense. Thanks for your help!

Ben Gaines
Ben Gaines

Sid: Great question. Negative values are not supported in Data Sources uploads. What we recommend instead is to upload reconciliation data into an available custom event using Data Sources, and then to subtract that event from the affected metric. For example, [Adjusted Revenue] = [Revenue] - [Billing Reconciliation] Here, you'd be uploading data into the "Billing Reconciliation" event using Data Sources and then subtracting it from Revenue to get the number you're looking for. I hope this helps!


I'm just looking around at Data Sources info, and wondering about the import of "negative" data. All the documentation talks about each import being additive on the last if you were to make a mistake and reimport. Could you reimport a negative value to subtract the incorrect data? Say it was spend data after a billing reconciliation?