Data Sources [Inside Omniture SiteCatalyst]

Analytics · By On · 8 Comments

While more and more busi­ness is tak­ing place on the web each day, we online mar­keters can­not for­get about suc­cesses and/or con­ver­sions that take place beyond the web­site.  While Omni­ture Site­Cat­a­lyst can cap­ture all of your online suc­cess, there are times when you need to cou­ple this with busi­ness met­rics that take place offline or in other chan­nels.  To do this, Omni­ture Site­Cat­a­lyst pro­vides a Data Sources fea­ture.  In this post I will briefly explain what Data Sources is and how it can be used.

Under­stand­ing Data Sources
So what is Data Sources?  Data Sources is a mech­a­nism to man­u­ally import addi­tional met­rics and met­ric dimen­sions into Site­Cat­a­lyst.  The pri­mary rea­son that cus­tomers choose to import this data is to com­pare online met­rics to these non-website met­rics.  The fol­low­ing are some exam­ples of sit­u­a­tions in which you might want to use Data Sources to import data:

  1. Import his­tor­i­cal traf­fic data from log files or a pre­vi­ous vendor
  2. Import offline suc­cess met­rics unre­lated to the web­site or tak­ing place after a visit to the website
  3. Import met­rics from a sep­a­rate chan­nel such as e-mail or call cen­ter so you can com­pare them to online met­rics and cre­ate cal­cu­lated metrics
  4. Import met­rics from other prod­ucts through Omni­ture Gen­e­sis such as e-mail or CRM met­rics (most Gen­e­sis inte­gra­tions use Data Sources behind the scenes)

In all of these cases, clients are look­ing to sup­ple­ment their online data with related data so they can get a more com­plete pic­ture of their busi­ness.  I find a good way to under­stand Data Sources is through a prac­ti­cal exam­ple.  Let’s say that an Omni­ture cus­tomer decides to start an e-mail mar­ket­ing pro­gram.  They select a ven­dor and the ven­dor hap­pens to be part of Omniture’s Gen­e­sis net­work.  As part of the Gen­e­sis inte­gra­tion, the client begins receiv­ing some new met­rics in Site­Cat­a­lyst via Data Sources includ­ing E-mail Sends, E-mail Opens, etc…  Each of these met­rics appears along­side exist­ing online Suc­cess Events.  There­fore, the client can view key e-mail sta­tis­tics from within Site­Cat­a­lyst with­out hav­ing to look two places.  Finally, the client can cre­ate a cal­cu­lated met­ric that divides Sub­mit­ted Online Leads by E-mail Sends and many oth­ers.  All of this is accom­plished through Data Sources.

How Do I Enable Data Sources?
While I won’t go through the detailed steps required to set-up Data Sources, the fol­low­ing is a brief overview of the process of set­ting up a Data Source:

  1. Most Data Source imports use Incre­men­tor Suc­cess Events so you must cre­ate these ahead of time through the Admin­is­tra­tion Con­sole.  You will need one Suc­cess Event for every met­ric that you plan to import
  2. Cre­ate any desired Con­ver­sion Vari­ables (eVars).  In addi­tion to import­ing met­rics, you can option­ally choose to import eVar val­ues asso­ci­ated with those met­rics.  For exam­ple, if you track online activ­ity with a Cam­paign Track­ing Code, and have Cam­paign Track­ing Codes for the offline met­rics, you can import the met­rics with Cam­paign Track­ing Codes.  This will allow you to view both online and offline met­rics in Cam­paign reports.  If you do not import Data Sources met­rics with an asso­ci­ated eVar value, you will not be able to view Data Source met­rics bro­ken down by eVars, but rather, will only be able to see total met­rics (please re-read as this con­fuses many clients).
  3. Go through the Data Sources wiz­ard which will help you to cre­ate a Data Sources file import tem­plate and FTP site.  As you go through the wiz­ard you will need to know the Suc­cess Events and eVars for which you want to import data.  Please note that the wiz­ard only allows you to select a few Suc­cess Event Met­rics and a few eVar dimen­sions which can be con­fus­ing, but rest assured that you can add more met­rics and eVars directly to the tem­plate after­wards (Omni­ture Con­sult­ing or Client­Care can show you how to do this).
  4. Once you have the tem­plate and the FTP site, you are ready to import data.  Most clients will do the first few uploads man­u­ally and then find a way to auto­mate the FTP upload process on a daily or weekly basis.

Impor­tant Things To Know About Data Sources
The fol­low­ing are some impor­tant things to know about Data Sources:

  1. Once you import Data Sources data into Site­Cat­a­lyst, it can­not be undone or removed.  For this rea­son, you need to be very care­ful and make sure you have done thor­ough test­ing and have a good process in place for auto­mated data uploads.  Always test within a devel­op­ment report suite!
  2. All data imported via Data Sources is tied to a date so it does not mat­ter when it is imported.  Obvi­ously, the sooner it is imported, the more accu­rate your over­all Site­Cat­a­lyst data set will be
  3. You can­not have more than 50 Megabytes of data in your Data Sources FTP account at a time (not 50MB per file, but in total across all files) or it will become “locked” so it is rec­om­mended that you feed the data to Site­Cat­a­lyst accordingly
  4. Each row of data imported via Data Sources is charged ($$) as an image request (“hit”) at your con­tracted rate
  5. Some types of Data Source imports allow you to see the imported data in Site­Cat­a­lyst only, while oth­ers allow you to see imported data in Site­Cat­a­lyst, DataWare­house, ASI and Dis­cover.  For this rea­son it is impor­tant to be sure you select the cor­rect Data Source Type (for more infor­ma­tion read the Data Sources user man­ual found in Knowl­edge Base arti­cle # 527)

Real-World Exam­ple
In this week’s real-world exam­ple, we will look at a sce­nario for one of Greco Inc.‘s elec­tron­ics sub­sidiaries.  In this situation, Greco Inc. is doing its best to sell print­ers online, but at the time that the sale is com­pleted, it is not able to accu­rately deter­mine the exact ship­ping cost, which can vary based upon a few post-sale fac­tors.  In addi­tion, Greco Inc. would like to add the Cost of Good Sold to Site­Cat­a­lyst so it can cre­ate some more real­is­tic cal­cu­lated met­rics for judg­ing Cam­paign suc­cess and other analyses.

To accom­plish these two goals, Greco Inc. sets up two new Incre­men­tor Suc­cess Events (COGS & Ship­ping Costs) and then deter­mines how it would like to break down these new offline met­rics.  For its online sales, Greco Inc. does most of its analy­sis by Track­ing Code, Order Num­ber, Brand, Prod­uct Type and Coupon (if used).  It already has eVars cre­ated for these and reg­u­larly looks at online met­rics such as Orders, Units and Rev­enue by these dimen­sions.  After going through the Data Sources wiz­ard, Greco Inc. has an FTP account set-up and the fol­low­ing Data Sources import tem­plate (shown here with one sam­ple line of offline data):

Once all test­ing is done, Greco Inc. can begin import­ing the offline data via Data Sources and can then see a Site­Cat­a­lyst report that includes both online and offline data:

There is a great deal more to learn about Data Sources includ­ing dif­fer­ent types (Inte­gra­tion, Full Pro­cess­ing, etc…), but these are more advanced top­ics to be saved for a future post.  If you are inter­ested in dig­ging into Data Sources, Omni­ture pro­vides a spe­cial user man­ual on the topic (Knowl­edge Base arti­cle # 527) which I sug­gest you check out.

 

Have a ques­tion about any­thing related to Omni­ture Site­Cat­a­lyst?  Is there some­thing on your web­site that you would like to report on, but don’t know how?  Do you have any tips or best prac­tices you want to share?  If so, please leave a com­ment here or send me an e-mail at insidesitecatalyst@​omniture.​com and I will do my best to answer it right here on the blog so every­one can learn! (Don’t worry — I won’t use your name or com­pany name!).  If you are on Twit­ter, you can fol­low me at http://​twit​ter​.com/​O​m​n​i​_​man.

Learn more about Omni­ture Consulting
Learn more about Omni­ture University
Tagged with → • •  
  • Sid

    I’m just look­ing around at Data Sources info, and won­der­ing about the import of “neg­a­tive” data. All the doc­u­men­ta­tion talks about each import being addi­tive on the last if you were to make a mis­take and reim­port. Could you reim­port a neg­a­tive value to sub­tract the incor­rect data? Say it was spend data after a billing reconciliation?

  • http://blogs.omniture.com/author/bgaines Ben Gaines

    Sid: Great ques­tion. Neg­a­tive val­ues are not sup­ported in Data Sources uploads. What we rec­om­mend instead is to upload rec­on­cil­i­a­tion data into an avail­able cus­tom event using Data Sources, and then to sub­tract that event from the affected met­ric. For example,

    [Adjusted Rev­enue] = [Rev­enue] — [Billing Reconciliation]

    Here, you’d be upload­ing data into the “Billing Rec­on­cil­i­a­tion” event using Data Sources and then sub­tract­ing it from Rev­enue to get the num­ber you’re look­ing for. I hope this helps!

  • Sid

    Excel­lent! That makes com­plete sense. Thanks for your help!

  • http://Trip.com Abhi­lash

    Hi Ben,
    I have a list of prod­ucts on the page and I don’t want to dis­play or pass the price of them over the web. Each prod­uct has a unique Id. The options I am look­ing for are
    1. Cre­ate a SAINT clas­si­fi­ca­tion with Prod­uct ID and Price, and pass the prod­uct ID each time when my prod­uct is viewed and have Site Cat­a­lyst to gen­er­ate a report for rev­enue per prod­uct.
    2. Use Data Source — I am not sure how this works. But the ideal way I would like to see this work­ing is to send list of prod­uct Id’s and price to Omni­ture as a daily feed and pass the prod­uct ID each time when my prod­uct is viewed

    Can you please sug­gest the best approach? (I would pre­fer Option 1 if that can work)

    Thanks much
    Abhilash

  • http://blogs.omniture.com/author/bgaines Ben Gaines

    Hi Abhi­lash. Based on your expla­na­tion, it sounds like you’re basi­cally try­ing to cap­ture COGS (Cost of Goods Sold) infor­ma­tion in Site­Cat­a­lyst. This is a fairly com­mon need, and either of the two meth­ods you men­tioned would work.

    1. Using SAINT, you would set up a numeric clas­si­fi­ca­tion and then set the price per unit in that clas­si­fi­ca­tion col­umn, with the var­i­ous prod­uct IDs being the key val­ues. COGS would then be avail­able in Prod­ucts reports as a metric.

    2. Data Sources might be a bit more com­plex because you would have to upload (peri­od­i­cally) the aggre­gate COGS for each prod­uct each day. Cer­tainly pos­si­ble, though.

    There is also a third option—certainly the eas­i­est of the three to implement:

    3. Engi­neer­ing Ser­vices offers a COGS VISTA rule which allows you to upload a list of prod­uct IDs and unit costs so that Site­Cat­a­lyst can apply the unit costs in a cus­tom event auto­mat­i­cally on each purchase.

  • http://Trip.com Abhi­lash

    Thanks Ben.
    Much appre­ci­ated for a detailed response!

  • Moira

    I uploaded a row using the Data Source yes­ter­day, this row refer to 14th Novem­ber 2010. I used the tem­plate Trans­ac­tionID for being sure to have the data also in Dis­cover BUT in Dis­cover this data is not vis­i­ble. After a long debate with the Client­Care, I have finally under­stood that for see­ing the data in Dis­cover I have to upload the data with in 48 hrs of the day I want it asso­ci­ated to. This is due to the fact Dis­cover does not reprocess his­tor­i­cal data, but only the last 48 hours. So, in my opin­ion this is a big issue.

  • http://www.sudlows.com/ Daniel Lum

    I’m curi­ous about the inte­gra­tion of data through site cat­a­lyst. Are there tem­plates or data struc­ture exam­ples that I can use to assem­ble my off-line data into the best for­mat for inte­gra­tion with my online files? See­ing as I can­not undo an upload I’d like to know the import will be exactly how I need it struc­tured so I can work with it.