In my last post, I dis­cussed how you can use the Omni­ture Data Inser­tion API along with the Twit­ter Search API to pass Twit­ter data into Site­Cat­a­lyst. Based on con­ver­sa­tions with a few read­ers, it seems that those of you who dove in and began devel­op­ing after read­ing Adam Greco’s post last week were think­ing along those same lines.

How­ever, there is an impor­tant alter­na­tive method­ol­ogy to allow you to insert Twit­ter data (“tweets”) regard­ing your brand into Site­Cat­a­lyst, and it lever­ages the new Omni­ture SAINT API, which allows you to upload clas­si­fi­ca­tion data into Site­Cat­a­lyst on the fly.

Why use this method? Instead of con­sum­ing four or five of the max­i­mum 50 eVar vari­ables in your report suite(s), it uses a sin­gle eVar. Fur­ther­more, full sub­re­la­tions are not required in order to break down one type of Twit­ter data, such as tweet author, by another. Another key advan­tage is that SAINT clas­si­fi­ca­tions can be changed retroac­tively, which means that using SAINT allows you to add addi­tional clas­si­fi­ca­tions, such as tweet cat­e­gory (e.g. com­plaint, praise, sup­port request, etc.) quickly and eas­ily, and apply it to your pre­vi­ously col­lected Twit­ter data. Here’s how it works.

(Note: This post assumes famil­iar­ity with the solu­tion dis­cussed in my last post, as it builds on much of the same func­tion­al­ity explained there.)

What to do first

Other than enabling the eVar and cus­tom event that you will use to store Twit­ter data, the only pre­req­ui­site for this solu­tion is the appro­pri­ate clas­si­fi­ca­tion struc­ture based on the eVar that you will use. I have given a sim­ple exam­ple in the screen shot below; you may choose to add addi­tional clas­si­fi­ca­tions (such as “tweet cat­e­gory,” men­tioned above) based on your own busi­ness needs.

Classifications set up to track Twitter data

SAINT API

The SAINT API is described thor­oughly on the Omni­ture Devel­oper Con­nec­tion site (login required); the impor­tant thing to take away from that doc­u­men­ta­tion as it relates to this blog post is that import­ing SAINT data using Omniture’s Web Ser­vices API con­sists of call­ing four meth­ods: Saint.getCompatibilityMetrics, Saint.ImportCreateJob, Saint.ImportPopulateJob, and Saint.ImportCommitJob.

Saint.getCompatibilityMetrics returns the “relation_id” val­ues that used to tell the other meth­ods what vari­able is being clas­si­fied. Saint.ImportCreateJob pre­pares the API to upload SAINT data into Site­Cat­a­lyst and defines the SAINT “columns” (i.e. clas­si­fi­ca­tions) that will be uploaded. Saint.ImportPopulateJob allows you to input the clas­si­fi­ca­tion meta­data that you will be tying to the raw vari­able val­ues. Saint.ImportCommitJob tells the API that the full set of data is now present, and trans­mits the data into SiteCatalyst.

Adding the SAINT API to the SiteCatalyst-Twitter solution

In my pre­vi­ous post, I men­tioned (almost in pass­ing) the need to grab the unique “tweet ID” out of the <id> ele­ment present in each <entry> ele­ment returned by queries to the Twit­ter Search API; this was done to help ensure that searches are not double-counted by caus­ing the Twit­ter API to return only tweets newer than the given ID. You will still need to add this de-duplication mech­a­nism when using the SAINT API solu­tion, but here the tweet ID takes on addi­tional impor­tance. Here is a sam­ple result from the Twit­ter Search API:

<entry>
<id>tag:search.twitter.com,2005:1242044829</id>
<published>2009-02-23T19:58:41Z</published>
<link type="text/html" rel="alternate" href="http://twitter.com/jeffjordan/statuses/1242044829"/>
<title>RT: @Omni_man: Want to learn how to integrate Omniture SiteCatalyst and Twitter? Check out my latest blog post: http://is.gd/kzLk</title>
<content type="html">RT: @Omni_man: Want to learn how to integrate <b>Omniture</b> SiteCatalyst and Twitter? Check out my latest blog post: <a href="http://is.gd/kzLk">http://is.gd/kzLk</a></content>
<updated>2009-02-23T19:58:41Z</updated>
<link type="image/png" rel="image" href="http://s3.amazonaws.com/twitter_production/profile_images/70442609/photo_normal.jpg"/>
<twitter:source><a href="http://www.twhirl.org/">twhirl</a></twitter:source>
<author>
<name>jeffjordan (jeffjordan)</name>
<uri>http://twitter.com/jeffjordan</uri>
</author>
</entry>

The ID of this tweet is 1242044829; this ID is unique to this par­tic­u­lar tweet. In this case, you should grab this ID out of every tweet returned by your Twit­ter API query, and pass it into Site­Cat­a­lyst using an eVar in a Data Inser­tion API request like the one below (which uses eVar3 for this purpose).

<?xml version=1.0 encoding=UTF-8?>
<request>
<sc_xml_ver>1.0</sc_xml_ver>
<reportsuiteid>yourrsid</reportsuiteid>
<pageurl>http://www.yoursite.com</pageurl>
<linkType>lnk_o</linkType>
<linkName>Twitter Mention</linkName>
<events>event1</event>
<evar3>1242044829</evar3>
</request>

(Note that you should con­tinue to use a cus­tom event (event1, in this case) to count total Twit­ter mentions.)

You may be won­der­ing why I am rec­om­mend­ing only pass­ing an obscure, seem­ingly mean­ing­less tweet ID into Site­Cat­a­lyst. This is where the SAINT API comes in. Because we know quite a bit about the use­ful con­tent of the tweet, you can use SAINT to import this con­tent into Site­Cat­a­lyst as a clas­si­fi­ca­tion of the tweet ID. In this exam­ple, I will also need the tweet author (in the ele­ment in the Twit­ter results), tweet receip­i­ent (which can be derived using the ele­ment), and the tweet text (which is the full con­tents of the ele­ment). See my pre­vi­ous post for more infor­ma­tion on grab­bing this infor­ma­tion out of the Twit­ter API results.

(Again, I will not pro­vide the frame­work ele­ments of the SAINT API meth­ods here, and will instead focus on key com­po­nents of your SAINT API imple­men­ta­tion. Note that the SAINT API call does not replace the Data Inser­tion API call shown above. It is a sep­a­rate API, with both APIs being key to this solu­tion. The steps below assume that you are mak­ing Data Inser­tion API calls like the one shown above, and that you are able to receive valid search results from the Twit­ter Search API and are able to parse those, putting key data into server-side vari­ables as appropriate.)

Using Saint.getCompatibilityMetrics

This method allows you to pass an array con­tain­ing report suite IDs and returns a list of vari­ables that have clas­si­fi­ca­tions set up on them. The key piece of infor­ma­tion that you will need out of this method is the relation_id value which cor­re­sponds to the eVar that you have cho­sen to use to track Twit­ter data.

Using Saint.ImportCreateJob

In this method, make sure to set the fol­low­ing appro­pri­ate. Of par­tic­u­lar impor­tance is the array stor­ing the clas­si­fi­ca­tion names. Make sure to input these com­pletely, and in the cor­rect order. The first ele­ment in this array should always be “Key,” which refers to the raw data (in this case, the tweet IDs) passed into the eVar itself.

$report_suite_array = array('yourrsid');
$header = array('Key','@jeffjordan','Tweet Recipient','Tweet Text');

$params = array(
'report_suite_array'=>$report_suite_array,
'header'=>$header,
'relation_id'=>107,
'email_address'=>"omniturecare@omniture.com",
'overwrite_conflicts'=>true,
'export_results'=>0,
'description'=>"Twitter Import Job"
);

As a response, you should receive a job_id value. You will use this when you call Saint.ImportCommitJob.

Using Saint.ImportPopulateJob

Saint.ImportPopulateJob uses a mul­ti­di­men­sional array to store val­ues for the “key” and for each clas­si­fi­ca­tion that you entered using Saint.ImportCreateJob to allow you to call this method once for each tweet that you have processed using the Data Inser­tion API (and since your last call to the SAINT API). Clas­si­fi­ca­tion val­ues should be sur­rounded by sin­gle quotes and comma-separated. For example:

$row1 = array('1242044829','@jeffjordan','@Omni_man','RT: @Omni_man: Want to learn how to integrate Omniture SiteCatalyst and Twitter? Check out my latest blog post: http://is.gd/kzLk');
$row2 = array('1250352120','@OmnitureCare','','Working on a blog post.');
$row3 = array('1250300090','@Omni_man','@OmnitureCare','@OmnitureCare just posted about how to implement the Twitter-SiteCatalyst concept. Check it out and please re-tweet!');

$r_rows[] = array('row'=>$row1);
$r_rows[] = array('row'=>$row2);
$r_rows[] = array('row'=>$row3);

A cou­ple of impor­tant things to note:

  • When pars­ing the XML returned by the Twit­ter Search API, and plac­ing the appro­pri­ate val­ues in the array shown above, make sure to escape char­ac­ters cor­rectly! Many tweets will use sin­gle quotes in one way or another, and these will need to be escaped so that they do not incor­rectly sug­gest the end of the string.
  • If you need to leave a clas­si­fi­ca­tion blank—for exam­ple, if there was no tweet recip­i­ent and you do not want to pass a generic value—make sure to leave an empty set of sin­gle quotes as a place­holder to pre­vent the next value from being counted in the wrong category.

Using Saint.ImportCommitJob

This is the eas­i­est of the meth­ods to work with. All you will need is the job_id value that you received in response to your Saint.ImportCreateJob method call. This should com­plete the SAINT API upload, and cause the “friendly” Twit­ter data to be added to the tweet ID that you passed using the Data Inser­tion API in the Site­Cat­a­lyst inter­face, Omni­ture Dis­cover, and Data Warehouse.

The end result

As you can see in the screen shot below, this leaves us with a report for each of the clas­si­fi­ca­tions that we set up above:

Reports available for Twitter data

Full sub­re­la­tions are auto­mat­i­cally avail­able between an eVar and the clas­si­fi­ca­tion reports that stem from it, so within these reports we can break any­thing down by any­thing else. For exam­ple, here is tweet author bro­ken down by tweet recipient:

Twitter data available in SiteCatalyst

There is no loss of report­ing func­tion­al­ity from using the SAINT API to upload the tweet meta­data into clas­si­fi­ca­tions rather than to pass them into sep­a­rate eVars. Best of all, you save a few eVars for use in track­ing other impor­tant data on your site or in the world of social media!

1 comments
Phil
Phil

Thanks, Ben! This sparked a project that I think is really going to benefit my team.