SiteCatalyst and Twitter: The SAINT API Approach
In my last post, I discussed how you can use the Omniture Data Insertion API along with the Twitter Search API to pass Twitter data into SiteCatalyst. Based on conversations with a few readers, it seems that those of you who dove in and began developing after reading Adam Greco’s post last week were thinking along those same lines.
However, there is an important alternative methodology to allow you to insert Twitter data (“tweets”) regarding your brand into SiteCatalyst, and it leverages the new Omniture SAINT API, which allows you to upload classification data into SiteCatalyst on the fly.
Why use this method? Instead of consuming four or five of the maximum 50 eVar variables in your report suite(s), it uses a single eVar. Furthermore, full subrelations are not required in order to break down one type of Twitter data, such as tweet author, by another. Another key advantage is that SAINT classifications can be changed retroactively, which means that using SAINT allows you to add additional classifications, such as tweet category (e.g. complaint, praise, support request, etc.) quickly and easily, and apply it to your previously collected Twitter data. Here’s how it works.
(Note: This post assumes familiarity with the solution discussed in my last post, as it builds on much of the same functionality explained there.)
What to do first
Other than enabling the eVar and custom event that you will use to store Twitter data, the only prerequisite for this solution is the appropriate classification structure based on the eVar that you will use. I have given a simple example in the screen shot below; you may choose to add additional classifications (such as “tweet category,” mentioned above) based on your own business needs.
The SAINT API is described thoroughly on the Omniture Developer Connection site (login required); the important thing to take away from that documentation as it relates to this blog post is that importing SAINT data using Omniture’s Web Services API consists of calling four methods: Saint.getCompatibilityMetrics, Saint.ImportCreateJob, Saint.ImportPopulateJob, and Saint.ImportCommitJob.
Saint.getCompatibilityMetrics returns the “relation_id” values that used to tell the other methods what variable is being classified. Saint.ImportCreateJob prepares the API to upload SAINT data into SiteCatalyst and defines the SAINT “columns” (i.e. classifications) that will be uploaded. Saint.ImportPopulateJob allows you to input the classification metadata that you will be tying to the raw variable values. Saint.ImportCommitJob tells the API that the full set of data is now present, and transmits the data into SiteCatalyst.
Adding the SAINT API to the SiteCatalyst-Twitter solution
In my previous post, I mentioned (almost in passing) the need to grab the unique “tweet ID” out of the <id> element present in each <entry> element returned by queries to the Twitter Search API; this was done to help ensure that searches are not double-counted by causing the Twitter API to return only tweets newer than the given ID. You will still need to add this de-duplication mechanism when using the SAINT API solution, but here the tweet ID takes on additional importance. Here is a sample result from the Twitter Search API:
<link type="text/html" rel="alternate" href="http://twitter.com/jeffjordan/statuses/1242044829"/>
<title>RT: @Omni_man: Want to learn how to integrate Omniture SiteCatalyst and Twitter? Check out my latest blog post: http://is.gd/kzLk</title>
<content type="html">RT: @Omni_man: Want to learn how to integrate <b>Omniture</b> SiteCatalyst and Twitter? Check out my latest blog post: <a href="http://is.gd/kzLk">http://is.gd/kzLk</a></content>
<link type="image/png" rel="image" href="http://s3.amazonaws.com/twitter_production/profile_images/70442609/photo_normal.jpg"/>
The ID of this tweet is 1242044829; this ID is unique to this particular tweet. In this case, you should grab this ID out of every tweet returned by your Twitter API query, and pass it into SiteCatalyst using an eVar in a Data Insertion API request like the one below (which uses eVar3 for this purpose).
<?xml version=1.0 encoding=UTF-8?>
(Note that you should continue to use a custom event (event1, in this case) to count total Twitter mentions.)
You may be wondering why I am recommending only passing an obscure, seemingly meaningless tweet ID into SiteCatalyst. This is where the SAINT API comes in. Because we know quite a bit about the useful content of the tweet, you can use SAINT to import this content into SiteCatalyst as a classification of the tweet ID. In this example, I will also need the tweet author (in the element in the Twitter results), tweet receipient (which can be derived using the element), and the tweet text (which is the full contents of the element). See my previous post for more information on grabbing this information out of the Twitter API results.
(Again, I will not provide the framework elements of the SAINT API methods here, and will instead focus on key components of your SAINT API implementation. Note that the SAINT API call does not replace the Data Insertion API call shown above. It is a separate API, with both APIs being key to this solution. The steps below assume that you are making Data Insertion API calls like the one shown above, and that you are able to receive valid search results from the Twitter Search API and are able to parse those, putting key data into server-side variables as appropriate.)
This method allows you to pass an array containing report suite IDs and returns a list of variables that have classifications set up on them. The key piece of information that you will need out of this method is the relation_id value which corresponds to the eVar that you have chosen to use to track Twitter data.
In this method, make sure to set the following appropriate. Of particular importance is the array storing the classification names. Make sure to input these completely, and in the correct order. The first element in this array should always be “Key,” which refers to the raw data (in this case, the tweet IDs) passed into the eVar itself.
$report_suite_array = array('yourrsid');
$header = array('Key','@jeffjordan','Tweet Recipient','Tweet Text');
$params = array(
'description'=>"Twitter Import Job"
As a response, you should receive a job_id value. You will use this when you call Saint.ImportCommitJob.
Saint.ImportPopulateJob uses a multidimensional array to store values for the “key” and for each classification that you entered using Saint.ImportCreateJob to allow you to call this method once for each tweet that you have processed using the Data Insertion API (and since your last call to the SAINT API). Classification values should be surrounded by single quotes and comma-separated. For example:
$row1 = array('1242044829','@jeffjordan','@Omni_man','RT: @Omni_man: Want to learn how to integrate Omniture SiteCatalyst and Twitter? Check out my latest blog post: http://is.gd/kzLk');
$row2 = array('1250352120','@OmnitureCare','','Working on a blog post.');
$row3 = array('1250300090','@Omni_man','@OmnitureCare','@OmnitureCare just posted about how to implement the Twitter-SiteCatalyst concept. Check it out and please re-tweet!');
$r_rows = array('row'=>$row1);
$r_rows = array('row'=>$row2);
$r_rows = array('row'=>$row3);
A couple of important things to note:
- When parsing the XML returned by the Twitter Search API, and placing the appropriate values in the array shown above, make sure to escape characters correctly! Many tweets will use single quotes in one way or another, and these will need to be escaped so that they do not incorrectly suggest the end of the string.
- If you need to leave a classification blank—for example, if there was no tweet recipient and you do not want to pass a generic value—make sure to leave an empty set of single quotes as a placeholder to prevent the next value from being counted in the wrong category.
This is the easiest of the methods to work with. All you will need is the job_id value that you received in response to your Saint.ImportCreateJob method call. This should complete the SAINT API upload, and cause the “friendly” Twitter data to be added to the tweet ID that you passed using the Data Insertion API in the SiteCatalyst interface, Omniture Discover, and Data Warehouse.
The end result
As you can see in the screen shot below, this leaves us with a report for each of the classifications that we set up above:
Full subrelations are automatically available between an eVar and the classification reports that stem from it, so within these reports we can break anything down by anything else. For example, here is tweet author broken down by tweet recipient:
There is no loss of reporting functionality from using the SAINT API to upload the tweet metadata into classifications rather than to pass them into separate eVars. Best of all, you save a few eVars for use in tracking other important data on your site or in the world of social media!