Under the Hood with Visits and Visitors

Today I’d like to dis­cuss how Site­Cat­a­lyst keeps track of page views and orga­nizes them into vis­its and vis­i­tors. Since vis­its and vis­i­tors rep­re­sent KPIs (or are involved in KPIs) for many of our cus­tomers, under­stand­ing how they work can be immensely help­ful both in exe­cut­ing a suc­cess­ful imple­men­ta­tion and in interpreting/resolving poten­tial imple­men­ta­tion issues.

All of this cen­ters around a per­sis­tent cookie called s_vi, which stores the vis­i­tor ID pro­vided by Omniture’s servers when­ever a user first vis­its your site. This cookie is set on your spec­i­fied data col­lec­tion domain, which is the domain of the Omni­ture image request (i.e. server call); this domain ends with 2o7​.net if your site uses third-party cook­ies, and is your domain with a spe­cial sub­do­main (e.g., met​rics​.your​site​.com) if you are using first-party cook­ies. The s_vi cookie has a life­time of five years, so it’s about as per­sis­tent as can be.

When a user arrives at your site for the first time, the Omni­ture JavaScript code down­loads and exe­cutes. An “image request” is sent into Omniture’s data col­lec­tion servers. Since the user is mak­ing his first-ever visit to your site, an image request occurs, but no s_vi cookie exists on your data col­lec­tion domain. This trig­gers a 302 redi­rect, which allows the data col­lec­tion server to gen­er­ate a vis­i­tor ID value and write it to a new s_vi cookie on your data col­lec­tion domain. Take a look at the screen shot below.

Here you can see the 302 redi­rect, fol­lowed by a 200 sta­tus, which indi­cates a suc­cess­ful and nor­mal image request and trans­fer of page view data into Site­Cat­a­lyst. You will also notice that this sec­ond request (high­lighted) shows an s_vi cookie value in the last row under “Request Header Value.”

All sub­se­quent page views that this user gen­er­ates will pass this cook­ied vis­i­tor ID into Site­Cat­a­lyst in the HTTP header, and there­fore will not gen­er­ate addi­tional 302 redi­rects. A cou­ple of points to note:

1. In the first image request (200 sta­tus) fol­low­ing the redi­rect, you will see an extra para­me­ter passed in the request; “vidn=” will be set to the vis­i­tor ID value stored in the cookie. This will not occur on sub­se­quent page views.
2. This first image request will also con­tain a para­me­ter, “pccr=true,” which is set to pre­vent an end­less loop of redi­rects if the cookie can­not actu­ally be set (as may occur when, for exam­ple, the user has dis­abled per­sis­tent cook­ies in his web browser).

Why does this matter?

It mat­ters because this vis­i­tor ID is the glue that binds mul­ti­ple page views together and allows Site­Cat­a­lyst to inter­pret them as dis­tinct vis­its and visitors.

As page views flow in to Omniture’s data col­lec­tion and pro­cess­ing servers, Site­Cat­a­lyst exam­ines the vis­i­tor ID value passed in the s_vi cookie; if it does not have a record of another page view within the last 30 min­utes from that same vis­i­tor ID in the given report suite, then it knows to count a new visit. If it does have a record of a page view within the last 30 min­utes from this vis­i­tor ID, then it con­tin­ues a visit; pathing data can then be derived by com­par­ing time­stamps to sequen­tial­ize these page views.

Based on this expla­na­tion, you may already have inferred how Site­Cat­a­lyst can track unique visitors—daily, weekly, monthly, etc. If a page view is passed to Omni­ture servers with a vis­i­tor ID that hasn’t been seen in the given report suite dur­ing that day, then the page view rep­re­sents a new daily unique vis­i­tor; if the vis­i­tor ID has not been seen dur­ing the cur­rent month, then it is a new monthly unique vis­i­tor, as so on and so forth.

As new vis­its are detected, Site­Cat­a­lyst can also com­pare the vis­i­tor ID to his­tor­i­cal data to deter­mine if it has ever been seen before in the given report suite. If so, it is a repeat vis­i­tor; if not, it is a new vis­i­tor. This enables Site­Cat­a­lyst to record the total num­ber of vis­its for var­i­ous visitors.

Again, a cou­ple of points to keep in mind:

1. Site­Cat­a­lyst does not track vis­its or pathing data for users who have dis­abled per­sis­tent cook­ies in their web browsers, nor does it track this data for users whose browsers are fun­da­men­tally inca­pable of accept­ing cook­ies. A pri­mary rea­son for this is that with­out the s_vi cookie being set and passed on each page view, Site­Cat­a­lyst relies on the user’s IP address and user-agent string to derive a vis­i­tor ID dur­ing data processing.
2. As has been men­tioned pre­vi­ously, the s_vi cookie is set on your data col­lec­tion domain. You may have mul­ti­ple report suites and web sites shar­ing a data col­lec­tion domain; this should not cause mis-calculation of paths, vis­its, or unique vis­i­tors. While a user who vis­its mul­ti­ple sites that use the same data col­lec­tion domain may there­fore pass the same vis­i­tor ID into mul­ti­ple report suites, Site­Cat­a­lyst only com­pares these page views to prior vis­i­tor ID records within the given report suite.

A closer look at s_vi

Recently, a grow­ing num­ber of sites have begun using non-JavaScript data col­lec­tion meth­ods such as the Data Inser­tion API and full-processing Data Sources. The data passed into Site­Cat­a­lyst by these means should, wher­ever pos­si­ble, be tied back to vis­its and vis­i­tors estab­lished using JavaScript data col­lec­tion, so that rev­enue and other met­rics are cor­rectly attrib­uted to the users who gen­er­ated them.

One way to do this is to take own­er­ship of the vis­i­tor ID process by assign­ing and pass­ing your own vis­i­tor ID val­ues, which super­sede the cook­ied value when passed cor­rectly. Another way is to read the s_vi cookie, parse it, pull out the actual vis­i­tor ID, and pass it using your cho­sen alter­na­tive data col­lec­tion method.

Here’s a quick exam­ple: Some retail sites have recently switched over to the Data Inser­tion API to pass order data; all other site data is passed using JavaScript. This means that vis­its and vis­i­tors are first counted using JavaScript, but the visit may con­clude with a Data Inser­tion API call. If you are a devel­oper for a retail site con­sid­er­ing this switch, you’ll want to ensure that the orders passed using the API are tied to vis­its that began with JavaScript requests. To do this, you will need to ensure that the same vis­i­tor ID passed by JavaScript is passed by the API post.

Warn­ing: If you choose do this by read­ing the s_vi cookie, please be aware that the format/syntax within the cookie may change in the future. Omni­ture has com­mit­ted to pro­vide as much advance warn­ing as pos­si­ble should such a change be imple­mented, but it is worth not­ing this caveat up front.

While I won’t explain here how you can read and parse a cookie, nor how to set a vis­i­tor ID using the Data Inser­tion API, I will show you the con­tents of the s_vi cookie below so that you can grab the vis­i­tor ID for any purpose.

The impor­tant por­tion of the cookie shown above is 49C083D100005A40-A02087F00000039, and sim­i­lar pairs of 16-character strings will be found in all other s_vi cook­ies. This is the vis­i­tor ID set by Omniture’s servers on the user’s first page view, and it is the value you would need to cap­ture out of the cookie and pass as the vis­i­tor ID using an alter­na­tive data col­lec­tion method.

Here are two more items to note regard­ing the vis­i­tor ID stored in the s_vi cookie:

1. Note that this value is hex-encoded (e.g., the range of char­ac­ters is 0-F rather than 0–9); it is decoded by Omniture’s servers and is ulti­mately stored as an inte­ger value.
2. If your site uses third-party cook­ies, the s_vi cookie is stored on 2o7​.net and your servers will not be able to read it. If this is the case and you are plan­ning to imple­ment a non-JavaScript data col­lec­tion method, Omni­ture rec­om­mends assign­ing your own vis­i­tor IDs to all users (even on JavaScript-tagged pages, using the s.visitorID vari­able). Details are avail­able in the Site­Cat­a­lyst Imple­men­ta­tion Manual.

How can this help you?

Per­haps the “value propo­si­tion” for this blog post should have come a few thou­sand words ago, but after try­ing to artic­u­late it at the begin­ning of the post, I real­ized that it would only make sense com­ing after the bulk of the infor­ma­tion. Let me list just a few ways that under­stand­ing vis­i­tor IDs and the s_vi cookie can help you in your imple­men­ta­tion and report­ing efforts.

1. Using a packet mon­i­tor, you can check the s_vi value on suc­ces­sive hits as you browse your site. If the s_vi cookie value ever changes from page view to page view, you may have incon­sis­ten­cies in your imple­men­ta­tion that will inflate visit and vis­i­tor counts, as well as those of cer­tain other met­rics. Specif­i­cally, check to ensure that all of your pages use the same s_code.js file, or at least that all of the s_code.js files on the given site are set to use the same data col­lec­tion domain. If using first-party cook­ies, this is set in the s.trackingServer and s.trackingServerSecure vari­ables. If using third-party cook­ies, this is set using the s.visitorNamespace vari­able. Warn­ing: If you find a dis­crep­ancy in these vari­ables, please con­tact Omni­ture Client­Care before mak­ing changes.
2. As has just been dis­cussed, under­stand­ing how the s_vi cookie works allows you to use the vis­i­tor ID to tie data passed using meth­ods other than JavaScript back to vis­its and vis­i­tors recorded on your web site using JavaScript.
3. Gen­er­ally, this infor­ma­tion gives you a con­text for under­stand­ing visit-, visitor-, and pathing-based data. If you really dig into it, I’ll bet you can even use the infor­ma­tion I’ve pre­sented here to fig­ure out how Site­Cat­a­lyst causes eVars to per­sist (hint: it isn’t in a cookie on the user’s com­puter!). Since knowl­edge is power, if you know how Site­Cat­a­lyst cal­cu­lates this data, you’re empow­ered to use it excit­ing new ways—even to invent your own solu­tions to solve crit­i­cal busi­ness questions!

Admit­tedly, the rest of this post is fairly dense, and yet could eas­ily be spun off in a num­ber of dif­fer­ent direc­tions. If there’s any­thing you’d like to see as the sub­ject of a future post, please let me know. As always, I’m avail­able by e-mail (omni­ture­care at omni­ture dot com) and on Twit­ter (@OmnitureCare) and I’d love to hear from you!