Summit Topic #1: Visitor Scoring in SiteCatalyst

Whew! Omni­ture Sum­mit 2010 has come and gone, but hope­fully you’re still deriv­ing real value from some of the thoughts, tac­tics, and strate­gies that were dis­cussed, both for­mally and infor­mally, at the con­fer­ence. As promised, I will be blog­ging about each of the three advanced Site­Cat­a­lyst solu­tions that my team and I cov­ered in our break­out ses­sion. (It’s also worth not­ing that, while we dis­cussed Site­Cat­a­lyst specif­i­cally in my ses­sion, these ideas become even more pow­er­ful when cou­pled with an advanced seg­men­ta­tion tool, such as Dis­cover or ASI, or when used in the con­text of Test & Tar­get.) Let’s get started with the first topic from our session.

Vis­i­tor Scoring—an overview

You prob­a­bly have a num­ber key activ­i­ties on your site that con­sti­tute a highly engaged vis­i­tor. These are things that you want users to do, because they either con­sti­tute direct suc­cess (i.e., con­ver­sion), indi­cate the build­ing of loy­alty (e.g., social ele­ments such as “Write a Review”), or sug­gest that the user is truly inter­act­ing with your site (e.g., inter­nal searches, view­ing pho­tos, etc.). Vis­i­tor scor­ing in Site­Cat­a­lyst allows you to assign numer­i­cal “scores” to each of these key activ­i­ties, and then aggre­gate these scores as vis­i­tors move through these dif­fer­ent site elements.

The idea is that you can begin to see how vary­ing vis­i­tor (and visit) scores affect con­ver­sion and cus­tomer loy­alty. For exam­ple, you may notice that users with a score of 30 or higher spend twice as much money on each order when com­pared to users with score less than 30. You can also drill down to see which site ele­ments are pre­ferred by highly engaged vis­i­tors so that you can focus your efforts on those areas. Addi­tion­ally, the solu­tion allows you to see which vis­i­tor acqui­si­tion chan­nels gen­er­ate the high­est lev­els of inter­ac­tion with key site ele­ments, and which cam­paigns are the stick­i­est in terms of lead­ing users to these aspects of your site.

To do this, you will need three things:

1. A scor­ing sys­tem that assigns val­ues to each of your top 5–10 key engage­ment points.
2. One “counter eVar,” which will be explained below.
3. One cus­tom event, set to “numeric” as its type.

The Scor­ing System

This is the most com­plex aspect of the Vis­i­tor Scor­ing solu­tion because you will need to lay out a rel­a­tive points sys­tem for the 5–10 top site ele­ments that you want users to engage. Some users may have access to advanced sys­tems for sci­en­tif­i­cally assign­ing rel­a­tive value to dif­fer­ent site activ­i­ties, but in the absence of such a sys­tem here is how we rec­om­mend doing this:

1. Iden­tify the top sev­eral key “things” that you want users to do on your site (includ­ing, but not nec­es­sar­ily lim­ited to, con­ver­sion). As described above, these should be activ­i­ties that show engage­ment; the idea is to gauge how active a visitor’s expe­ri­ence is.
2. Iso­late either the activ­ity that you con­sider most valu­able or the activ­ity that you con­sider least valu­able on your list.
3. Assign that value an arbi­trary score, such as “5.” (This can be any­thing, and can be adjusted later.)
4. Assign other scores rel­a­tive to that first score. For exam­ple, if we believe that “inter­nal search” is our impor­tant, but is our least “valu­able” activ­ity of those on our list, we might give it a value of 4. Let’s say that sub­scrib­ing to a newslet­ter is sig­nif­i­cantly more valu­able; we might give it a value of 8. Read­ing a review may be even less valu­able in our minds than per­form­ing a search; we’ll give it a value of 3.

This chart shows an exam­ple of how this sys­tem might be laid out:

This will be entirely unique to your busi­ness and, at the begin­ning, may involve a bit of edu­cated esti­ma­tion. For­tu­nately, as you work more closely with these reports, you’ll be able to see where scores are being inflated (or deflated) by user activ­i­ties that have been over­val­ued or under­val­ued, and you can adjust your scor­ing appropriately.

The Counter eVar

A sta­ple of almost any Site­Cat­a­lyst imple­men­ta­tion is the “eVar” vari­able, which allows you to set a value (e.g., an inter­nal search key­word) and per­sist it for a cus­tomiz­able period of time so that you can tie sub­se­quent suc­cess met­rics back to the val­ues. (In the con­ve­nient “inter­nal search key­word” exam­ple, an eVar allows you to view the amount of rev­enue, num­ber of leads, etc. that occurred after the given key­word was searched.) These eVar vari­ables accept text strings by default, but also have a nifty option for accept­ing num­bers. Adam Greco defined counter eVars in out­stand­ing detail in a pre­vi­ous blog post, which con­tains infor­ma­tion on enabling counter eVars as well as a num­ber of use cases.

In the case of Vis­i­tor Scor­ing, we’re going to take advan­tage of eVars’ per­sis­tence by set­ting the num­ber of points for the key activ­i­ties that were defined in the pre­vi­ous step into the eVar each time the activ­ity occurs. For exam­ple, based on the chart above, we would set a value of “+3″ into the counter eVar imme­di­ately after the user rates a prod­uct, “+5″ every time the user e-mails an item to a friend, and so forth. Implementation-wise, using eVar1 as an exam­ple, it looks like this:

// after an internal search
s.eVar1="+4"

// after writing a review
s.eVar1="+5"

(Not that this has any­thing to do with Vis­i­tor Scor­ing, but it’s worth men­tion­ing here—because we for­got to do it in one of our two chances to present this topic—that you CAN pass neg­a­tive num­bers into counter eVars to sub­tract from the over­all vis­i­tor value.)

As the user moves through an expe­ri­ence on your site, inter­act­ing with the var­i­ous key ele­ments that you have defined, he/she will accrue points. If the user con­verts, you’ll be able to see their “score” at the time of con­ver­sion. If they don’t con­vert, you’ll be able to see that, too.

Pretty con­fus­ing, right? We can see that scores of 16–17 appears to be the “sweet spot,” but it’s hard to really under­stand what we’re look­ing at here. For­tu­nately, you can use SAINT to “group” vis­i­tor scores, and note that SAINT is flex­i­ble and allows you to rearrange/reallocate scores into dif­fer­ent buck­ets depend­ing on your chang­ing needs and obser­va­tions. After doing this, the report is much more digestible and actionable:

Now we’ve got some­thing we can use. This makes it much eas­ier to see how dif­fer­ent engage­ment lev­els affect con­ver­sion. Again, how you divide up the dif­fer­ent scores into groups is com­pletely up to you, and you’ll prob­a­bly want to adjust it as you dig into these reports. Also, note that you can do some really pow­er­ful things here with seg­men­ta­tion; using Dis­cover, ASI, or Data Ware­house, you can focus in on the user expe­ri­ence both for users in the “Very High” group and in the “Low” group to see what they’re doing, and opti­mize your site around those findings.

The Numeric Event

The final piece of our Vis­i­tor Scor­ing sys­tem involves a numeric (a.k.a. “incre­men­tor”) event, which you can enable using the Admin Con­sole. (Mr. Greco doc­u­mented numeric events as well in a pre­vi­ous post.) This might be my favorite part of the solution.

In addi­tion to set­ting a counter eVar when­ever a user does some­thing that we’re scor­ing, this method also sets the value in an event so that you can view the score as a met­ric in var­i­ous reports. This does involve using the s.products string, but don’t worry; we’re not going to mess with any actual prod­uct data. The imple­men­ta­tion would look some­thing like this (expand­ing on the exam­ples given above and using event2 as our numeric event):

// after an internal search
s.eVar1="+4"
s.events="event2"
s.products=";;;;event2=4"

// after writing a review
s.eVar1="+5"
s.events="event2"
s.products=";;;;event2=5"

(Make sure to note that “event2” exists both in s.events and in s.products in this case, and that there are exactly four semi-colons in s.products before event2 gets set.)

The great thing about this is that it allows you to see how var­i­ous data dimen­sions affect engage­ment with key site ele­ments. You’ll prob­a­bly want to set up a cal­cu­lated met­ric to divide this “score” met­ric by vis­its, because the raw score may be higher for dif­fer­ent data dimen­sions sim­ply due to vary­ing lev­els of traf­fic; for exam­ple, when view­ing this met­ric in the Cam­paigns report, a cam­paign that has 10,000 click-throughs will likely have a higher vis­i­tor score than a cam­paign that has 10 click-throughs sim­ply because the over­all traf­fic level is higher. When we view var­i­ous mar­ket­ing chan­nels through the lens of this cal­cu­lated met­ric, we imme­di­ately get a great report:

Social media sites, nat­ural search, and part­ners are the clear win­ners in terms of bring­ing inter­ested, engaged vis­i­tors to our site. Looks like we know where to focus our efforts—especially if we already know (from our expe­ri­ence with our counter eVar) how much more con­ver­sion a high-score vis­i­tor is likely to gen­er­ate on our site. This gets even bet­ter when we focus on indi­vid­ual items within our top chan­nels, break­ing down this report by refer­ring domain:

Not only do we know that social media brings eager vis­i­tors to our site, but we even know exactly which social media efforts/campaigns were most pow­er­ful (in this case, Fark​.com, Deli­cious, LinkedIn, and Facebook).

Conclusion—and two bonus tips!

We didn’t men­tion this dur­ing the Advanced Site­Cat­a­lyst ses­sion at Sum­mit, so be glad you read down this far. A really use­ful twist on this solu­tion if you have two avail­able eVars for use with vis­i­tor scor­ing is to con­fig­ure one of them to expire at the end of the visit and another to a much longer expi­ra­tion (such as “never”). The first eVar then gives a uni­form view of indi­vid­ual vis­its, and how vary­ing lev­els of inter­ac­tion with key site ele­ments affects con­ver­sion within the indi­vid­ual visit only. The sec­ond eVar would pro­vide a view of “life­time engage­ment” across mul­ti­ple vis­its since the user last cleared his/her cook­ies. You can slice and dice both data sets to get some pow­er­ful views into how user behav­iors may change over time.

Along these same lines, it’s pos­si­ble that dif­fer­ent teams or indi­vid­u­als may want to assign dif­fer­ent scores to cer­tain site activ­i­ties. Don’t fight about it! Don’t let this scor­ing sys­tem destroy the har­mony in your mar­ket­ing depart­ment. Instead, sim­i­lar to the tip just men­tioned, if you’ve got an extra eVar, you can actu­ally assign dif­fer­ent scores to the same key site activ­i­ties. Here’s a quick exam­ple using s.eVar1 and s.eVar2:

// this one is for the first team
s.eVar1="+5"
// this one is for the second team
s.eVar2="+30"

So there you have it. One tac­tic down, two to go. Next time, we’ll cover per­haps the most pop­u­lar of the top­ics that we covered—the brand-new get­Per­cent­PageViewed plug-in. If you have any ques­tions about Vis­i­tor Scor­ing, either based on this blog post or on the con­tent we shared at Omni­ture Sum­mit, please let me know by leav­ing a com­ment here. You can also con­tact me via Twit­ter (@OmnitureCare).