Begin­ning in April 2009, Google changed the way that search engine results pages ren­der links. Where pre­vi­ously these links pointed directly to your site’s land­ing page(s), in some cases they now point to a redi­rect and include the des­ti­na­tion URL (your land­ing page) in the query string of the redi­rect URL. When this occurs, the redi­rect page URL is the refer­ring URL on your land­ing page.

For exam­ple, you may be used to see­ing refer­rers such as this one in your Traf­fic Sources reports in Site­Cat­a­lyst and Discover:

http://www.google.com/search?hl=en&q=omniture+sitecatalyst&btnG=Google+Search

The new for­mat looks like this:

http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.yoursite.com%2Fhomepage.php&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=omniture+sitecatalyst&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w
&sig2=X8uCFh6IoPtnwmvGMULQfw

For­tu­nately, in almost all cases, these changes have no effect what­so­ever on Site­Cat­a­lyst report­ing. Site­Cat­a­lyst detects Google searches as refer­rers by look­ing for the following:

  1. The pres­ence of “google​.com” (or local equiv­a­lent, such as “google​.ca” for Canada) in the refer­ring URL
  2. The pres­ence of Google’s search key­word query para­me­ter (i.e. the query para­me­ter that con­tains the key­worrd for which the user searched), which is “q=”.

This new refer­ring URL still con­tains all of the data that Site­Cat­a­lyst uses in order to rec­og­nize Google click-throughs and searches. As you can see above, the domain is still “google​.com,” and “q=” is still present in the URL (see “&q=omniture+sitecatalyst” in both URLs above). Site­Cat­a­lyst does not require this para­me­ter to be in any par­tic­u­lar place in the query string, as long as it is there somewhere.

How­ever, the fact that there is a sig­nif­i­cant amount of data in this refer­ring URL prior to the “q=” query para­me­ter may cre­ate prob­lems for some sites that use long URLs. As men­tioned above, Google puts the des­ti­na­tion URL (your land­ing page) into the query string of the redi­rect URL, which then becomes the refer­ring URL on your land­ing page. Notice the “url=” para­me­ter in the exam­ple above; this para­me­ter will con­tain your land­ing page’s URL. Site­Cat­a­lyst JavaScript code trun­cates the refer­ring URL at 255 char­ac­ters in order to man­age the length of the request to Omniture’s servers (the IE browser has a built-in 2,083-character limit on all HTTP requests).

The result is that if there are 255 char­ac­ters in this redi­rect URL prior to the “q=” query para­me­ter, the search key­word will be not be avail­able to Omniture’s servers, and the Search Engines and Search Key­words reports in Site­Cat­a­lyst and other tools.

Early analy­sis of this change indi­cates that only about 5% of nat­ural searches will be affected by the long-URL prob­lem. How­ever, if your site has par­tic­u­larly long land­ing page URLs (more than roughly 150 char­ac­ters), your ana­lyt­ics may be affected at a rate higher than this aver­age. Also note that, cur­rently, the major­ity of searches are use the tra­di­tional refer­rer for­mat. It is not clear when Google may roll out this change across all searches, but at present there is no need to act rashly in imple­ment­ing the solu­tion described below.

In order to ensure that col­lec­tion of refer­rer data from Google to your site con­tin­ues unabated, Omni­ture has devel­oped a small patch that should be added to your s_code.js file (i.e. the global JavaScript file con­tain­ing the bulk of Site­Cat­a­lyst code; the file may be named dif­fer­ently on your site) which moves the “q=” query para­me­ter to the front of the refer­rer query string for users arriv­ing at your site from a Google search, ensur­ing that it appears within the first 255 char­ac­ters of the referrer.

The patch can be down­loaded by click­ing here.

NOTE: Site­Cat­a­lyst code ver­sions H.20.3 and newer will con­tain this patch natively; you will not need to add it man­u­ally. There­fore, you may choose to update your core Site­Cat­a­lyst code to the lat­est ver­sion avail­able in the Code Man­ager within Site­Cat­a­lyst, or as pro­vided by your Omni­ture Account Man­ager or Con­sul­tant, to take advan­tage of this func­tion­al­ity. Upgrad­ing the core JavaScript code from your cur­rent ver­sion may carry with it a num­ber of other advan­tages, such as media track­ing, speed improve­ments, and sup­port for addi­tional variables/plug-ins. Please con­sult your Account Man­ager for more information.

This patch works with all Site­Cat­a­lyst JavaScript code ver­sions H.0 through the most recent release, and will be auto­mat­i­cally included in future code ver­sions avail­able from the Site­Cat­a­lyst Admin Con­sole. Code ver­sions prior to H.0 (G.9 and ear­lier) require an update to the core JavaScript file to make the exist­ing imple­men­ta­tion com­pat­i­ble with H.x ver­sions of code. This patch should never be added to code ver­sions prior to H.0, as it will break their execution.

NOTE: The fol­low­ing changes should be made by an autho­rized, expe­ri­enced devel­oper who is famil­iar with your Site­Cat­a­lyst imple­men­ta­tion, as they require under­stand­ing of JavaScript, object-oriented pro­gram­ming, and Site­Cat­a­lyst vari­ables. Fur­ther­more, they should be tested care­fully in a devel­op­ment envi­ron­ment to ensure suc­cess­ful data col­lec­tion prior to deploy­ment on a pro­duc­tion site.

Fol­low the steps below to add the patch to your Site­Cat­a­lyst code. More detail on these steps will be pro­vided below.

  1. In your Site­Cat­a­lyst JavaScript code, locate a line which reads: var s=s_gi(s_account)
  2. Copy and paste the patch directly below this line, as shown in the screen shot below.
  3. Change [[INSTANCE]] at the end of the patch to the name of the Site­Cat­a­lyst object. Make sure to remove the dou­ble brackets.
  4. Save and test the JavaScript file in a devel­op­ment environment.

The patch should be added to your code imme­di­ately after the s_gi() func­tion is called. This is usu­ally done at the top of the s_code.js file. Please see the screen shot below, which shows a typ­i­cal Site­Cat­a­lyst JavaScript file, includ­ing the imple­men­ta­tion of the patch.

Patch code

Make sure that you have replaced [[INSTANCE]] at the end of the patch with the object name that your Site­Cat­a­lyst imple­men­ta­tion uses. In the major­ity of imple­men­ta­tions, this this name is “s” (with­out quotes). In this case, this line of code:

s._rf_hav()')}s_rf([[INSTANCE]]);

should be changed to:

s._rf_hav()')}s_rf(s);

This change is shown in the screen shot below.

Patch code

If your Site­Cat­a­lyst object is named any­thing other than “s”, sim­ply enter your object name in place of “s” in the exam­ple above. Make sure to remove the brack­ets, and do not put quotes around the object name.

Val­i­da­tion of this patch is most eas­ily per­formed by tem­porar­ily hard-coding the s.referrer value to an exam­ple such as the one pro­vided below:

s.referrer="http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.yoursite.com%2Fhomepage.php&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=omniture+sitecatalyst&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w&sig2=X8uCFh6IoPtnwmvGMULQfw"

Run a packet mon­i­tor (such as Tam­per Data for Fire­fox or HTTP Watch for IE) while load­ing the page, and check the “r=” para­me­ter  in the Omni­ture request (which you can iden­tify by the pres­ence of “/b/ss/” in the path of the request). This para­me­ter will con­tain the refer­ring URL. It should show the “q=” query para­me­ter toward the begin­ning of the URL, rather than toward the end (as in the exam­ple above).

As always, your organization’s sup­ported users should work with Omni­ture Client­Care if ques­tions arise dur­ing the process of adding and test­ing this patch. When suc­cess­fully imple­mented, it will ensure that report­ing on refer­rals from Google in the Search Engines and Search Key­words reports in Site­Cat­a­lyst, Data Ware­house, Dis­cover. and through­out the Omni­ture Suite.

Fre­quently Asked Questions

Will this patch also cause the nat­ural search rank­ing to be cap­tured by SiteCatalyst?

No. For infor­ma­tion on obtain­ing that func­tion­al­ity, please see the recent blog post by my col­league, Jor­dan LeBaron.

How can I tell if my report suite is affected by Google’s change?
Omni­ture rec­om­mends the fol­low­ing steps to help deter­mine whether or not a report suite is affected by the change described above.

  1. Run a Refer­rers report (Traf­fic Sources > Refer­rers in Site­Cat­a­lyst v14).
  2. Select a date range with a rea­son­able amount of refer­rer data; this will vary from report suite to report suite.
  3. Apply a search fil­ter of “www​.google​.com url” to iso­late refer­rers con­tain­ing both “www​.google​.com” and the new “url” query parameter.
  4. Click the e-mail icon to have the report sent to you.
  5. Click Advanced Deliv­ery Options.
  6. Under “Report For­mat,” choose CSV format.
  7. Under “Report Con­tents,” set the num­ber of rows to 50000 (with no comma).
  8. Enter your e-mail address in the “To” field.
  9. Click the “Sched­ul­ing Options” tab and ensure that “Send report now” is selected.
  10. Click Send.
  11. When the report arrives, open it in Excel.
  12. Search for rows where “q=” does not exist or does not have a sub­se­quent amper­sand (“&”).

The result of this search will be the rows where the refer­rer was trun­cated because of the changes described above.

If I do not add this patch to my Site­Cat­a­lyst code, will I lose all refer­rer data from Google?
This is unlikely. If a link to your site on a Google search results page points con­tains a rel­a­tively short URL, it is likely that the “q=” query para­me­ter will be present in the refer­ring URL in spite of the full land­ing page URL being included in the refer­rer. This will allow Google refer­rers to be tracked in the Search Engines and Search Key­words reports. How­ever, if land­ing page URLs are longer — for exam­ple, if they con­tain many query para­me­ters — the patch may be nec­es­sary to ensure that the search key­word para­me­ter is present after trun­ac­tion to 255 characters.

Do I need this patch if my site URLs are short?
Google may add addi­tional query para­me­ters in its redi­rect URLs in the future, so even if your land­ing page URLs are less than 150 char­ac­ters long, Omni­ture rec­om­mends adding this patch or upgrad­ing to the lat­est ver­sion of Site­Cat­a­lyst JavaScript code within the next few months.

How can I tell what ver­sion of code I am using?
This is most eas­ily done by search­ing for a line of code that con­tains “Site­Cat­a­lyst code ver­sion.” In the exam­ple shown above, this line is at the top of the s_code.js file and shows that the code ver­sion is H.20.2. This ver­sion is part of the “H” gen­er­a­tion of code and is there­fore com­pat­i­ble with the patch.

What should I do if my site uses a ver­sion of Site­Cat­a­lyst code prior to H.0?
Omni­ture rec­om­mends upgrad­ing to the lat­est ver­sion of Site­Cat­a­lyst code to take advan­tage of a num­ber of new fea­tures in H.x code, includ­ing this patch. Omniture’s “G to H Code Migra­tion” white paper, avail­able within SiteCatalyst’s help site, dis­cusses another option for code upgrade; this method allows you to update your core JavaScript file with­out chang­ing page-level tags. If nei­ther of these rec­om­men­da­tions is pos­si­ble, please con­tact your Omni­ture Account Man­ager, who will be able to dis­cuss alter­na­tive solutions.

Will this affect my Search­Cen­ter report­ing?
No. Search­Cen­ter uses a track­ing code that is passed into Site­Cat­a­lyst in the land­ing page URL query string rather than the refer­rer for report­ing. Search­Cen­ter report­ing and bid man­age­ment func­tion­al­ity will not be affected by this change.

Will this affect Inter­nal URL Fil­ters?
The con­cern here is that, because your own domain is in the refer­ring URL (in the query string), SiteCatalyst’s Inter­nal URL Fil­ters will cause the refer­rer to be con­sid­ered inter­nal and there­fore not reported. For­tu­nately, Inter­nal URL Fil­ters are not applied to the query string of the refer­rer, so this change will not cause Google-referred traf­fic to be con­sid­ered internal.

Will this affect non-search Google traf­fic (e.g., refer­rers from mail​.google​.com)?
The exis­tence of the “q=” query para­me­ter in refer­ring URLs from Google indi­cates a user search; it will only exist when Google links to your site as a result of a user search. Links to your site in other Google prod­ucts, such as Gmail, will not be affected, as these links are typ­i­cally pro­vided with­out the user hav­ing per­formed a key­word search.

Why does Site­Cat­a­lyst rely on the key­word query para­me­ter to iden­tify a search engine?
Many search engines, includ­ing Google, offer many ser­vices (e-mail, news, blogs, etc.) in addi­tion to their search functionality. These ser­vices often do not involve actual user searches, but may link to your site. By requir­ing the “q=” query para­me­ter in the refer­rer for a Google refer­rer to be con­sid­ered a search engine refer­rer, Site­Cat­a­lyst ensures that clicks to your site from other Google ser­vices, such as Gmail, do not inflate Search Engines and Search Key­words reports.

How will this change affect report­ing in the Refer­rers and Refer­ring Domains reports?
Because these reports do not rely on spe­cific indi­ca­tors within the refer­ring URL, the data and met­rics recorded in these reports gen­er­ally will not be affected.

1 comments
Adam Berlinger
Adam Berlinger

Ben; What is the solution when ASP.net strips off the referring URL after the redirect page sends the visitors the landing page? Thanks, AB