Beginning in April 2009, Google changed the way that search engine results pages render links. Where previously these links pointed directly to your site’s landing page(s), in some cases they now point to a redirect and include the destination URL (your landing page) in the query string of the redirect URL. When this occurs, the redirect page URL is the referring URL on your landing page.

For example, you may be used to seeing referrers such as this one in your Traffic Sources reports in SiteCatalyst and Discover:

http://www.google.com/search?hl=en&q=omniture+sitecatalyst&btnG=Google+Search

The new format looks like this:

http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.yoursite.com%2Fhomepage.php&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=omniture+sitecatalyst&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w
&sig2=X8uCFh6IoPtnwmvGMULQfw

Fortunately, in almost all cases, these changes have no effect whatsoever on SiteCatalyst reporting. SiteCatalyst detects Google searches as referrers by looking for the following:

  1. The presence of “google.com” (or local equivalent, such as “google.ca” for Canada) in the referring URL
  2. The presence of Google’s search keyword query parameter (i.e. the query parameter that contains the keyworrd for which the user searched), which is “q=”.

This new referring URL still contains all of the data that SiteCatalyst uses in order to recognize Google click-throughs and searches. As you can see above, the domain is still “google.com,” and “q=” is still present in the URL (see “&q=omniture+sitecatalyst” in both URLs above). SiteCatalyst does not require this parameter to be in any particular place in the query string, as long as it is there somewhere.

However, the fact that there is a significant amount of data in this referring URL prior to the “q=” query parameter may create problems for some sites that use long URLs. As mentioned above, Google puts the destination URL (your landing page) into the query string of the redirect URL, which then becomes the referring URL on your landing page. Notice the “url=” parameter in the example above; this parameter will contain your landing page’s URL. SiteCatalyst JavaScript code truncates the referring URL at 255 characters in order to manage the length of the request to Omniture’s servers (the IE browser has a built-in 2,083-character limit on all HTTP requests).

The result is that if there are 255 characters in this redirect URL prior to the “q=” query parameter, the search keyword will be not be available to Omniture’s servers, and the Search Engines and Search Keywords reports in SiteCatalyst and other tools.

Early analysis of this change indicates that only about 5% of natural searches will be affected by the long-URL problem. However, if your site has particularly long landing page URLs (more than roughly 150 characters), your analytics may be affected at a rate higher than this average. Also note that, currently, the majority of searches are use the traditional referrer format. It is not clear when Google may roll out this change across all searches, but at present there is no need to act rashly in implementing the solution described below.

In order to ensure that collection of referrer data from Google to your site continues unabated, Omniture has developed a small patch that should be added to your s_code.js file (i.e. the global JavaScript file containing the bulk of SiteCatalyst code; the file may be named differently on your site) which moves the “q=” query parameter to the front of the referrer query string for users arriving at your site from a Google search, ensuring that it appears within the first 255 characters of the referrer.

The patch can be downloaded by clicking here.

NOTE: SiteCatalyst code versions H.20.3 and newer will contain this patch natively; you will not need to add it manually. Therefore, you may choose to update your core SiteCatalyst code to the latest version available in the Code Manager within SiteCatalyst, or as provided by your Omniture Account Manager or Consultant, to take advantage of this functionality. Upgrading the core JavaScript code from your current version may carry with it a number of other advantages, such as media tracking, speed improvements, and support for additional variables/plug-ins. Please consult your Account Manager for more information.

This patch works with all SiteCatalyst JavaScript code versions H.0 through the most recent release, and will be automatically included in future code versions available from the SiteCatalyst Admin Console. Code versions prior to H.0 (G.9 and earlier) require an update to the core JavaScript file to make the existing implementation compatible with H.x versions of code. This patch should never be added to code versions prior to H.0, as it will break their execution.

NOTE: The following changes should be made by an authorized, experienced developer who is familiar with your SiteCatalyst implementation, as they require understanding of JavaScript, object-oriented programming, and SiteCatalyst variables. Furthermore, they should be tested carefully in a development environment to ensure successful data collection prior to deployment on a production site.

Follow the steps below to add the patch to your SiteCatalyst code. More detail on these steps will be provided below.

  1. In your SiteCatalyst JavaScript code, locate a line which reads: var s=s_gi(s_account)
  2. Copy and paste the patch directly below this line, as shown in the screen shot below.
  3. Change [[INSTANCE]] at the end of the patch to the name of the SiteCatalyst object. Make sure to remove the double brackets.
  4. Save and test the JavaScript file in a development environment.

The patch should be added to your code immediately after the s_gi() function is called. This is usually done at the top of the s_code.js file. Please see the screen shot below, which shows a typical SiteCatalyst JavaScript file, including the implementation of the patch.

Patch code

Make sure that you have replaced [[INSTANCE]] at the end of the patch with the object name that your SiteCatalyst implementation uses. In the majority of implementations, this this name is “s” (without quotes). In this case, this line of code:

s._rf_hav()')}s_rf([[INSTANCE]]);

should be changed to:

s._rf_hav()')}s_rf(s);

This change is shown in the screen shot below.

Patch code

If your SiteCatalyst object is named anything other than “s”, simply enter your object name in place of “s” in the example above. Make sure to remove the brackets, and do not put quotes around the object name.

Validation of this patch is most easily performed by temporarily hard-coding the s.referrer value to an example such as the one provided below:

s.referrer="http://www.google.com/url?sa=t&source=web&ct=res&cd=7&url=http%3A%2F%2Fwww.yoursite.com%2Fhomepage.php&ei=0SjdSa-1N5O8M_qW8dQN&rct=j&q=omniture+sitecatalyst&usg=AFQjCNHJXSUh7Vw7oubPaO3tZOzz-F-u_w&sig2=X8uCFh6IoPtnwmvGMULQfw"

Run a packet monitor (such as Tamper Data for Firefox or HTTP Watch for IE) while loading the page, and check the “r=” parameter  in the Omniture request (which you can identify by the presence of “/b/ss/” in the path of the request). This parameter will contain the referring URL. It should show the “q=” query parameter toward the beginning of the URL, rather than toward the end (as in the example above).

As always, your organization’s supported users should work with Omniture ClientCare if questions arise during the process of adding and testing this patch. When successfully implemented, it will ensure that reporting on referrals from Google in the Search Engines and Search Keywords reports in SiteCatalyst, Data Warehouse, Discover. and throughout the Omniture Suite.

Frequently Asked Questions

Will this patch also cause the natural search ranking to be captured by SiteCatalyst?

No. For information on obtaining that functionality, please see the recent blog post by my colleague, Jordan LeBaron.

How can I tell if my report suite is affected by Google’s change?
Omniture recommends the following steps to help determine whether or not a report suite is affected by the change described above.

  1. Run a Referrers report (Traffic Sources > Referrers in SiteCatalyst v14).
  2. Select a date range with a reasonable amount of referrer data; this will vary from report suite to report suite.
  3. Apply a search filter of “www.google.com url” to isolate referrers containing both “www.google.com” and the new “url” query parameter.
  4. Click the e-mail icon to have the report sent to you.
  5. Click Advanced Delivery Options.
  6. Under “Report Format,” choose CSV format.
  7. Under “Report Contents,” set the number of rows to 50000 (with no comma).
  8. Enter your e-mail address in the “To” field.
  9. Click the “Scheduling Options” tab and ensure that “Send report now” is selected.
  10. Click Send.
  11. When the report arrives, open it in Excel.
  12. Search for rows where “q=” does not exist or does not have a subsequent ampersand (“&”).

The result of this search will be the rows where the referrer was truncated because of the changes described above.

If I do not add this patch to my SiteCatalyst code, will I lose all referrer data from Google?
This is unlikely. If a link to your site on a Google search results page points contains a relatively short URL, it is likely that the “q=” query parameter will be present in the referring URL in spite of the full landing page URL being included in the referrer. This will allow Google referrers to be tracked in the Search Engines and Search Keywords reports. However, if landing page URLs are longer — for example, if they contain many query parameters — the patch may be necessary to ensure that the search keyword parameter is present after trunaction to 255 characters.

Do I need this patch if my site URLs are short?
Google may add additional query parameters in its redirect URLs in the future, so even if your landing page URLs are less than 150 characters long, Omniture recommends adding this patch or upgrading to the latest version of SiteCatalyst JavaScript code within the next few months.

How can I tell what version of code I am using?
This is most easily done by searching for a line of code that contains “SiteCatalyst code version.” In the example shown above, this line is at the top of the s_code.js file and shows that the code version is H.20.2. This version is part of the “H” generation of code and is therefore compatible with the patch.

What should I do if my site uses a version of SiteCatalyst code prior to H.0?
Omniture recommends upgrading to the latest version of SiteCatalyst code to take advantage of a number of new features in H.x code, including this patch. Omniture’s “G to H Code Migration” white paper, available within SiteCatalyst’s help site, discusses another option for code upgrade; this method allows you to update your core JavaScript file without changing page-level tags. If neither of these recommendations is possible, please contact your Omniture Account Manager, who will be able to discuss alternative solutions.

Will this affect my SearchCenter reporting?
No. SearchCenter uses a tracking code that is passed into SiteCatalyst in the landing page URL query string rather than the referrer for reporting. SearchCenter reporting and bid management functionality will not be affected by this change.

Will this affect Internal URL Filters?
The concern here is that, because your own domain is in the referring URL (in the query string), SiteCatalyst’s Internal URL Filters will cause the referrer to be considered internal and therefore not reported. Fortunately, Internal URL Filters are not applied to the query string of the referrer, so this change will not cause Google-referred traffic to be considered internal.

Will this affect non-search Google traffic (e.g., referrers from mail.google.com)?
The existence of the “q=” query parameter in referring URLs from Google indicates a user search; it will only exist when Google links to your site as a result of a user search. Links to your site in other Google products, such as Gmail, will not be affected, as these links are typically provided without the user having performed a keyword search.

Why does SiteCatalyst rely on the keyword query parameter to identify a search engine?
Many search engines, including Google, offer many services (e-mail, news, blogs, etc.) in addition to their search functionality. These services often do not involve actual user searches, but may link to your site. By requiring the “q=” query parameter in the referrer for a Google referrer to be considered a search engine referrer, SiteCatalyst ensures that clicks to your site from other Google services, such as Gmail, do not inflate Search Engines and Search Keywords reports.

How will this change affect reporting in the Referrers and Referring Domains reports?
Because these reports do not rely on specific indicators within the referring URL, the data and metrics recorded in these reports generally will not be affected.

1 comments
Adam Berlinger
Adam Berlinger

Ben; What is the solution when ASP.net strips off the referring URL after the redirect page sends the visitors the landing page? Thanks, AB