Beginning in April 2009, Google changed the way that search engine results pages render links. Where previously these links pointed directly to your site’s landing page(s), in some cases they now point to a redirect and include the destination URL (your landing page) in the query string of the redirect URL. When this occurs, the redirect page URL is the referring URL on your landing page.
For example, you may be used to seeing referrers such as this one in your Traffic Sources reports in SiteCatalyst and Discover:
The new format looks like this:
Fortunately, in almost all cases, these changes have no effect whatsoever on SiteCatalyst reporting. SiteCatalyst detects Google searches as referrers by looking for the following:
- The presence of “google.com” (or local equivalent, such as “google.ca” for Canada) in the referring URL
- The presence of Google’s search keyword query parameter (i.e. the query parameter that contains the keyworrd for which the user searched), which is “q=”.
This new referring URL still contains all of the data that SiteCatalyst uses in order to recognize Google click-throughs and searches. As you can see above, the domain is still “google.com,” and “q=” is still present in the URL (see “&q=omniture+sitecatalyst” in both URLs above). SiteCatalyst does not require this parameter to be in any particular place in the query string, as long as it is there somewhere.
The result is that if there are 255 characters in this redirect URL prior to the “q=” query parameter, the search keyword will be not be available to Omniture’s servers, and the Search Engines and Search Keywords reports in SiteCatalyst and other tools.
Early analysis of this change indicates that only about 5% of natural searches will be affected by the long-URL problem. However, if your site has particularly long landing page URLs (more than roughly 150 characters), your analytics may be affected at a rate higher than this average. Also note that, currently, the majority of searches are use the traditional referrer format. It is not clear when Google may roll out this change across all searches, but at present there is no need to act rashly in implementing the solution described below.
The patch can be downloaded by clicking here.
Follow the steps below to add the patch to your SiteCatalyst code. More detail on these steps will be provided below.
- Copy and paste the patch directly below this line, as shown in the screen shot below.
- Change [[INSTANCE]] at the end of the patch to the name of the SiteCatalyst object. Make sure to remove the double brackets.
Make sure that you have replaced [[INSTANCE]] at the end of the patch with the object name that your SiteCatalyst implementation uses. In the majority of implementations, this this name is “s” (without quotes). In this case, this line of code:
should be changed to:
This change is shown in the screen shot below.
If your SiteCatalyst object is named anything other than “s”, simply enter your object name in place of “s” in the example above. Make sure to remove the brackets, and do not put quotes around the object name.
Validation of this patch is most easily performed by temporarily hard-coding the s.referrer value to an example such as the one provided below:
Run a packet monitor (such as Tamper Data for Firefox or HTTP Watch for IE) while loading the page, and check the “r=” parameter in the Omniture request (which you can identify by the presence of “/b/ss/” in the path of the request). This parameter will contain the referring URL. It should show the “q=” query parameter toward the beginning of the URL, rather than toward the end (as in the example above).
As always, your organization’s supported users should work with Omniture ClientCare if questions arise during the process of adding and testing this patch. When successfully implemented, it will ensure that reporting on referrals from Google in the Search Engines and Search Keywords reports in SiteCatalyst, Data Warehouse, Discover. and throughout the Omniture Suite.
Frequently Asked Questions
Will this patch also cause the natural search ranking to be captured by SiteCatalyst?
No. For information on obtaining that functionality, please see the recent blog post by my colleague, Jordan LeBaron.
How can I tell if my report suite is affected by Google’s change?
Omniture recommends the following steps to help determine whether or not a report suite is affected by the change described above.
- Run a Referrers report (Traffic Sources > Referrers in SiteCatalyst v14).
- Select a date range with a reasonable amount of referrer data; this will vary from report suite to report suite.
- Apply a search filter of “www.google.com url” to isolate referrers containing both “www.google.com” and the new “url” query parameter.
- Click the e-mail icon to have the report sent to you.
- Click Advanced Delivery Options.
- Under “Report Format,” choose CSV format.
- Under “Report Contents,” set the number of rows to 50000 (with no comma).
- Enter your e-mail address in the “To” field.
- Click the “Scheduling Options” tab and ensure that “Send report now” is selected.
- Click Send.
- When the report arrives, open it in Excel.
- Search for rows where “q=” does not exist or does not have a subsequent ampersand (“&”).
The result of this search will be the rows where the referrer was truncated because of the changes described above.
If I do not add this patch to my SiteCatalyst code, will I lose all referrer data from Google?
This is unlikely. If a link to your site on a Google search results page points contains a relatively short URL, it is likely that the “q=” query parameter will be present in the referring URL in spite of the full landing page URL being included in the referrer. This will allow Google referrers to be tracked in the Search Engines and Search Keywords reports. However, if landing page URLs are longer — for example, if they contain many query parameters — the patch may be necessary to ensure that the search keyword parameter is present after trunaction to 255 characters.
Do I need this patch if my site URLs are short?
How can I tell what version of code I am using?
This is most easily done by searching for a line of code that contains “SiteCatalyst code version.” In the example shown above, this line is at the top of the s_code.js file and shows that the code version is H.20.2. This version is part of the “H” generation of code and is therefore compatible with the patch.
What should I do if my site uses a version of SiteCatalyst code prior to H.0?
Will this affect my SearchCenter reporting?
No. SearchCenter uses a tracking code that is passed into SiteCatalyst in the landing page URL query string rather than the referrer for reporting. SearchCenter reporting and bid management functionality will not be affected by this change.
Will this affect Internal URL Filters?
The concern here is that, because your own domain is in the referring URL (in the query string), SiteCatalyst’s Internal URL Filters will cause the referrer to be considered internal and therefore not reported. Fortunately, Internal URL Filters are not applied to the query string of the referrer, so this change will not cause Google-referred traffic to be considered internal.
Will this affect non-search Google traffic (e.g., referrers from mail.google.com)?
The existence of the “q=” query parameter in referring URLs from Google indicates a user search; it will only exist when Google links to your site as a result of a user search. Links to your site in other Google products, such as Gmail, will not be affected, as these links are typically provided without the user having performed a keyword search.
Why does SiteCatalyst rely on the keyword query parameter to identify a search engine?
Many search engines, including Google, offer many services (e-mail, news, blogs, etc.) in addition to their search functionality. These services often do not involve actual user searches, but may link to your site. By requiring the “q=” query parameter in the referrer for a Google referrer to be considered a search engine referrer, SiteCatalyst ensures that clicks to your site from other Google services, such as Gmail, do not inflate Search Engines and Search Keywords reports.
How will this change affect reporting in the Referrers and Referring Domains reports?
Because these reports do not rely on specific indicators within the referring URL, the data and metrics recorded in these reports generally will not be affected.