Let’s start with a sum­mary of what you’ve been read­ing. In an effort to make “search more secure,” Google is encrypt­ing more search queries through SSL. Searchers signed-in to a Google account are redi­rected to https://​www​.google​.com (notice the s) and (here’s the rub) search key­words are stripped from the refer­rer [cor­rec­tion fol­lows] when a nat­ural key­word is clicked. The typed key­word is hid­den from you and your site opti­miza­tion sys­tems (that’s us).

What’s going to change? Nat­ural search key­words and search engines will be under-reported in Site­Cat­a­lyst. If you’ve imple­mented organic search inte­gra­tion in Search­Cen­ter you will see a reduc­tion in organic search term traf­fic. How­ever, Search­Cen­ter paid key­word data is not affected.

Google engi­neer Matt Cutts esti­mates that search encryp­tion will impact less than 10% of organic search queries. To date we have seen that 0.5% to 2% of Google searches are affected, though we expect that num­ber to increase. As a per­cent of total search traf­fic, we’ve seen a 0.3% to 1.5% impact.

Rather than hope the 0.5%, 2%, or 10% ratio applies to your site, fol­low these steps to mea­sure the impact directly:

1. Run the Traf­fic Sources > Refer­rers report using the Instance metric

Referrers Report

2. Select the cur­rent day in the date selector

3. Fil­ter the report to show refer­rers from major search engines by adding this to the search box: google​.com/​s​e​a​rch OR google​.com/​url OR search​.yahoo​.com OR bing​.com. Some tips:

  • Add non-US domain end­ings, like google​.co​.uk, where appropriate
  • Don’t for­get to cap­i­tal­ize the OR operator
  • In Site­Cat­a­lyst 14, this search may take a few minutes

Major Search Engine Filter

4. Note the total in the bottom-right cor­ner of the page (I drop it into Excel, but your mem­ory may be bet­ter than mine)

Total

5. Fil­ter the report to encrypted search refer­rers by search­ing for google​.com/​url AND q=& (again, cap­i­tal­iza­tion mat­ters for the AND operator)

Encrypted Search Filter

6. Divide the sec­ond total by the first to esti­mate the change to total search traf­fic caused by encrypted search.

Because the google​.com domain is used for so much more than search, Site­Cat­a­lyst only counts a “search” when there is a vis­i­ble key­word. This pre­vents mail​.google​.com from being counted as a search engine.  Unfor­tu­nately, this will also pre­vent Google encrypted searches from being counted as search traf­fic. As a result, the fol­low­ing Site­Cat­a­lyst reports will be impacted:

  • Refer­rer Types: Search Engine traf­fic will decrease in favor of traf­fic for “Other Web Sites”
  • Search Key­words (All, Paid and Nat­ural): Met­rics asso­ci­ated with key­words will decrease in favor of “None.”
  • Search Engines (All, Paid and Nat­ural): Met­rics asso­ci­ated with Google search engines will decrease in favor of “None.”

We are cur­rently eval­u­at­ing changes to the pro­cess­ing plat­form to improve report­ing of encrypted search traf­fic, though there is no way to uncover the orig­i­nal search query.

Impact on Land­ing Page Optimization

Google search encryp­tion will affect retar­get­ing and land­ing page adap­ta­tion if site con­tent is cus­tomized based on the query para­me­ter. How­ever, because search encryp­tion does not apply to all data, there should be an action­able sam­ple of data for both paid and organic search mar­keters to per­form con­ver­sion analy­sis and land­ing page opti­miza­tion (espe­cially since Bing and Yahoo do not offer encrypted search options).

For more infor­ma­tion, Search Engine Land has a writ­ten a detailed post about Google’s recent search encryp­tion changes here: http://​searchengineland​.com/​g​o​o​g​l​e​-​t​o​-​b​e​g​i​n​-​e​n​c​r​y​p​t​i​n​g​-​s​e​a​r​c​h​e​s​-​o​u​t​b​o​u​n​d​-​c​l​i​c​k​s​-​b​y​-​d​e​f​a​u​l​t​-​9​7​435

The esteemed Chris Haleua from the Search­Cen­ter prod­uct man­age­ment team con­tributed heav­ily to this post (in other words, if you don’t like it, blame him).

UPDATE (2): Start­ing Tues­day, 8 Thurs­day, 10 Novem­ber 2011, you will see the term “Key­word Unavail­able” in the Search Key­words (All and Nat­ural), Mar­ket­ing Chan­nel Detail and Search­Cen­ter Organic to Paid Key­word reports. Addi­tion­ally, any loss in Nat­ural Search attri­bu­tion for Google in the Search Engines report will return to nor­mal lev­els. This change will not be retroac­tive. The grad­ual decrease you’ve noticed in the Search Engines, Mar­ket­ing Chan­nel, and Refer­rer Types reports will jump back up by 2–4% 14–20% (accord­ing to most cus­tomers I’ve spo­ken with) start­ing Thurs­day after­noon, Pacific Time.

UPDATE 3: The changes are live. You will now see “Key­word Unavail­able” in key­word reports and Search chan­nel and refer­rer types will return to nor­mal lev­els. If you’re using the Chan­nel Man­ager plu­gin, you will need to use an updated version.

We are still inves­ti­gat­ing the sta­tus of Uni­fied Sources DB VISTA rules, and hope to have an update on those soon.

Thank you for the com­ments on this topic, both here and in the Ideas Exchange.

27 comments
Start-up coach
Start-up coach

I savor, cause I discovered exactly what I used to be looking for. You've ended my 4 day lengthy hunt! God Bless you man. Have a nice day. Bye

Joe
Joe

I use the ExcelClient plugin on a weekly (sometimes daily) basis. Up until last month it was working fine with keyword unavailable. I was filtering out "None" (it shows as Unspecified in Excel though) to remove any traffic that was not organic and was getting accurate results. Then last month when I filtered "None" it was removing both Keyword Unavailable and Unspecified which is a drastic reduction in traffic. My question is....what should I enter to filter out the Unspecified traffic? So far I've tried null and Unspecified but neither seems to work. Thanks! Joe

Connell Blackett
Connell Blackett

Great post, however I've realised that by searching for "q=&" we're actually overestimating our encrypted searches, as there are other query string parameters used by Google that match this pattern e.g. "oq=&". Because the search term parameter is always first we're using "?q=&" instead.

Axekap
Axekap

Hi, Wondering whether the percentage of missing keywords vs.organic search is still going up due to Google Plus etc ... or the 15-20 % of total organic is in hte ball park ? Thx !

Jonathan Schwarz
Jonathan Schwarz

Regarding for example this topic, can anybody tell me further message boards or even benchmark circles where companies exchange experiences about web analytics in general and problems like the not provided keyword in particular. Would be great if someone has some information for me. Thanks!

Bret Gundersen
Bret Gundersen

For those of you with the Unified Sources DB VISTA rule in place, we have begun rolling out an update, and will continue to do so over the next few months. Due to the nature of this change, we cannot update all rules at once, so it will take some time before all report suites have been updated. If you need to expedite the process, please work with your account manager to discuss the options available to you.

Derrick
Derrick

Any news on the Unified Sources DB Vista rules? At least a timeline for the fix would be greatly appreciated by all, I'm sure. Thanks!

Kevin Rogers
Kevin Rogers

Has there been anything found out yet about the impact to the Unified Sources VISTA rule?

Franklin
Franklin

Does the 11/10/11 "fix" impact EVAR2 reports?

Scott Calise
Scott Calise

I just want to be sure that the updates you are making will also be made for those who use the Unified Sources DB Vista Rule. We do not use the standard referral reports, but exclusively rely on this Vista rule. Can someone please confirm? Thanks, Scott

Tad Miller
Tad Miller

Seeing 19% of Google Searches missing keyword data as of November 4. So much for "single digit percentages" being effected.

Bret Gundersen
Bret Gundersen

Vedha, There have been no changes to how the filters work in SiteCatalyst. Just make sure you're doing your analysis on the current day's referrers, to avoid running into the unique URL limit Jeff mentioned.

Vedha
Vedha

Did something happen with the way these filters are set up? Until Sunday a client's encrypted searches were between 0.6%-1.7% which spiked to 6% on Monday and 11% on Tuesday. As of 2 Nov, 1:30pm ESP we were still trending at 11%. I want to make sure this methodology still works right before hitting the panic button.

Bret Gundersen
Bret Gundersen

Thanks for the comments, everyone. I'll update this post as soon as we have more information about the change to how we're reporting searches. Jeff, very astute comment about 25k referrer limit for large sites. The good news is that this limit is applied after the day is complete. If you limit your time period to the current day, you'll pull from the full data set.

Scott
Scott

Great post, thanks for being proactive in explaining the implications to us. We use the Unified Sources DB VISTA rule for our referral traffic, does this change also impact that? If so are you also considering a revision to that, similar to the native solution you're exploring for SC? Thanks.

Emily Hill
Emily Hill

@Bret ... weighing in on Hugh's feedback. Yes, identifying it as Encrypted Search bucket would be GREAT. (and even better than Google Analytic's naming of the "Not Provided" bucket, which would surely elicit many questions from users of "What's the difference between 'None' and 'Not Provided' in the Omniture search reports?) I know our SEM team works very hard for every 1-2% of organic search traffic they can grow, so this is a much needed enhancement from that perspective even if we can't see the actual search terms. Hugh logged this on Ideas Exchange, so anyone who agrees - please go Promote it here: http://ideas.omniture.com/t5/Adobe-Idea-Exchange-for-Omniture/Google-logged-in-searches/idi-p/5723

Nate Orshan
Nate Orshan

Cheers Bret & Chris for the explanation. So far for the past week I'm seeing a 2.8% to 3.5% loss of recorded Google organic KW (i.e., "instances of Google searches with a blank 'q' parameter" divided by "total instances of Google searches") across a couple of sites. Not the end of the world...but definitely disturbing, especially given that "Big G" could turn off the referrer spigot completely on a whim. I'd also recommend Danny Sullivan's "Google Puts A Price On Privacy" ( http://searchengineland.com/google-puts-a-price-on-privacy-98029 ) to get a sense of the righteous outrage directed at Google's hypocrisy for starting to cut off organic keyword data to site publishers in the name of protecting user privacy...but not doing likewise for AdWords keyword data.

John Hunter
John Hunter

Very interesting. I am surprised the % of "lost data" will be so low. The article you reference is very interesting. It is good that, at least, google webmaster central will continue to show all keyword data. It is sad to lose nay keyword data though - it is very interesting. Setting to "encrypted search" does seem much better than nothing.

Chris Haleua
Chris Haleua

@Paulsen Thanks for the proactive work on that plugin. A native solution is being considered and we will make sure to update the blog once we know more. @Sylvain The Paid search reports should not be impacted because they should always have a query value. Can you tell us more about how your report is being impacted?

Sylvain
Sylvain

Hi, If Google change only impacts Natural Search, why is it impacting Omniture Paid Search Keywords and Engines report? Thank you, Sylvain

Bret Gundersen
Bret Gundersen

Hugh, you're right. There is enough information in the referrer to identify this as natural search and we are currently evaluating a SiteCatalyst update. We could, for example, just set the keyword value to "Encrypted Search" when it's empty. I'm interested in your thoughts on what would make the most sense to users.

Hugh W
Hugh W

Is there going to be an update to sitecatalyst so that visits satisfying the conditions in point 5 can be correctly attributed in the search engines and referrer types report? The lack of a keyword shouldn't impact the ability to identify these visits as from Google organic search as there is still sufficient information in the referrer to differentiate these from google non-search (such as iGoogle).

Jonathan Kay
Jonathan Kay

Very useful post - thanks. Happy to share that I'm seeing a 2.5% impact for one of my UK clients when looking at google.com traffic to their .co.uk site (for this morning). Google hasn't yet made the change to google.co.uk

Bret Gundersen
Bret Gundersen

Scott, we are still evaluating whether and how to update existing DB VISTA rules, but I will update this post when I have more information.