Let’s start with a sum­mary of what you’ve been read­ing. In an effort to make “search more secure,” Google is encrypt­ing more search queries through SSL. Searchers signed-in to a Google account are redi­rected to https://​www​.google​.com (notice the s) and (here’s the rub) search key­words are stripped from the refer­rer [cor­rec­tion fol­lows] when a nat­ural key­word is clicked. The typed key­word is hid­den from you and your site opti­miza­tion sys­tems (that’s us).

What’s going to change? Nat­ural search key­words and search engines will be under-reported in Site­Cat­a­lyst. If you’ve imple­mented organic search inte­gra­tion in Search­Cen­ter you will see a reduc­tion in organic search term traf­fic. How­ever, Search­Cen­ter paid key­word data is not affected.

Google engi­neer Matt Cutts esti­mates that search encryp­tion will impact less than 10% of organic search queries. To date we have seen that 0.5% to 2% of Google searches are affected, though we expect that num­ber to increase. As a per­cent of total search traf­fic, we’ve seen a 0.3% to 1.5% impact.

Rather than hope the 0.5%, 2%, or 10% ratio applies to your site, fol­low these steps to mea­sure the impact directly:

1. Run the Traf­fic Sources > Refer­rers report using the Instance metric

Referrers Report

2. Select the cur­rent day in the date selector

3. Fil­ter the report to show refer­rers from major search engines by adding this to the search box: google​.com/​s​e​a​rch OR google​.com/​url OR search​.yahoo​.com OR bing​.com. Some tips:

  • Add non-US domain end­ings, like google​.co​.uk, where appropriate
  • Don’t for­get to cap­i­tal­ize the OR operator
  • In Site­Cat­a­lyst 14, this search may take a few minutes

Major Search Engine Filter

4. Note the total in the bottom-right cor­ner of the page (I drop it into Excel, but your mem­ory may be bet­ter than mine)

Total

5. Fil­ter the report to encrypted search refer­rers by search­ing for google​.com/​url AND q=& (again, cap­i­tal­iza­tion mat­ters for the AND operator)

Encrypted Search Filter

6. Divide the sec­ond total by the first to esti­mate the change to total search traf­fic caused by encrypted search.

Because the google​.com domain is used for so much more than search, Site­Cat­a­lyst only counts a “search” when there is a vis­i­ble key­word. This pre­vents mail​.google​.com from being counted as a search engine.  Unfor­tu­nately, this will also pre­vent Google encrypted searches from being counted as search traf­fic. As a result, the fol­low­ing Site­Cat­a­lyst reports will be impacted:

  • Refer­rer Types: Search Engine traf­fic will decrease in favor of traf­fic for “Other Web Sites”
  • Search Key­words (All, Paid and Nat­ural): Met­rics asso­ci­ated with key­words will decrease in favor of “None.”
  • Search Engines (All, Paid and Nat­ural): Met­rics asso­ci­ated with Google search engines will decrease in favor of “None.”

We are cur­rently eval­u­at­ing changes to the pro­cess­ing plat­form to improve report­ing of encrypted search traf­fic, though there is no way to uncover the orig­i­nal search query.

Impact on Land­ing Page Optimization

Google search encryp­tion will affect retar­get­ing and land­ing page adap­ta­tion if site con­tent is cus­tomized based on the query para­me­ter. How­ever, because search encryp­tion does not apply to all data, there should be an action­able sam­ple of data for both paid and organic search mar­keters to per­form con­ver­sion analy­sis and land­ing page opti­miza­tion (espe­cially since Bing and Yahoo do not offer encrypted search options).

For more infor­ma­tion, Search Engine Land has a writ­ten a detailed post about Google’s recent search encryp­tion changes here: http://​searchengineland​.com/​g​o​o​g​l​e​-​t​o​-​b​e​g​i​n​-​e​n​c​r​y​p​t​i​n​g​-​s​e​a​r​c​h​e​s​-​o​u​t​b​o​u​n​d​-​c​l​i​c​k​s​-​b​y​-​d​e​f​a​u​l​t​-​9​7​435

The esteemed Chris Haleua from the Search­Cen­ter prod­uct man­age­ment team con­tributed heav­ily to this post (in other words, if you don’t like it, blame him).

UPDATE (2): Start­ing Tues­day, 8 Thurs­day, 10 Novem­ber 2011, you will see the term “Key­word Unavail­able” in the Search Key­words (All and Nat­ural), Mar­ket­ing Chan­nel Detail and Search­Cen­ter Organic to Paid Key­word reports. Addi­tion­ally, any loss in Nat­ural Search attri­bu­tion for Google in the Search Engines report will return to nor­mal lev­els. This change will not be retroac­tive. The grad­ual decrease you’ve noticed in the Search Engines, Mar­ket­ing Chan­nel, and Refer­rer Types reports will jump back up by 2–4% 14–20% (accord­ing to most cus­tomers I’ve spo­ken with) start­ing Thurs­day after­noon, Pacific Time.

UPDATE 3: The changes are live. You will now see “Key­word Unavail­able” in key­word reports and Search chan­nel and refer­rer types will return to nor­mal lev­els. If you’re using the Chan­nel Man­ager plu­gin, you will need to use an updated version.

We are still inves­ti­gat­ing the sta­tus of Uni­fied Sources DB VISTA rules, and hope to have an update on those soon.

Thank you for the com­ments on this topic, both here and in the Ideas Exchange.

  • http://www.120feet.com Jonathan Kay

    Very use­ful post — thanks.

    Happy to share that I’m see­ing a 2.5% impact for one of my UK clients when look­ing at google​.com traf­fic to their .co​.uk site (for this morn­ing). Google hasn’t yet made the change to google​.co​.uk

  • Hugh W

    Is there going to be an update to site­cat­a­lyst so that vis­its sat­is­fy­ing the con­di­tions in point 5 can be cor­rectly attrib­uted in the search engines and refer­rer types report? The lack of a key­word shouldn’t impact the abil­ity to iden­tify these vis­its as from Google organic search as there is still suf­fi­cient infor­ma­tion in the refer­rer to dif­fer­en­ti­ate these from google non-search (such as iGoogle).

  • Jeff
  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    Hugh, you’re right. There is enough infor­ma­tion in the refer­rer to iden­tify this as nat­ural search and we are cur­rently eval­u­at­ing a Site­Cat­a­lyst update. We could, for exam­ple, just set the key­word value to “Encrypted Search” when it’s empty. I’m inter­ested in your thoughts on what would make the most sense to users.

  • Paulsen
  • http://RedcatsUSA Syl­vain

    Hi,
    If Google change only impacts Nat­ural Search, why is it impact­ing Omni­ture Paid Search Key­words and Engines report?
    Thank you,
    Sylvain

  • http://www.omniture.com Chris Haleua

    @Paulsen Thanks for the proac­tive work on that plu­gin. A native solu­tion is being con­sid­ered and we will make sure to update the blog once we know more.

    @Sylvain The Paid search reports should not be impacted because they should always have a query value. Can you tell us more about how your report is being impacted?

  • http://management.curiouscatblog.net/ John Hunter

    Very inter­est­ing. I am sur­prised the % of “lost data” will be so low.

    The arti­cle you ref­er­ence is very inter­est­ing. It is good that, at least, google web­mas­ter cen­tral will con­tinue to show all key­word data. It is sad to lose nay key­word data though — it is very interesting.

    Set­ting to “encrypted search” does seem much bet­ter than nothing.

  • Nate Orshan

    Cheers Bret & Chris for the expla­na­tion. So far for the past week I’m see­ing a 2.8% to 3.5% loss of recorded Google organic KW (i.e., “instances of Google searches with a blank ‘q’ para­me­ter” divided by “total instances of Google searches”) across a cou­ple of sites. Not the end of the world…but def­i­nitely dis­turb­ing, espe­cially given that “Big G” could turn off the refer­rer spigot com­pletely on a whim.

    I’d also rec­om­mend Danny Sullivan’s “Google Puts A Price On Pri­vacy” ( http://​searchengineland​.com/​g​o​o​g​l​e​-​p​u​t​s​-​a​-​p​r​i​c​e​-​o​n​-​p​r​i​v​a​c​y​-​9​8​029 ) to get a sense of the right­eous out­rage directed at Google’s hypocrisy for start­ing to cut off organic key­word data to site pub­lish­ers in the name of pro­tect­ing user privacy…but not doing like­wise for AdWords key­word data.

  • Emily Hill

    @Bret … weigh­ing in on Hugh’s feed­back. Yes, iden­ti­fy­ing it as Encrypted Search bucket would be GREAT. (and even bet­ter than Google Analytic’s nam­ing of the “Not Pro­vided” bucket, which would surely elicit many ques­tions from users of “What’s the dif­fer­ence between ‘None’ and ‘Not Pro­vided’ in the Omni­ture search reports?)

    I know our SEM team works very hard for every 1–2% of organic search traf­fic they can grow, so this is a much needed enhance­ment from that per­spec­tive even if we can’t see the actual search terms.

    Hugh logged this on Ideas Exchange, so any­one who agrees — please go Pro­mote it here: http://​ideas​.omni​ture​.com/​t​5​/​A​d​o​b​e​-​I​d​e​a​-​E​x​c​h​a​n​g​e​-​f​o​r​-​O​m​n​i​t​u​r​e​/​G​o​o​g​l​e​-​l​o​g​g​e​d​-​i​n​-​s​e​a​r​c​h​e​s​/​i​d​i​-​p​/​5​723

  • Scott

    Great post, thanks for being proac­tive in explain­ing the impli­ca­tions to us. We use the Uni­fied Sources DB VISTA rule for our refer­ral traf­fic, does this change also impact that? If so are you also con­sid­er­ing a revi­sion to that, sim­i­lar to the native solu­tion you’re explor­ing for SC? Thanks.

  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    Thanks for the com­ments, every­one. I’ll update this post as soon as we have more infor­ma­tion about the change to how we’re report­ing searches.

    Jeff, very astute com­ment about 25k refer­rer limit for large sites. The good news is that this limit is applied after the day is com­plete. If you limit your time period to the cur­rent day, you’ll pull from the full data set.

  • Vedha

    Did some­thing hap­pen with the way these fil­ters are set up? Until Sun­day a client’s encrypted searches were between 0.6%-1.7% which spiked to 6% on Mon­day and 11% on Tues­day. As of 2 Nov, 1:30pm ESP we were still trend­ing at 11%. I want to make sure this method­ol­ogy still works right before hit­ting the panic button.

  • Bret Gun­der­sen

    Vedha,

    There have been no changes to how the fil­ters work in Site­Cat­a­lyst. Just make sure you’re doing your analy­sis on the cur­rent day’s refer­rers, to avoid run­ning into the unique URL limit Jeff mentioned.

  • http://search-mojo.com Tad Miller

    See­ing 19% of Google Searches miss­ing key­word data as of Novem­ber 4. So much for “sin­gle digit per­cent­ages” being effected.

  • Scott Calise

    I just want to be sure that the updates you are mak­ing will also be made for those who use the Uni­fied Sources DB Vista Rule. We do not use the stan­dard refer­ral reports, but exclu­sively rely on this Vista rule.

    Can some­one please con­firm?
    Thanks,
    Scott

    • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

      Scott, we are still eval­u­at­ing whether and how to update exist­ing DB VISTA rules, but I will update this post when I have more information.

  • Franklin

    Does the 11÷10÷11 “fix” impact EVAR2 reports?

  • http://keystonesolutions.com Kevin Rogers

    Has there been any­thing found out yet about the impact to the Uni­fied Sources VISTA rule?

  • http://www.bodybuilding.com Der­rick

    Any news on the Uni­fied Sources DB Vista rules? At least a time­line for the fix would be greatly appre­ci­ated by all, I’m sure. Thanks!

  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    For those of you with the Uni­fied Sources DB VISTA rule in place, we have begun rolling out an update, and will con­tinue to do so over the next few months. Due to the nature of this change, we can­not update all rules at once, so it will take some time before all report suites have been updated. If you need to expe­dite the process, please work with your account man­ager to dis­cuss the options avail­able to you.

  • http://www.aboutus.org/ConsideringSelfemployment.com con­sid­er­ing self employment

    Inter­est­ing stuff, It’s fab to keep return­ing to a help­ful web­site Such as this. Many thanks.

  • Jonathan Schwarz

    Regard­ing for exam­ple this topic, can any­body tell me fur­ther mes­sage boards or even bench­mark cir­cles where com­pa­nies exchange expe­ri­ences about web ana­lyt­ics in gen­eral and prob­lems like the not pro­vided key­word in particular.

    Would be great if some­one has some infor­ma­tion for me. Thanks!

  • http://www.axekap.fr/ Axekap

    Hi,
    Won­der­ing whether the per­cent­age of miss­ing key­words vs.organic search is still going up due to Google Plus etc … or the 15–20 % of total organic is in hte ball park ?
    Thx !

  • Con­nell Blackett

    Great post, how­ever I’ve realised that by search­ing for “q=&” we’re actu­ally over­es­ti­mat­ing our encrypted searches, as there are other query string para­me­ters used by Google that match this pat­tern e.g. “oq=&”. Because the search term para­me­ter is always first we’re using “?q=&” instead.

  • Joe

    I use the Excel­Client plu­gin on a weekly (some­times daily) basis. Up until last month it was work­ing fine with key­word unavail­able. I was fil­ter­ing out “None” (it shows as Unspec­i­fied in Excel though) to remove any traf­fic that was not organic and was get­ting accu­rate results. Then last month when I fil­tered “None” it was remov­ing both Key­word Unavail­able and Unspec­i­fied which is a dras­tic reduc­tion in traffic.

    My ques­tion is.…what should I enter to fil­ter out the Unspec­i­fied traf­fic? So far I’ve tried null and Unspec­i­fied but nei­ther seems to work.

    Thanks!
    Joe

  • http://www.ecademy.com/account.php?id=90022 Start-up coach

    I savor, cause I dis­cov­ered exactly what I used to be look­ing for. You’ve ended my 4 day lengthy hunt! God Bless you man. Have a nice day. Bye