Adobe
Adobe Digital Marketing Blog
  • Digital Marketing
    • Mobile
    • Social Media
    • Digital Advertising
    • Search Engine Marketing
  • Analytics
  • Personalization
  • Industries
    • Financial Services
    • Media & Entertainment
    • Retail & Travel
  • Executive Insights
    • Aseem Chandra
The Economics of Maximizing Publisher Ad Revenue — Part 2 Announcing: Adobe AudienceResearch

The Impact of Google Encrypted Search

Analytics · By Bret Gundersen On October 19, 2011 · 27 Comments

Let’s start with a sum­mary of what you’ve been read­ing. In an effort to make “search more secure,” Google is encrypt­ing more search queries through SSL. Searchers signed-in to a Google account are redi­rected to https://​www​.google​.com (notice the s) and (here’s the rub) search key­words are stripped from the refer­rer [cor­rec­tion fol­lows] when a nat­ural key­word is clicked. The typed key­word is hid­den from you and your site opti­miza­tion sys­tems (that’s us).

What’s going to change? Nat­ural search key­words and search engines will be under-reported in Site­Cat­a­lyst. If you’ve imple­mented organic search inte­gra­tion in Search­Cen­ter you will see a reduc­tion in organic search term traf­fic. How­ever, Search­Cen­ter paid key­word data is not affected.

Google engi­neer Matt Cutts esti­mates that search encryp­tion will impact less than 10% of organic search queries. To date we have seen that 0.5% to 2% of Google searches are affected, though we expect that num­ber to increase. As a per­cent of total search traf­fic, we’ve seen a 0.3% to 1.5% impact.

Rather than hope the 0.5%, 2%, or 10% ratio applies to your site, fol­low these steps to mea­sure the impact directly:

1. Run the Traf­fic Sources > Refer­rers report using the Instance metric

Referrers Report

2. Select the cur­rent day in the date selector

3. Fil­ter the report to show refer­rers from major search engines by adding this to the search box: google​.com/​s​e​a​rch OR google​.com/​url OR search​.yahoo​.com OR bing​.com. Some tips:

  • Add non-US domain end­ings, like google​.co​.uk, where appropriate
  • Don’t for­get to cap­i­tal­ize the OR operator
  • In Site­Cat­a­lyst 14, this search may take a few minutes

Major Search Engine Filter

4. Note the total in the bottom-right cor­ner of the page (I drop it into Excel, but your mem­ory may be bet­ter than mine)

Total

5. Fil­ter the report to encrypted search refer­rers by search­ing for google​.com/​url AND q=& (again, cap­i­tal­iza­tion mat­ters for the AND operator)

Encrypted Search Filter

6. Divide the sec­ond total by the first to esti­mate the change to total search traf­fic caused by encrypted search.

Because the google​.com domain is used for so much more than search, Site­Cat­a­lyst only counts a “search” when there is a vis­i­ble key­word. This pre­vents mail​.google​.com from being counted as a search engine.  Unfor­tu­nately, this will also pre­vent Google encrypted searches from being counted as search traf­fic. As a result, the fol­low­ing Site­Cat­a­lyst reports will be impacted:

  • Refer­rer Types: Search Engine traf­fic will decrease in favor of traf­fic for “Other Web Sites”
  • Search Key­words (All, Paid and Nat­ural): Met­rics asso­ci­ated with key­words will decrease in favor of “None.”
  • Search Engines (All, Paid and Nat­ural): Met­rics asso­ci­ated with Google search engines will decrease in favor of “None.”

We are cur­rently eval­u­at­ing changes to the pro­cess­ing plat­form to improve report­ing of encrypted search traf­fic, though there is no way to uncover the orig­i­nal search query.

Impact on Land­ing Page Optimization

Google search encryp­tion will affect retar­get­ing and land­ing page adap­ta­tion if site con­tent is cus­tomized based on the query para­me­ter. How­ever, because search encryp­tion does not apply to all data, there should be an action­able sam­ple of data for both paid and organic search mar­keters to per­form con­ver­sion analy­sis and land­ing page opti­miza­tion (espe­cially since Bing and Yahoo do not offer encrypted search options).

For more infor­ma­tion, Search Engine Land has a writ­ten a detailed post about Google’s recent search encryp­tion changes here: http://​searchengineland​.com/​g​o​o​g​l​e​-​t​o​-​b​e​g​i​n​-​e​n​c​r​y​p​t​i​n​g​-​s​e​a​r​c​h​e​s​-​o​u​t​b​o​u​n​d​-​c​l​i​c​k​s​-​b​y​-​d​e​f​a​u​l​t​-​9​7​435

The esteemed Chris Haleua from the Search­Cen­ter prod­uct man­age­ment team con­tributed heav­ily to this post (in other words, if you don’t like it, blame him).

UPDATE (2): Start­ing Tues­day, 8 Thurs­day, 10 Novem­ber 2011, you will see the term “Key­word Unavail­able” in the Search Key­words (All and Nat­ural), Mar­ket­ing Chan­nel Detail and Search­Cen­ter Organic to Paid Key­word reports. Addi­tion­ally, any loss in Nat­ural Search attri­bu­tion for Google in the Search Engines report will return to nor­mal lev­els. This change will not be retroac­tive. The grad­ual decrease you’ve noticed in the Search Engines, Mar­ket­ing Chan­nel, and Refer­rer Types reports will jump back up by 2–4% 14–20% (accord­ing to most cus­tomers I’ve spo­ken with) start­ing Thurs­day after­noon, Pacific Time.

UPDATE 3: The changes are live. You will now see “Key­word Unavail­able” in key­word reports and Search chan­nel and refer­rer types will return to nor­mal lev­els. If you’re using the Chan­nel Man­ager plu­gin, you will need to use an updated version.

We are still inves­ti­gat­ing the sta­tus of Uni­fied Sources DB VISTA rules, and hope to have an update on those soon.

Thank you for the com­ments on this topic, both here and in the Ideas Exchange.

Tagged with: Google Encrypted Search • SearchCenter • SiteCatalyst 
  • http://www.120feet.com Jonathan Kay

    Very use­ful post — thanks.

    Happy to share that I’m see­ing a 2.5% impact for one of my UK clients when look­ing at google​.com traf­fic to their .co​.uk site (for this morn­ing). Google hasn’t yet made the change to google​.co​.uk

  • Hugh W

    Is there going to be an update to site­cat­a­lyst so that vis­its sat­is­fy­ing the con­di­tions in point 5 can be cor­rectly attrib­uted in the search engines and refer­rer types report? The lack of a key­word shouldn’t impact the abil­ity to iden­tify these vis­its as from Google organic search as there is still suf­fi­cient infor­ma­tion in the refer­rer to dif­fer­en­ti­ate these from google non-search (such as iGoogle).

  • Jeff

    Keep in mind that for larger sites, you may run into the unique refer­rer URL limit of 25k per day — https://​omni​ture​-help​.cus​thelp​.com/​a​p​p​/​a​n​s​w​e​r​s​/​d​e​t​a​i​l​/​a​_​i​d​/​1​0​4​3​/​k​w​/​r​e​f​e​r​rer

  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    Hugh, you’re right. There is enough infor­ma­tion in the refer­rer to iden­tify this as nat­ural search and we are cur­rently eval­u­at­ing a Site­Cat­a­lyst update. We could, for exam­ple, just set the key­word value to “Encrypted Search” when it’s empty. I’m inter­ested in your thoughts on what would make the most sense to users.

  • Paulsen

    The guys over at Key­stone (Kevin Rogers) have devel­oped a plu­gin to do just that from what I under­stand. Check it out here:

    http://​www​.key​stones​o​lu​tions​.com/​c​o​m​m​u​n​i​t​y​/​2​0​1​1​/​1​0​/​t​r​a​c​k​i​n​g​-​g​o​o​g​l​e​-​s​e​c​u​r​e​-​s​e​a​r​c​h​-​i​n​-​s​i​t​e​c​a​t​a​l​y​st/

    But it would be nice to have a native solution.

  • http://RedcatsUSA Syl­vain

    Hi,
    If Google change only impacts Nat­ural Search, why is it impact­ing Omni­ture Paid Search Key­words and Engines report?
    Thank you,
    Sylvain

  • http://www.omniture.com Chris Haleua

    @Paulsen Thanks for the proac­tive work on that plu­gin. A native solu­tion is being con­sid­ered and we will make sure to update the blog once we know more.

    @Sylvain The Paid search reports should not be impacted because they should always have a query value. Can you tell us more about how your report is being impacted?

  • http://management.curiouscatblog.net/ John Hunter

    Very inter­est­ing. I am sur­prised the % of “lost data” will be so low.

    The arti­cle you ref­er­ence is very inter­est­ing. It is good that, at least, google web­mas­ter cen­tral will con­tinue to show all key­word data. It is sad to lose nay key­word data though — it is very interesting.

    Set­ting to “encrypted search” does seem much bet­ter than nothing.

  • Nate Orshan

    Cheers Bret & Chris for the expla­na­tion. So far for the past week I’m see­ing a 2.8% to 3.5% loss of recorded Google organic KW (i.e., “instances of Google searches with a blank ‘q’ para­me­ter” divided by “total instances of Google searches”) across a cou­ple of sites. Not the end of the world…but def­i­nitely dis­turb­ing, espe­cially given that “Big G” could turn off the refer­rer spigot com­pletely on a whim.

    I’d also rec­om­mend Danny Sullivan’s “Google Puts A Price On Pri­vacy” ( http://​searchengineland​.com/​g​o​o​g​l​e​-​p​u​t​s​-​a​-​p​r​i​c​e​-​o​n​-​p​r​i​v​a​c​y​-​9​8​029 ) to get a sense of the right­eous out­rage directed at Google’s hypocrisy for start­ing to cut off organic key­word data to site pub­lish­ers in the name of pro­tect­ing user privacy…but not doing like­wise for AdWords key­word data.

  • Emily Hill

    @Bret … weigh­ing in on Hugh’s feed­back. Yes, iden­ti­fy­ing it as Encrypted Search bucket would be GREAT. (and even bet­ter than Google Analytic’s nam­ing of the “Not Pro­vided” bucket, which would surely elicit many ques­tions from users of “What’s the dif­fer­ence between ‘None’ and ‘Not Pro­vided’ in the Omni­ture search reports?)

    I know our SEM team works very hard for every 1–2% of organic search traf­fic they can grow, so this is a much needed enhance­ment from that per­spec­tive even if we can’t see the actual search terms.

    Hugh logged this on Ideas Exchange, so any­one who agrees — please go Pro­mote it here: http://​ideas​.omni​ture​.com/​t​5​/​A​d​o​b​e​-​I​d​e​a​-​E​x​c​h​a​n​g​e​-​f​o​r​-​O​m​n​i​t​u​r​e​/​G​o​o​g​l​e​-​l​o​g​g​e​d​-​i​n​-​s​e​a​r​c​h​e​s​/​i​d​i​-​p​/​5​723

  • Scott

    Great post, thanks for being proac­tive in explain­ing the impli­ca­tions to us. We use the Uni­fied Sources DB VISTA rule for our refer­ral traf­fic, does this change also impact that? If so are you also con­sid­er­ing a revi­sion to that, sim­i­lar to the native solu­tion you’re explor­ing for SC? Thanks.

  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    Thanks for the com­ments, every­one. I’ll update this post as soon as we have more infor­ma­tion about the change to how we’re report­ing searches.

    Jeff, very astute com­ment about 25k refer­rer limit for large sites. The good news is that this limit is applied after the day is com­plete. If you limit your time period to the cur­rent day, you’ll pull from the full data set.

  • Vedha

    Did some­thing hap­pen with the way these fil­ters are set up? Until Sun­day a client’s encrypted searches were between 0.6%-1.7% which spiked to 6% on Mon­day and 11% on Tues­day. As of 2 Nov, 1:30pm ESP we were still trend­ing at 11%. I want to make sure this method­ol­ogy still works right before hit­ting the panic button.

  • Bret Gun­der­sen

    Vedha,

    There have been no changes to how the fil­ters work in Site­Cat­a­lyst. Just make sure you’re doing your analy­sis on the cur­rent day’s refer­rers, to avoid run­ning into the unique URL limit Jeff mentioned.

  • http://search-mojo.com Tad Miller

    See­ing 19% of Google Searches miss­ing key­word data as of Novem­ber 4. So much for “sin­gle digit per­cent­ages” being effected.

  • Scott Calise

    I just want to be sure that the updates you are mak­ing will also be made for those who use the Uni­fied Sources DB Vista Rule. We do not use the stan­dard refer­ral reports, but exclu­sively rely on this Vista rule.

    Can some­one please con­firm?
    Thanks,
    Scott

    • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

      Scott, we are still eval­u­at­ing whether and how to update exist­ing DB VISTA rules, but I will update this post when I have more information.

  • Franklin

    Does the 11÷10÷11 “fix” impact EVAR2 reports?

  • http://keystonesolutions.com Kevin Rogers

    Has there been any­thing found out yet about the impact to the Uni­fied Sources VISTA rule?

  • http://www.bodybuilding.com Der­rick

    Any news on the Uni­fied Sources DB Vista rules? At least a time­line for the fix would be greatly appre­ci­ated by all, I’m sure. Thanks!

  • http://blogs.omniture.com/author/bgundersen Bret Gun­der­sen

    For those of you with the Uni­fied Sources DB VISTA rule in place, we have begun rolling out an update, and will con­tinue to do so over the next few months. Due to the nature of this change, we can­not update all rules at once, so it will take some time before all report suites have been updated. If you need to expe­dite the process, please work with your account man­ager to dis­cuss the options avail­able to you.

  • http://www.aboutus.org/ConsideringSelfemployment.com con­sid­er­ing self employment

    Inter­est­ing stuff, It’s fab to keep return­ing to a help­ful web­site Such as this. Many thanks.

  • Jonathan Schwarz

    Regard­ing for exam­ple this topic, can any­body tell me fur­ther mes­sage boards or even bench­mark cir­cles where com­pa­nies exchange expe­ri­ences about web ana­lyt­ics in gen­eral and prob­lems like the not pro­vided key­word in particular.

    Would be great if some­one has some infor­ma­tion for me. Thanks!

  • http://www.axekap.fr/ Axekap

    Hi,
    Won­der­ing whether the per­cent­age of miss­ing key­words vs.organic search is still going up due to Google Plus etc … or the 15–20 % of total organic is in hte ball park ?
    Thx !

  • Con­nell Blackett

    Great post, how­ever I’ve realised that by search­ing for “q=&” we’re actu­ally over­es­ti­mat­ing our encrypted searches, as there are other query string para­me­ters used by Google that match this pat­tern e.g. “oq=&”. Because the search term para­me­ter is always first we’re using “?q=&” instead.

  • Joe

    I use the Excel­Client plu­gin on a weekly (some­times daily) basis. Up until last month it was work­ing fine with key­word unavail­able. I was fil­ter­ing out “None” (it shows as Unspec­i­fied in Excel though) to remove any traf­fic that was not organic and was get­ting accu­rate results. Then last month when I fil­tered “None” it was remov­ing both Key­word Unavail­able and Unspec­i­fied which is a dras­tic reduc­tion in traffic.

    My ques­tion is.…what should I enter to fil­ter out the Unspec­i­fied traf­fic? So far I’ve tried null and Unspec­i­fied but nei­ther seems to work.

    Thanks!
    Joe

  • http://www.ecademy.com/account.php?id=90022 Start-up coach

    I savor, cause I dis­cov­ered exactly what I used to be look­ing for. You’ve ended my 4 day lengthy hunt! God Bless you man. Have a nice day. Bye

  • Follow Adobe Digital Marketing

    Fol­low @AdobeDigMktg
  • Popular Posts

    • Excellent Blog Post — Getting More from your Omniture Implementation.4
    • Tim Tebow and Mobile Marketing in 2012 (Part 1)2
    • Change the Conversation: What does “Efficiency” really mean?2
    • My work, My passion — Customer Analytics1
    Adobe Digital Marketing Blog

    Pages

    • Digital Marketing
    • Analytics
    • Personalization
    • Industries
    • Executive Insights

    The Latest

    • Using Dependent Code: Adding the Twitter Handle Name to Your Referring Traffic
      A common request I hear from customers is the desire to integrate […]

    More

    See how Adobe is changing the world through digital experiences. We are the leader in delivering solutions that let customers produce, distribute, and realize value from great content, whether in media and publishing or digital marketing.
    © 2012 Adobe Systems Incorporated. All Rights Reserved.
    Tweet