There is a pesky fear that pervades the marketing world around the testing of digital marketing content. The perception is that testing will somehow replace or limit the creative brainstorming and “gut instincts” of a marketing team or ad agency, and their desire to push the boundaries of engagement.  That somehow “content determination” or uncovering what works best with different customers will make cutting edge campaigns less desirable.

On the contrary, it allows companies to quickly try new engagement styles and methods, and see what “clicks” with their customers immediately, rather than wasting time with material that doesn’t.  Optimization tools such as Adobe Test&Target have also become extremely user friendly;  they can be controlled completely by a non-technical marketing team, with IT involvement at an advisory level (freeing up resources and offering efficient control by the business owners.)  In terms of investment, many companies are seeing impressive lifts in conversion (10s to 1000s of percentage lifts and beyond!) with minor adjustments to their web experiences, within months and weeks of starting, recouping yearly investments quickly.  This is why optimization and marketing managers have quickly become evangelists of the testing process within their organizations, and it’s always a blast to meet with them at different events throughout the year to “geek out” over their latest findings.

I spent 3 inspiring days at the Hudson Hotel in Manhattan last week at an optimization event called Click Summit.  For those of you who are not familiar with it, I highly recommend signing up: It is hosted by Brooks Bell, a company which specializes in assisting enterprise-level companies and SMBs with organizing, building and perfecting their online testing programs within their current infrastructure. Owner and President Brooks Bell has a true passion for testing and has been a strong evangelist for the optimization process for a decade.  For the past 3 years, she has gathered folks from the tech, media, retail, and finance industries and other verticals, as well as industry analysts, for testing roundtable discussions at Click Summit.  It calls into question why we all don’t get together more often: “A-ha” moments occurred every few minutes at the event (no, we were not singing music from the 1980s rock band.) It was immensely helpful to identify common obstacles in test administration, and how others are working towards resolving these issues.  We also shared best practices in terms of building momentum and being a good advocate for a testing culture within our organizations.

Hudson Hotel New York


Here are some of my “a-ha” moments, that “won’t be goooooooooone in a daaaaaaaaayyyyyyy” (OK, so now I am singing 80s music – couldn’t resist):

1)      The State of the Testing Union: First and foremost, I was extremely intrigued by Forrester analyst Joe Stanhope’sState of Online Testing” survey presentation, in terms of current adoption of testing practices and what the future will hold.  On average, most companies are just beginning to appreciate the importance of testing and targeting in terms of improving customer engagement and lifetime customer value. 50% of survey participants were new to testing, but a substantial number were willing to spend upwards of $100K per year on a testing budget. Most are only running 2-3 non-concurrent tests per month, and 16% of those that are testing are unclear as to what a multivariate test is.  This points to the need for greater education on more effective testing practices, beyond the single A/B test at high traffic locations.  Looking to the future, Joe also mentioned that there is a surge towards behavioral targeting, or targeting content based upon highly predictive anonymous variables relative to a customer’s behavior.  This allows for greater personalization of content to the individual based upon their statistical preferences.  Behavioral Targeting capabilities are now being offered in an automated fashion, such as within Test&Target 1:1, which allows predictive delivery of the most engaging content to anonymous customer profiles on highly trafficked webpages.

2)      Data is only useful if it’s actionable: whether we’re talking analytics, or 3rd party customer information accessed through your own data warehouse or a third party vendor, this data is only useful if you can use it to build out your user profiles and make your segmentation and targeting more refined, improving your accuracy when delivering relevant content.  A strong majority of this data can be customized for aggregation with the 1st party data within your testing tools, to bolster user profiles and make them more complete.

3)      Take risks: rather than limiting your creative content in terms of executives’ or marketing team’s “gut feelings”, create hypotheses that test new engagement approaches and see if they click with your customers.  With testing, you often have the ability to “fail fast”, or see what doesn’t work relatively quickly, and results don’t lie (as long as your tests are administered correctly.)  Look for a tool that allows flexibility and scalability in terms of the quantity of content and how you test it (it was great to hear that a lot of attendees felt that there is a lot of freedom and flexibility with Test&Target – its open platform and scalability allows for a larger quantity of variables and segments, vast opportunities for integration, as well as multi-page, onsite/offsite testing capabilities.)

4)      Take into account ancillary/adjacent content or KPIs around your test: it can be easy to overlook the impact of our content decisions further down the conversion funnel or customer experience. While an offer can be optimized and targeted for a particular customer segment, the next page or process on your website may be impacted negatively, and customer behavior should be measured and tested to optimize around your test event.

5)      Determine what not to test: don’t waste your time with material or locations on your webpages that don’t move the needle.  Instead, set up tests (multivariate tests are exceptional for this) which show you what areas or content elements are not critical in terms of user engagement, so you can focus on testing and targeting the material that is.  Time and budget savings!!!

6)      IP addresses offer powerful information: an IP address can tell you a lot of information about your visitor, even if their cookies do not.  For example, a static IP address can identify that this person is accessing through a company account and information about that company.  It is very important to respect a customer’s privacy, though; the data you are gathering should be evaluated in terms of maintaining and respecting a customer’s anonymity.  Good rule of thumb: keep abreast of standard practices and rules and regulations surrounding privacy, and how you might feel if particular data were gathered by a company you were engaging with.

7)      Free trials can be doubly useful: free trials offer great incentives to new customers to experience and ultimately purchase your product, but they also have the added bonus of allowing you to target customers at particular stages in the trial process.  This targeted content can be tested to see what offers or information are the most engaging for different groups.  “Did you know you can do this?” emails can give additional information on overlooked features of your product, as well as subscription notifications, renewal info and updates as their trial is coming to an end.

Brooks Bell at Click Summit

My final “a-ha” moment from the event was underscored in Brooks Bell’s opening remarks and described in depth by marketing guru Seth Godin’s keynote address. Seth made allusions to “Mad Men”, how advertisers and marketers in the past would mark success by how much they could get an ad out there; the more it played, the more people saw it, whether the material was relevant or not.  He also spoke to how the traditional bell curve of customer consumption has now shifted outwards, as customers’ desires and expectations are pushed to the extremes due to the heightened competition for attention, the desire for personalized experiences and the viral nature of the web.   With so many content choices and channels, the traditional methods of marketing and advertising are no longer adequate, as witnessed by the precipitous fall of the record labels when music found its way on the web.

So, what’s the solution?  How can a company quickly find its audience, compete in terms of creative approach and style, and deliver the best customer service across all digital devices in the Wild West of the web?   One of the best investments of time and money when rolling out a brand new cutting edge campaign is in testing and optimization.  I’m thrilled to see so many companies, involved in Click Summit and other testing events, making so much headway in terms of building their testing programs and lifting their customer satisfaction and conversions to new heights.  The freedom offered by “content determination” through testing and targeting will allow them to adapt to the changing trends of customer needs, as our digital world continues to evolve at such a rapid pace.