With 2009 predictions whizzing around the web, I decided to write instead about what I’m wishing for in 2009. I’m hoping for an “if you build it, he will come” moment. Without further ado…

1) Agencies Get On Board

Aside from the handful of agencies that specialize in testing and conversion optimization, I’ve seen far less progress than I expected this year. To do a sanity check, I googled “[agency name] multivariate” on the top 4 digital agencies of 2007 from AdAge. Here’s what I found:

Avenue A | Razorfish is the only search that yields any substantial result, and that is to a partner’s press release focusing on the platform rather than their own in-house expertise. Contrast that with searching for “[agency name] creative”, and you’ll find that Digitas and Ogilvy rank #1 in SEO and Avenue A | Razorfish at #3.

I know that part of the problem is that agencies have a tough time convincing clients to test. But I suspect that agencies are also not yet accountable for testing, so why deliver some options that may quantifiably fail when you can deliver a single option that unquantifiably wins? The latter requires less level-of-effort and chance of hassle. Unfortunately it also has less upside for the customer. Hopefully agencies will consider the significance of providing more value to their customers, and what that may yield for them long-term.

My reason for wanting agencies to get on board is selfish. I believe that agencies (at their best) are the strategic partners of customers, and they can help prove the value of testing. But it feels a bit like the chicken-and-the-egg. Agencies aren’t compelled to actively bring testing skills and experience in-house without clients asking for it, and clients may not be educated enough to take on testing without an agency resource!

2) Offsite + Onsite = 1 Visitor Experience

I’ve definitely seen great strides already taken this year in trying to treat a visitor’s path through offsite and onsite as one cohesive experience. Some of our customers are doing really interesting projects involving the testing of retargeted ads out on 3rd-party sites and building a single profile that can be extended and reinforced at every online touch point. I’m excited to see how that plays out in 2009.

However, I’ve also seen a lot of poorly executed landing page experiences that show there’s no discussion going on between those in charge of acquisition and those in charge of the site. If there is any confusion around where to begin testing, I’d highly recommend starting there. Landing pages are typically easier to change and tag, and transforming a post-click experience from general and 80% irrelevant to targeted and 100% relevant usually yields great lift!

3) Multivariate Testing (MVT) Becomes Another Tool in the Toolbox

There’s no question that multivariate testing is an incredibly powerful tool. But it’s certainly not the only tool you need or have to increase conversion on your site. I think of it as the hammer in the saying, “when all you have is a hammer, everything looks like a nail.” There are many efforts that multivariate testing is not the best tool for, including massive functionality changes (e.g. 5-page checkout vs. 1-page checkout) and automated behavioral targeting. The problem with looking at MVT as the all-purpose cleaner is that it limits your vision to consider other types of tests, and it also reinforces a fragmented marketing division. I can’t count how many meetings I’ve been to where the catalyst is an RFP for MVT. Once we start talking about the site’s initiatives though, we begin to understand that there is also interest around segmentation and targeting. However, that’s a separate project manager and a separate RFP, and nobody is interested in getting together in the same room.

4) Targeting Belongs in the Same Toolbox

Targeting for the sake of targeting is not always effective. I have seen numerous cases where the first attempt strikes out. Sometimes people don’t want to see their first name read back to them, but they do like seeing their search terms repeated. Others like to see an offer reinforcing the deal they saw in the ad, but they may not want to see the same exact image. Without testing, you’re still not making data-driven decisions. At best, you’re making more educated guesses.

5) Web Optimization Collaboration Takes Off

Are you sensing a trend here? : I want more collaboration between acquisition marketers and site marketers, companies and agencies, vendors and companies! I think everybody wins when we share more of our testing experience and I’m convinced there has to be a way to do it without giving up competitive intelligence. So far the weboptimization group has had some good discussion around getting testing buy-in, and some very spicy discussion around vendor comparisons. Tell me how you think a forum should be constructed to be most beneficial, and then let’s work together to make it a reality.

6) Succeed Often and Fail Fast

This one’s a little off the web optimization rails, but I wish President Obama all the best in making 2009 a year of recovery and successes. I don’t think it’s possible to not fail, especially given these tumultuous and uncertain times, so I’d just like to encourage him and all the rest of us to fail fast and keep on truckin’!

Happy New Year!

Photo credits:

Lily Chiu
Lily Chiu

Hey Florian, thanks for the insightful comments! I agree that many of a 4A agency's efforts go towards branding but I don't think that should automatically disqualify them from measurement. It seems too easy to get a pass on trying to measure and interpret how a campaign performs by putting it under the branding umbrella, even if the success may not be a single conversion event like an order or sign-up. In response to the decision-making process, I would think that the perfect way to incorporate the voice of the customer is to integrate testing into the design process. Otherwise agencies are making a deliberate decision to work in a customer-excluded silo, prioritizing their designs and hypotheses above all others. I don't see how that works as a move-forward strategy as more companies continue to test, iterate and improve. (Am I looking at the world through rose-colored glasses?!) I'd also love to hear how you and others feel about the announcement of WPP's investment in Omniture. I personally think it will go a long way towards fulfilling my first wish :) Thanks! - Lily

Florian Pihs
Florian Pihs

Coming from an agency, I am not surprised that few agencies offer MVT, or A/B testing for that matter. Some of the reasons I see are: 1) Most 4A agencies work for brand advertisers. -Most of their traffic goes to campaign sites, campaign sites are mainly in Flash, Flash is hard to MVT even A/B test are more resource intensive since you need to build 2 Flash experiences. Plus campaign sites are often not online long enough to allow effective testing & optimization, since turn around times are slow. 2) These brand advertisers often do not define success as actions taken on the website (although that is changing a bit) but as changes in brand awareness or preference. MVT testing does not move the needles on these metrics. 3) Decisions on design are made in large "all hands on" meetings. The voice of the customer does not sit on a table. 4) The closer the website get to the core business $$ the less likely companies are outsourcing key functions like analytics and testing to agencies, but build their team.