Shortly before we were married, my wife and I went on a trip to Brazil. Preparing for the trip, I learned that Rio’s unique combination of steep mountains on a breezy coast offered one activity I was unlikely to experience anywhere else in the world–urban hang-gliding.
The first bad sign was my designated instructor’s pronounced limp. His scars were long and his pain seemed fresh. “Should I really jump off of a cliff with this person?” I asked myself as I signed my life away. We hopped in a cab and ascended the mountain.

The silence was unusual. Brazilians are the most engaging people you will ever meet, yet this guy was one of the most unfriendly people I had ever met, anywhere. Two-thirds of the way up the mountain he prepped me for the flight. “When we get to the platform, we will strap ourselves to the kite,” he said looking through the windshield as if instructing a tree. “When I say ‘Run!’ you run. When you get to the edge of the platform, you keep running. Whatever you do, do not stop running.” And then he turned back, locked his gaze on me, and telegraphed the origins of his limp, “Do not stop running! This is what gives me NIGHTMARES!”

Years later, as I reflect on my youthful adventures, I ask myself “What in my professional life gives me nightmares? If I could give my clients just one instruction that would lodge itself directly into their fear center, what would it be?” Oddly, it’s not that hard—it would be to never do a complete site redesign without iterative testing.

I once joined a company in the midst of a redesign. The prospect-facing site had been redesigned and recoded from the ground up by an expensive agency. To their credit, they did try to test the new site, though not iteratively, but their test design was faulty and they didn’t leverage the knowledge of their optimization consultants.

The new site bombed and no one knew why. The homepage, every single product page, the navigation, the logo, the checkout process–everything had changed. Where was the hole that was sinking the ship? There were just too many places to look. In the end, the new site was pushed to all traffic despite the negative consequences. The next year of my professional life, I spent optimizing the new site to get it to perform as well as the old site.

After becoming an optimization consultant with Adobe, one of my clients redesigned their checkout process and, against our advice, launched it without testing. Their conversion rate instantly dropped by 25%. Shockingly, they had no technical mechanism to undo the disastrous code changes. (One great feature of Test&Target is that any testing success or failure can easily be “dialed up or down” to maximize revenue without IT involvement).

In both of these examples, the companies certainly ran off the cliff, but they didn’t take the steps necessary to fly. At Adobe, we always recommend an iterative approach to testing. That’s not to say you can’t be bold. Go ahead, have your agency redesign everything from the ground up–but launch the redesign incrementally. Below is an example progression of an Iterative Redesign Test Plan:

  1. Homepage
  2. Category page template
  3. Article/Product page template
  4. Global elements (navigation, header, logo, footers, colors, etc. , tested separately)
  5. Subscription/Checkout/Lead submission process

These can be done in any order and there are many ways to customize your plan based on resource availability: plan on several rounds of testing for each.

By testing incrementally, you will be able to spot which changes help your key metrics and which changes hurt. At my former company, we found that it was just a single change made to the checkout process that caused the performance drop! You can adjust your approach immediately and keep moving forward.

The biggest resistance to iterative testing is companies think it will slow them down. Think of each iterative test as another footstep down the ramp, building momentum, so that when you go over the cliff you can actually fly.

0 comments