Shortly before we were mar­ried, my wife and I went on a trip to Brazil. Prepar­ing for the trip, I learned that Rio’s unique com­bi­na­tion of steep moun­tains on a breezy coast offered one activ­ity I was unlikely to expe­ri­ence any­where else in the world–urban hang-gliding.
The first bad sign was my des­ig­nated instructor’s pro­nounced limp. His scars were long and his pain seemed fresh. “Should I really jump off of a cliff with this per­son?” I asked myself as I signed my life away. We hopped in a cab and ascended the mountain.

The silence was unusual. Brazil­ians are the most engag­ing peo­ple you will ever meet, yet this guy was one of the most unfriendly peo­ple I had ever met, any­where. Two-thirds of the way up the moun­tain he prepped me for the flight. “When we get to the plat­form, we will strap our­selves to the kite,” he said look­ing through the wind­shield as if instruct­ing a tree. “When I say ‘Run!’ you run. When you get to the edge of the plat­form, you keep run­ning. What­ever you do, do not stop run­ning.” And then he turned back, locked his gaze on me, and telegraphed the ori­gins of his limp, “Do not stop run­ning! This is what gives me NIGHTMARES!”

Years later, as I reflect on my youth­ful adven­tures, I ask myself “What in my pro­fes­sional life gives me night­mares? If I could give my clients just one instruc­tion that would lodge itself directly into their fear cen­ter, what would it be?” Oddly, it’s not that hard—it would be to never do a com­plete site redesign with­out iter­a­tive testing.

I once joined a com­pany in the midst of a redesign. The prospect-facing site had been redesigned and recoded from the ground up by an expen­sive agency. To their credit, they did try to test the new site, though not iter­a­tively, but their test design was faulty and they didn’t lever­age the knowl­edge of their opti­miza­tion consultants.

The new site bombed and no one knew why. The home­page, every sin­gle prod­uct page, the nav­i­ga­tion, the logo, the check­out process–everything had changed. Where was the hole that was sink­ing the ship? There were just too many places to look. In the end, the new site was pushed to all traf­fic despite the neg­a­tive con­se­quences. The next year of my pro­fes­sional life, I spent opti­miz­ing the new site to get it to per­form as well as the old site.

After becom­ing an opti­miza­tion con­sul­tant with Adobe, one of my clients redesigned their check­out process and, against our advice, launched it with­out test­ing. Their con­ver­sion rate instantly dropped by 25%. Shock­ingly, they had no tech­ni­cal mech­a­nism to undo the dis­as­trous code changes. (One great fea­ture of Test&Target is that any test­ing suc­cess or fail­ure can eas­ily be “dialed up or down” to max­i­mize rev­enue with­out IT involvement).

In both of these exam­ples, the com­pa­nies cer­tainly ran off the cliff, but they didn’t take the steps nec­es­sary to fly. At Adobe, we always rec­om­mend an iter­a­tive approach to test­ing. That’s not to say you can’t be bold. Go ahead, have your agency redesign every­thing from the ground up–but launch the redesign incre­men­tally. Below is an exam­ple pro­gres­sion of an Iter­a­tive Redesign Test Plan:

  1. Home­page
  2. Cat­e­gory page template
  3. Article/Product page template
  4. Global ele­ments (nav­i­ga­tion, header, logo, foot­ers, col­ors, etc. , tested separately)
  5. Subscription/Checkout/Lead sub­mis­sion process

These can be done in any order and there are many ways to cus­tomize your plan based on resource avail­abil­ity: plan on sev­eral rounds of test­ing for each.

By test­ing incre­men­tally, you will be able to spot which changes help your key met­rics and which changes hurt. At my for­mer com­pany, we found that it was just a sin­gle change made to the check­out process that caused the per­for­mance drop! You can adjust your approach imme­di­ately and keep mov­ing forward.

The biggest resis­tance to iter­a­tive test­ing is com­pa­nies think it will slow them down. Think of each iter­a­tive test as another foot­step down the ramp, build­ing momen­tum, so that when you go over the cliff you can actu­ally fly.

0 comments