In “Your Guide to Choosing an Online Testing and Optimization Solution,” Brian Hawkins, a partner at Web Analytics Demystified, outlines the core considerations and best practices in the selection and integration of online testing and optimization solutions. He even provides a step-by-step requirements checklist for identifying a solution and partner—what’s important to the organization, what’s supported by the vendor, and the effort required—to help marketers and stakeholders determine what’s core to building and applying their initial campaigns. However, ultimately, Hawkins states that the introduction of these systems must be wholly informed by a host of organizational considerations, capabilities, existing systems, and cultures.

Where do you start? According to Hawkins, it’s with a clear pronouncement and alignment of organizational goals, a clear path to integration, and a thorough assessment of prospective partners.

Map Your Goals—And Your Current Ability to Meet Them

You likely already have some sense of the desired outcomes of your testing and optimization efforts. Are you an e-commerce site looking to increase conversion, raise the average order value, or upsell current consumers? Want to deliver targeted content to site visitors based on the types of financial services they’re searching for? Increase video engagement, article consumption, or registrations? Something else?

Optimization service providers can help you connect the dots when testing implications—what systems exist and how they will help you deepen and measure these goals—but it’s also critical to look internally. Who will manage the process from your end? Does your organization have the know-how, or is training required? How will you integrate the analytics that emerge into existing data sets and processes? And, beyond Hawkins’ considerations, is your organization culturally and systematically mature enough and ready for these shifts? How much of an optimization organization are you—and will your efforts be supported?

Determine the “What” and the “How”

For those considering online testing and optimization for the first time, Hawkins provides a simple overview of the types of tests and testing channels for initial and ongoing efforts. Short, compelling case studies illustrate their respective benefits.

Test Types Testing Channels
A/B tests Email
Multivariate tests Mobile
Auto-optimizing Display
Mutual exclusivity Internal search and recommendations

As Hawkins explains, “targeting or segmenting subsets of visitors for analysis is the most efficient way to get as much value from your tests and testing efforts,” steering marketers toward testing service providers who offer targeted testing and segmentation analysis universally. This type of system inherently provides better results sooner while delivering in-depth insights into potential personalization points. The most basic targeting starts with environmental variables—think browsers, operating systems, and general temporal information. This is followed by geographic data, real-time behavioral targeting, and even the ability to integrate offline or internal customer relationship management (CRM) data.

So how do you weigh these considerations and choose the optimal testing solution? Think about what’s involved in integrating potential data sets into your existing marketing systems. What do you need to support and maintain the integration? What will you actually tap into, and what will your internal and external stakeholders need to see to deem the testing and subsequent optimization a success?

Evaluate the vendors through these lenses and don’t get sidetracked by all of the “nice to haves” if your “need to haves” will get the job done. Then, of course, be sure you have the tools and the resources—AKA the marketing manpower—to report, segment, analyze, and effectively utilize the results.

Think Collaboratively

It’s simple: changing your site for the purpose of testing and optimization can be a daunting task for marketers and stakeholders alike. It’s not only the change involved, Hawkins writes, but also the seemingly foreign concepts such as HTML, JavaScript, style sheets, and Document Object Model manipulation, that can overwhelm and off-road even the most dedicated team.

To avoid these pitfalls, make the workflow collaborative, clean, and as seamless as possible from the beginning. By selecting the right tools and the right third-party vendor, you’ll have a comprehensive roadmap in place that makes the framework and objectives clear to everyone involved. This collaboration must carry through to the implementation stages of the process. Evaluation is critical, but implementation “can make or break an organization’s use of a testing solution.” Make sure those responsible for the execution and future integration are involved from the earliest stages, and incorporate their feedback in the assessment and planning processes. What it costs in startup time, it will more than make up for postpurchase.

The Checklist

Hawkins concludes with his thorough yet user-friendly requirements checklist. The process, he writes, starts with the internal audit and review period—determining what’s important, getting stakeholders on board, and identifying an implementation path—and then assessing solution providers. His sample checklist splits the review into three buckets:

  • Importance: What are the must-haves versus the nice-to-haves?
  • Supported by vendor: What does the potential partner offer, and what’s core, optional, and simply not available?
  • Implementation effort: On a high/medium/low scale, how much effort will go into the implementation process?

From here, marketers can make their way through a prepopulated list of tests, targeting capabilities, analysis capabilities, segmentation, automation options, considerations, and more, assessing each subcategory with regard to their requirements.

For testing novices, it’s critical to identify a tool that not only checks these boxes but also keeps an eye on setup and executional best practices and, moreover, provides ample support throughout. Hawkins calls out eight essential capabilities for those new to the process:

  • Workflow and collaboration best practices enablement
  • Range of test type capabilities
  • Comprehensive reporting interface
  • Segmentation
  • Targeting
  • Real-time reporting
  • Rich and robust targeting capabilities
  • Customizable graphical analysis

But not only new and relatively new organizations need to take a step back and evaluate their position. All organizations should include the following in their assessments:

  • Integration capabilities, especially with the analytics platform
  • Automation of test content
  • Application programming interfaces (APIs)
  • Mutual exclusivity
  • Ability to execute tests across channels

No matter the experience, comfort level, or objectives, though, you also need to consider whether yours is an optimization organization. The success of a testing and optimization initiative can only be as successful as the organization’s maturity, understanding, and available support. Where your organization stands during the planning and evaluation stages will dictate its success—don’t wait to “catch up” once the process begins.

Overall, Hawkins’ piece gives a comprehensive, digestible look at a complex process. Brands, agencies, schools, and content providers must deliver the most relevant, most targeted information, recommendations, and solutions to their users, making online target marketing critical to virtually any organization. No matter how you measure ROI—through revenue, engagement, lead generation, added content consumption, or loyalty—the proper testing and optimization solutions can deliver in spades.