In my last article, we examined what accuracy in digital marketing is and why it is important for a successful marketing optimization program. We also looked at how the quality of your data and analysis can affect the accuracy of your results. In this post we’ll examine some of the safeguards that can be put in place to ensure the accuracy of your tests, and look at ways to convey the information achieved from your targeting and testing.
Anomaly or Extreme Order Filtering
Few tools on the market today leverage the benefits of anomaly or extreme order filtering in terms of the accuracy of results. Anomaly or extreme order filtering entails the ability to automatically identify and remove outlier results that can severely skew result accuracy. This is a critical feature, particularly in the retail space, where extreme orders can significantly impact the content that you might assume is winning, and even skew the potential conversion or revenue lift in results to a huge degree.
Tools like Adobe Target use built-in filters that automatically identify and eliminate data outliers that may skew your results and sabotage your efforts. Eliminating these anomalies affords a more efficient and accurate view of your test results, allowing you to further pinpoint key customer segments and the content that most resonates with them.
As a basic rule, it is a good idea to remove outliers that are more than two standard deviations from the mean, but these thresholds can also be custom set based on your business requirements. It is important to understand the effect that removing certain data points can have on your results.
There is a fine line between removing outliers from your data and deleting legitimate test results, and it is important to understand where that line exists. This type of filtering can be difficult to accomplish in software packages without this built-in capability.
Another important tactic, mutual exclusivity, lets you easily pick and choose which customers to include or exclude from specific tests or programs. People who participate in multiple tests may be influenced by the content of one, and it is important to be able to identify the effect this may have on your test results. In addition to excluding participants from your tests, you can compare their responses to others in order to incorporate the information gained from their activities without skewing the entire program. You can also run a test within a test scenario, which shows further correlations between your hypotheses and your results. This allows you to better understand how one test might affect another, and how customers might respond to a series of marketing campaigns run in tandem.
Targeting at the campaign, location, and individual experience level lets you clearly define the rules for including or excluding particular visitors within a test. Campaign-level targeting lets you target the entire test to a specific group based on available segments or variables. This means that people who meet certain criteria will only be included within a particular campaign or series of tests. Location-level targeting lets you show content in a particular location only when the visitor meets certain real-time conditions. It lets you limit when certain offers are displayed based on the criteria dictated at that location. Experience, or offer-level, targeting lets you dictate particular content to particular visitor segments within the same test, and is immensely useful in landing page campaigns. This level of fine-tuning and specification in the test design and set-up process contributes to the richness and accuracy of results, even within a single test.
The ability to manipulate or filter results based on different groups or variables within your test population is also a huge factor in the accuracy of your analyses. With solutions like Adobe Target, you can easily view the results of your testing based on different control groups or timeframes, or within a visual graphic context. You can also set and change your control group. The control is what you would define as your baseline for the test, or the expected result from your efforts. By changing who the control group is in your testing, you can look at your results in a different way based on the expected behavior of the control group. Making these dials accessible and easy to adjust allows for more efficiency and confidence in your findings and the ability to share them with key stakeholders faster.
And, of course, a final, and often overlooked, piece of ensuring accuracy within your program as a whole is your ability to convey your results accurately to the stakeholders. You need robust custom reporting functionalities that allow you to quickly generate reports that provide a comprehensive view of your tests and their results. Good data and analysis is worthless if it is not delivered in an understandable way, and many software packages lack the ability to effectively create summaries of the tests, instead opting for unwieldy CSV files to convey results. Custom lists and reports and robust data visualization increase the flexibility with which you can accurately report your results.
So what do you think? How does your organization ensure accurate results from your testing efforts? What other factors play into accuracy, and how do you account for them in your testing?