The Need for Speed
One of the largest differentiators between starting, middling, and mature successful programs is the speed at which they can execute. The more you move, and the more efficiently that you move, the more value you get and the more you learn. You can’t get anything from running 100 bad tests, but if you have even a little discipline in your test design, then the faster you move, and the faster you can act on that data, then the more value you will receive and the greater your chance to learn and grow. If it takes you 2 months to just answer a simple business question, how can you expect to keep up with the industry? Speed is the differentiator in the magnitudes of value you receive once you have acted on and enabled the other disciplines of optimization. You have to move fast to insure that you are getting the most from your program.
Dealing with hundreds of different testing groups, you see a very different approach to what is acceptable as far as the speed of getting a test live, from concept to execution. Some groups, it can take months to get a test live, while others are able to execute in mere hours or minutes. No matter what you think is normal, the reality is that every group needs to work every day to get faster and more efficient with their program. The sad truth is that most groups treat acting fast as a failure of the system, letting the weight of their own size work against them to the point that it feels like you are sysyphus whenever you try to get different teams to work together and an expedited manner. If you want to really get value from your program, you have to create the infrastructure before you try to run, in order to maximize speed and make sure that you are executing at a high enough level for success.
One of the most common things I talk about with clients is that nothing will show your organizational inefficiencies more than trying to act quickly. Whether it is an out of touch IT group, a lack of training, a need to get approval for every change, a slow unresponsive design team, or a thousand other barriers, every group has a large number of roadblocks that stop them from moving quickly. But this doesn’t mean that you can just accept these barriers as being status quo; you have to maximize each effort and make sure that you conversations and energies are spent on improving the infrastructure of your testing program, and not just on running any one test.
There are many different ways to organize in order to succeed, whether it is a L3PS model, a hub and spoke model, or many others, each group needs to figure out what works best for them. It is the act of finding and creating these efficiencies that make programs great, not that great programs have these items available to them. Great programs have common items in place and aligned in order to facilitate speed and to make sure that barriers and eventual road bumps are dealt with as quickly as possible:
1) Sponsorship – All programs that really achieve levels of success have someone higher up who is taking an active role in the program and making sure that things are communicated properly in order to facilitate the various groups working together. It is nearly impossible to really grow your program without someone greasing the gears of your organization.
2) Consistent resources – Testing can’t be a project type of basis, it has to be a core team who are learning and growing their skills in order to become better at what they do. There are many different ways to allocate these resources, but successful teams start with dedicated time and usually move to large teams of people who do nothing but work on optimization efforts.
3) Technical infrastructure – You need to make sure that your site can properly track your single success metric, and that you can get a large majority of tests live with no technical involvement. This means that you need both global pieces of infrastructure, but also properly set up the pieces needed to tackle the key pages on your site.
4) Knowledge Transfer / Knowledge repository – There is a large amount of education needed in order to really move forward with your program. The programs that have the most trouble are the ones that try to run optimization like they do analytics, or visa-versa. You need to train people on how best to think about testing, how best to think about data, and how best to answer their key questions. At the same time, you need some sort of living knowledge base that accumulates your knowledge and is available for those that want to become more involved. Make sure that you are separating the lessons learned from the individual test results.
5) Program built around efficiency – So often where programs falter and slowdown is that they try to only test massive tests that are built to do the one idea that someone higher up wants to try. While you shouldn’t shun these tests all together, you will find that almost always those tests do not get the results that you expect and are not built for learning. Instead test with the resources you have and try to maximize the amount of learning and recipes that you are getting from each test. The goal is not to run as many tests as possible, as 50 awful tests will not be as valuable as one well run test, but instead focus on maximizing returns and keeping a steady cadence of testing to make sure you are always moving forward.
There are many other pitfalls that can hurt the infrastructure of a program, from overdone QA processes, too many sign off points, blocking or lack of involvement from senior leadership, unwillingness to listen to numbers or feedback from your UX or branding teams, to a thousand others. The real key is that you cannot ignore them, nor can you think they get solved with some magic elixir. It takes time to move your program forward, and there are always potholes and road bumps on the way. All programs go through an evolution, but at the end of the day, it is your willingness to move forward and try to overcome those barriers that will determine your success.
You have to want to move fast in order to do the things necessary to accomplish it. You have to have the need for speed if you ever want to achieve it. It is doable, and there are lots of resources out there to help, but in the end, it all starts with your willingness to act.