Standard Testing Methodology — Intro
Strategizing or Executioning
Did you know that when someone is executed in the electric chair, the first jolt of electric current is designed to cause immediate unconsciousness and brain death? This is interesting to me because the human body is composed of electric currents and energy. I’ll spare you the 7th grade proton, neutron, electron story, but suffice it to say that too much electricity and we die, no electricity and our bodies just wouldn’t work.
I have seen this same phenomenon in organizations that are testing. Often companies will have too much or not enough of something important. The end result is usually the same as too much or too little electricity in our bodies – the testing program stops working or suffers strategic brain death. As an example how testing electrocution can occur, I’ve seen companies end up executing themselves to death by running lots of tests. They execute so many tests that they don’t take the time before or after a test to strategize. They literally become the executioner and the victim all at once.
When any one aspect of testing is over or de-emphasized the testing program can be put at risk. Clients often get hung up with test execution, but this same thing can happen when other important testing steps have too much or too little emphasis. Another example is a client who was known to glorify results evaluation and had whole teams dedicated to interpreting test results. They would run a test and spend a lot of time and resources pouring over analytics data, backend data, and putting together pages and pages of metrics. In this case too, the whole program suffered. The ironic thing about this is the same company couldn’t tell me one thing they learned in their whole previous year of testing—they had lots of results and numbers to interpret those results but that is it. Fortunately this client has made the shift to give equal importance to the other important steps and their testing program is looking much more promising.
You might be asking yourself what those steps are. What process can every company follow to make sure they are remembering the right things at the right time? Let me introduce a simple testing methodology that outlines 6 repeatable steps that every testing program could follow to make their tests as strategic and successful as possible.
1. Question Creation
2. Question Definition
3. Question Evaluation
4. Test Execution
5. Results Evaluation
Mature and successful testing companies know that testing takes a lot of work and a lot of discipline for continued success. The discipline is adhering to a methodology that helps move testing forward and keeps the company doing the right amount of the right thing at the right time. My goal is to help you see that there is more to testing than any one aspect—including running tests.
I hope that these steps will help you focus your efforts and will bring the strategic nature of testing to the foreground. To that end, I will be exploring each of these steps in detail and what it means to have enough of each step to empower us, but help us avoid overdoing it and in turn keep us from strategic brain death or non-functioning testing programs.