Last Friday I submitted my presentation The Awful Truth About Test Estimation – Have I been wrong all along? to EuroSTAR. Turns out there is a lot to say, discuss, consider, think, explore,… about estimation and that 45 minutes, including loads of interaction, is pretty short.
According to wikipedia, “Estimation is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. The value is nonetheless usable because it is derived from the best information available.” To make life a little easier, wikipedia explains that “An approximation is anything that is similar but not exactly equal to something else. The term can be applied to various properties (e.g. value, quantity, image, description) that are nearly but not exactly correct; similar, but not exactly the same.”
So in fact, test effort estimation is a the process of defining a number – hours, man days, lapse time, … – that is nearly but not exactly correct AND is based on incomplete, uncertain and unstable information.
For me that is awful yet true.
Should we stop doing it?
Unfortunately, for most of us that is not an option. We need to provide some insight into the necessary effort for testing, because (project)management needs it to establish a budget request, (line)management wants insight into the resources needed for a project, et cetera.
We should however always treat it as an estimate: nearly but not exactly correct, based on incomplete, uncertain and unstable information. And share the uncertainties, instabilities and incompleteness!
You should never stop doing it!
One of the most important lessons to learn about estimation is that you’re never done with it. Each day one of the uncertainties can become a certainty, decisions are made that remove instabilities, etcetera. As a tester you should continuously look at your estimates and adjust them to the new situation.
Biggest challenge: you need to convince (project)management that an estimate is an incomplete, uncertain and unstable number. And that fixed deadlines sometimes just mean that not all of the testing work is done… You could compromise but my suggestion is to do the best you can within the given restraints. And be open and honest about it. One of the best results I got was in a project where the project manager decided to stop detailing the requirements and start coding immediately to save time and effort on the development side (approximately 200 man hours). I did not complain or blame. I just told him that I expected that as a consequence the test effort would go up by at least 300 man hours due to the fact that we needed to find out about the requirements ourselves now and we expected more defects… After a prolonged discussion, he withdrew his decision.
See you in Gothenburg?
If you are coming to Gothenburg and are interested in attending my talk, please register here
You can also do so by visiting this page on the EuroSTAR website: “W9 – The Awful Truth About Test Estimation“