For me personally, one of the most difficult things in life is choosing. Especially because I don’t know what would have happened if I had chosen differently… And in the back of my mind, that “what if” game is constantly played.
So my expert advice in test improvement is to choose wisely and consider several (all?) options. Don’t make choosing an everlasting process, just make sure you choose what helps. To be able to choose, knowledge about and experience with several approaches and models is necessary. Try to find a way that enables you to compare them and find the combination that suits the context and the purpose.
What usually helps is looking at aspects like:
- Testing only?
Is the scope of the model testing-only or does it consider a broader area?
Is training available to ensure that everyone involved has the opportunity to learn about it.
- Staged or continuous?
Is the model prescribing a standard sequence of improvements in (big) steps or does it give free choice in the sequence of improvements in (small) steps.
- SPI related?
Is the approach or model linked to a Software (Process) Improvement model or approach that is already used within the organization or linked to a SPI approach that would help to improve more than just testing.
Can you apply it to help a single project? A team? A department? A complete organization?
- Are improvement suggestions “included”?
Some models and approaches focus on finding the current status of testing and the required status; several include concrete improvement actions and highlight enablers
- Required budget / investment?
How much time/effort is required for preparation, execution and implementation of the implementation. What is the claim on the organization? Especially since you take into account that Business as Usual will always need to continu. And that the required budget needs to be available; in other words, your approach has to be fit for budget as well.
- Top down? Bottom up? A mix?
- Alignment with current test and development approaches, e.g. waterfall, iterative, incremental, Agile, Scrum, DevOps.
Align your improvement approach with the culture of the organization and its peopke; although this is not typically something that is defined in a SMART way, this is key to success.
Do the people in the organization like the improvement approach and are they willing to change in line with it. Again not typically something that can be defined in a SMART manner…
When I choose based on these – or similar – aspects, I tend to organize a walkthrough of my approach with experienced fellow testers. For me, it softens the “what if” voice in the back of my head.
Thank you Cyanide & Happiness!
The models and approaches to choose from are plentiful, probably worth (yet) another blog post.
Digging through my personal archive, I found “Is There an Assessment in the House? Diagnosing Test Process Ailments in House”, an article Better Software published in December 2006. Re-reading the article, I decided it contained tips about performing an in-house assessment that are still very true.
Performing an assessment in house is a less expensive initiative than hiring outside consultants. Maybe not as inexpensive as the Low Budget approach I described in my previous post, but still. And it has other substantial benefits. An in-house assessment team knows the organization, its people, its processes, and its habits in substantially more detail than an “outsider” could learn during the relatively short assessment period. And the process improvement suggestions might be more realistic, better tuned to the situation, and more achievable.
Here are some – slightly updated – tips for leading a successful in house assessment:
- Define why you are performing the assessment and what you hope to accomplish.
- Choose a test process assessment model that suits you and your organization. Use it to make sure your assessment is objective and your conclusions are legitimate.
- Create questionnaires to ensure that you cover all major topics when talking to people.
- Familiarize yourself with all parts of the organization and how it actually tests software. Look for discrepancies between process information – the way they claim to do things – and project information – what they really do.
- Make sure you choose a representative sample of people to talk to. It’s easier to sit down with friends, coworkers, and people nearby, but they may not be the best sources of information.
- Don’t just talk to testers. Testing is part of software development, decisions made by any discipline significantly impact testers. So be sure to include people from these areas.
- If you organize a group interview, make sure they are peers. Otherwise, you’re bound to get “right” answer – what the manager wants said – instead of the truth.
- Arrange to meet in their offices. People feel more comfortable on their home turf and may be more open to your questions.
- Be an effective listener. You may be more of an expert in an area than the person you are talking to, but don’t jump to conclusions. Don’t fill in the blanks or use your words rather than theirs. It’s crucial not to interpret – just listen, learn, and record.
- Ask what problems exist in the software testing process and how they think those problems could be solved. You will discover a wealth of information.
- Keep confidences. You may hear some interesting things during your interviews. Don’t attach names to negative comments. Remember, you are trying to gather information, not set people up for punishment later.
- Take detailed notes even if you’re getting the answers you expected. You won’t be able to remember all the details later, and your notes will be vital.
- Don’t become enamored with certain people. They may be handsome or beautiful or powerful or famous or charismatic, but that doesn’t mean they know what they’re talking about.
- Work with a partner. While you are asking the questions, he records the answers. After a while, switch roles. I’ve always found my partner notices things I miss and asks questions I tend to forget.
- Stop regularly to evaluate what you’ve learned and what is still unknown. Use that knowledge to guide your next interviews.
- Present your findings to all who contributed to the assessment.
- Tell the truth. As testers, the only lasting power we have is our integrity. Don’t do anything to damage yours.
If you ensure that you’re not biased and you make everyone share their weak spots, you’ll achieve wonderful things through an in-house assessment.