better software | Ruud Teunissen

Digging through my personal archive, I found “Is There an Assessment in the House? Diagnosing Test Process Ailments in House”, an article Better Software published in December 2006. Re-reading the article, I decided it contained tips about performing an in-house assessment that are still very true.

Performing an assessment in house is a less expensive initiative than hiring outside consultants. Maybe not as inexpensive as the Low Budget approach I described in my previous post, but still. And it has other substantial benefits. An in-house assessment team knows the organization, its people, its processes, and its habits in substantially more detail than an “outsider” could learn during the relatively short assessment period. And the process improvement suggestions might be more realistic, better tuned to the situation, and more achievable.

Here are some – slightly updated – tips for leading a successful in house assessment:

Define why you are performing the assessment and what you hope to accomplish.
Choose a test process assessment model that suits you and your organization. Use it to make sure your assessment is objective and your conclusions are legitimate.
Create questionnaires to ensure that you cover all major topics when talking to people.
Familiarize yourself with all parts of the organization and how it actually tests software. Look for discrepancies between process information – the way they claim to do things – and project information – what they really do.
Make sure you choose a representative sample of people to talk to. It’s easier to sit down with friends, coworkers, and people nearby, but they may not be the best sources of information.
Don’t just talk to testers. Testing is part of software development, decisions made by any discipline significantly impact testers. So be sure to include people from these areas.
If you organize a group interview, make sure they are peers. Otherwise, you’re bound to get “right” answer – what the manager wants said – instead of the truth.
Arrange to meet in their offices. People feel more comfortable on their home turf and may be more open to your questions.
Be an effective listener. You may be more of an expert in an area than the person you are talking to, but don’t jump to conclusions. Don’t fill in the blanks or use your words rather than theirs. It’s crucial not to interpret – just listen, learn, and record.
Ask what problems exist in the software testing process and how they think those problems could be solved. You will discover a wealth of information.
Keep confidences. You may hear some interesting things during your interviews. Don’t attach names to negative comments. Remember, you are trying to gather information, not set people up for punishment later.
Take detailed notes even if you’re getting the answers you expected. You won’t be able to remember all the details later, and your notes will be vital.
Don’t become enamored with certain people. They may be handsome or beautiful or powerful or famous or charismatic, but that doesn’t mean they know what they’re talking about.
Work with a partner. While you are asking the questions, he records the answers. After a while, switch roles. I’ve always found my partner notices things I miss and asks questions I tend to forget.
Stop regularly to evaluate what you’ve learned and what is still unknown. Use that knowledge to guide your next interviews.
Present your findings to all who contributed to the assessment.
Tell the truth. As testers, the only lasting power we have is our integrity. Don’t do anything to damage yours.

If you ensure that you’re not biased and you make everyone share their weak spots, you’ll achieve wonderful things through an in-house assessment.