Test Improvement is choosing

For me personally, one of the most difficult things in life is choosing. Especially because I don’t know what would have happened if I had chosen differently… And in the back of my mind, that “what if” game is constantly played. 

keep-calm-and-choose-wisely-33So my expert advice in test improvement is to choose wisely and consider several (all?) options. Don’t make choosing an everlasting process, just make sure you choose what helps. To be able to choose, knowledge about and experience with several approaches and models is necessary. Try to find a way that enables you to compare them and find the combination that suits the context and the purpose.

What usually helps is looking at aspects like:

  • Testing only?
    Is the scope of the model testing-only or does it consider a broader area?
  • Training?
    Is training available to ensure that everyone involved has the opportunity to learn about it.
  • Staged or continuous?
    Is the model prescribing a standard sequence of improvements in (big) steps or does it give free choice in the sequence of improvements in (small) steps.
  • SPI related?
    Is the approach or model linked to a Software (Process) Improvement model or approach that is already used within the organization or linked to a SPI approach that would help to improve more than just testing.
  • Scalability?
    Can you apply it to help a single project? A team? A department? A complete organization?
  • Are improvement suggestions “included”?
    Some models and approaches focus on finding the current status of testing and the required status; several include concrete improvement actions and highlight enablers
  • Required budget / investment?
    How much time/effort is required for preparation, execution and implementation of the implementation. What is the claim on the organization? Especially since you take into account that Business as Usual will always need to continu. And that the required budget needs to be available; in other words, your approach has to be fit for budget as well.
  • Top down? Bottom up? A mix?
  • Alignment with current test and development approaches, e.g. waterfall, iterative, incremental, Agile, Scrum, DevOps.
  • Culture
    Align your improvement approach with the culture of the organization and its peopke; although this is not typically something that is defined in a SMART way, this is key to success.
  • People
    Do the people in the organization like the improvement approach and are they willing to change in line with it. Again not typically something that can be defined in a SMART manner…

When I choose based on these – or similar – aspects, I tend to organize a walkthrough of my approach with experienced fellow testers. For me, it softens the “what if” voice in the back of my head.

Peer review gone wrong

Thank you Cyanide & Happiness!

The models and approaches to choose from are plentiful, probably worth (yet) another blog post.

Advertisements

Back to test improvement – for now

Last week I organized a meeting with several colleagues who have, just like me, several years of experience in the improvement field. I’ll try to cover the topics we discussed in a number of posts, this is the first. The meeting turned into a great discussion about test improvement and we discovered – despite working in the same industry – so many different stories to tell. Any situation is different, isn’t it?

One tool to rule

Which underlines the most important lesson to learn in successful improvement: there is no such thing as “one tool to rule them all”, sometimes not even within one organization or department.

This is the what differentiates a manager from a leader: a manager sticks to his model of choice and tries to fit the context in his model; a leader uses what is “fit for context and purpose”. And in many cases, that’s more than one model or no model at all.

So, if you really want to improve, one of the things you have to keep in mind is that you need to focus on improving and not on the model you want to use.

The characteristics of each test improvement model define its added value and its limitations. And that is why the foundation for succes needs to be laid at the start of any improvement initiative: what model(s), if any, will you use?

To be able to decide, you’re first step needs to be defining the objectives of your initiative:

  • Reduce time
  • Save cost
  • Improve the quality of the delivered product
  • Improve the quality of the test process
  • Improve the skills & knowledge of the team
  • ……

On target

Scope comes second:

  • what area will you consider
  • what is the size of the group to consider
  • is only testing considered or do you include other disciplines like development and operations as well

Third is taking into account the resources that are available for the initiative:

  • Budget
  • Time
  • Skills
  • ……

Culture of the organization is fourth: are they open to change, eager to improve. Context is fifth: what approaches and tooling are already being used. And sixth is your best guess of the current maturity: the more mature an organization is, the easier it is to implement improvements and ensure that current “good” practices remain in place.

In other words, current & future test improvement has to be:

Current and future TI

The next step is to come up with an approach that suits all of the above. Worth another post, I guess.

Why improvement usually fails…

If you want to improve you have to invest time and money. What usually happens, is that you look for what everyone at that moment considers to be the right thing to do. So you tend to adopt the test approach, including tools and templates, that is “hot”.

Head in HandsLet’s say, you decide to implement risk based testing. You learn how to translate business risks into an appropriate test approach, including techniques, and create a beautiful plan and strategy.  And then? Your (pilot) project starts and fails…
Why, you wonder. You applied everything you learned, went to conferences, listened to the gurus, read the books, hired expensive consultants, …

As a test consultant I had the privilege to work in different organisations and projects and discovered a trend in “failures” and “successes”.  It boils down to one common denominator: did you or didn’t you involve the people who do the “real thing” in the test process? Yes, the ones that do the actual work a.k.a. the testers.  Somehow, they do not get involved in the early stages of test improvement.

We tend to start with test management, make the test organisation more professional, develop a test strategy, follow a lifecycle (plan, prepare and execute), use the right tools, select adequate test design techniques … According to the book! Don’t we make the same mistake at the start of a project? Forget to involve those same testers in the early stages?  When we define a test strategy and write a test plan, we talk to the business, the developers, all of our stakeholders, …

Don’t forget: the testers are the most important part of your test process and improvements, they have to learn to apply the new methods, tools and templates. In fact, they are the ones that have to change the most!

27-teamworkI found out the hard way that a few days training – even from the guys that wrote the book – is not enough for that senior business tester that has been there for over 10 years and knows the system inside out… And what about the “fire fighters” that have become heroes because they find, fix and retest defects quickly? They will not be convinced, they won ’t even attend!

So it’s all about people and how you involve them.  Take into account the capabilities and potential of your test engineers. Listen to what they have to say.  On the short term, you might end up at a maturity level you do not like. Or your approach is not exactly what you want it to be. But you know what, if you help them manage the transition, they will reward you with the highest level possible in the shortest possible time.  You will become truly successful.

It’s not your test management approach that decides how successful you are, your test engineers make it or break it. TMap? ISTQB? RST? As long as your people are able and capable to apply it in their day to day work.

Triggers for Improvement? Opposites attract!

The result of the poll I held amongst fellow testers and on this blog, is in line with my expectations: most test improvement initiatives start when quality issues occur. So in that sense, management treats testing like any other discipline: why invest in change or improvements when there is no (obvious) reason for it.

In my experience, motivated testers continuously look for ways to change and improve themselves and the way they work. And they are not afraid to look for help. They’re active on the internet, participate in Special Interest Groups, and meet with each other quite frequently. That’s why most improvement initiatives start at the operational level: the testers want to improve.

The first hurdle is to convince management that it is worthwhile to invest in change and improvement, a.k.a. business case. Unfortunately, currently the focus is on numbers: what are the benefits (in Euro’s please)? what are the costs? And by the way, we want Return on Investment within 6 months. Again, management treats testing like anything else: most companies  only start a project if it is beneficial within 6 months.

Quality versus Time versus Costs
Testers focus on qualitative benefits:
– the right test at the right moment;
– deliver what was promised;
– the right coverage at the right spot.

Management on quantitative benefits:
– reduced time-to-market;
– deliver what is necessary;
– increased productivity.


Funny thing is that also in this situation opposites attract: management will never get what they want without achieving what testers strive for; testers will never be able to reach their goals without meeting management needs. I’ll even go one step further: by helping management to achieve its goals, testers will achieve their own goals as well. Without the right test with the right coverage at the right moment at the right spot, reduced time-to-market and increased productivity is impossible. The only thing we can and have to discuss is scope: deliver what is promised or deliver what is necessary?

What would trigger you to perform a Test Process Improvement assessment?

Triggered by a discussion with some colleagues, I decided to add a poll to this blog.

 

 

Hope you share your opinion with the rest of us!

Assessments by external consultants have major benefits too!

Although I personally believe that doing it yourself is one of the best ways to successfully improve, in-house assessments have severe drawbacks as well.

SEPNumber one: you are part of the team that is being assessed and are therefore biased. No matter how rational and objective you try to be. During one of my assignments, I came across an effect called “Somebody Else’s Problem (SEP)”. I think this describes best what the consequences of being biased can be. Maybe we even use a so called SEP-field generator to help overlook issues within our own approach and team.

Number two: your coworkers are hesitant to share their weak spots with you because they fear you might use anything they say against them in the next project. As a consequence you will not find the root cause of recurring issues or the real issues preventing things to improve.

Number three: experienced consultants have performed assessments many times before, understand the assessment models, and are efficient in surveying organizations. Because of their independence, they can dare to ask questions about “everything you’ve always wanted to know but were afraid to ask.”

Number four: external consultants bring a wealth of experience form other organizations. They have helped other organizations improve their testing and usually have access to several good practices that might help you make that next step. I’ve also learned that as an external consultant it’s easier to point out that certain existing habits should change.

Number five: when you’re leading an improvement initiative yourself, you might overdo it and strive to become the best in test. But your companies focus is on delivering the right product and/or service to your customers. Which implies that being efficient in testing might be enough. External consultants can help you decide what level is enough based on their experience at similar organizations.

Number six: even if you’re a craftsman, well-trained, experienced and knowledgeable, it is always good to have someone else come in and discuss with you what you’re doing and how you’re doing it.

CloutNumber seven: your findings and recommendations, no matter how well reasoned and well-presented they are, do not have the “clout” that an outside assessor brings. This is a strange phenomenon that I’ve encountered quite often: an in-house group has performed an assessment and has made recommendations to management that are simply ignored. Later, when we are asked as external consultant to perform an assessment, we make in essence the same recommendations, management responds positively to our report, and the recommendations are implemented.

The hints, tips and tricks I listed in my previous blog also apply to anyone performing an assessment. One additional, crucial tip when using an external assessment team. Make sure that you do not focus on knowledge of and experience with process improvement models, but please check if the assessment team incorporates a wealth of experience in testing at any level.

Test Process Improvement for a Low Budget

At the start of the current economic situation (some would call it an economic crisis), we were asked to create and present a workshop on test process improvement “on a shoestring” a.k.a. for a low-budget. Companies nowadays do not have the time and the means for extensive improvement initiatives. However, they are in dire need of improvements that help them in achieving cost reduction, shortening time-to-market and achieve a higher quality level. In other words: do more with fewer resources, in a shorter period of time and with higher quality, without (almost) any investment. Challenging, yet not impossible.

Do what is necessary When you have been working in an organization for a while, several actions and activities turn into routine: you perform them without really thinking about. For the majority of actions, that’s fine because they are necessary steps you routinely perform to get where you need to be. For some actions however, you need to fight the routine and perform them consciously every time. Because sometimes they’re not necessary, over the top or counterproductive. Most of us know that, yet do not take the time to discuss and implement changes to the way of working. Because “no one challenges the Way We Work” and this is, by the way, part of human nature. We do comment on the way we work and criticize things we want to be different, but don’t always act.

Another part of human nature however is that we knowingly and unknowingly collect ideas for improvement and ways to solve recurring issues. They just don’t always take the time or have the platform to turn those ideas into actions and go further than fighting the symptoms. By just doing that: creating a platform for collecting ideas to turn them into actions and for solving the root cause of the issues, you can improve your testing.

I’ve learned that an interactive workshop to gather improvement suggestions and select the most suitable measures with all involved parties, enables an organization to get the best out of their test team and create support for change and effectively improve the test process. All involved parties are needed, not just testers or test managers. So invite anyone who is working together with testing, from business to operations, from project management to developers. Crucial advantage: commitment is guaranteed by involving all parties.

Stop nagging

Gather ideas in small multidisciplinary teams. Ground rule: any idea or suggestion is welcome. Bad ideas don’t exist, even the suggestion to stop testing is fine! So, anything is OK.

Next step is to collectively cluster the ideas and set priorities: When do improvement ideas have effect? And what is the impact? What are the related costs (and benefits)? Last but not least, the easier it is to implement the measures the more successful it will be. The search is for “silver bullets”: Free measures that have immediate effect, high impact and are easy to implement. The workshop wraps up with an action list where each selected measure directly leads to concrete actions on the list.

Test Improvement for a low-budget does help, but is limited. Renowned test improvement models like TI4Automation®, TI4Agile®, TPI® Next and TMMi® as well as extensive experience in test improvement projects, still is needed to achieve improvements beyond picking the “low hanging fruit”. And by the way, any improvement initiative can only be successful if the people who need to do it are not only involved but are heard…