Starting something new

After 14 years, 4 months and 23 days, it’s time to say goodbye to Polteq and move on. On January 1^st, I joined ASML where I will work on the most innovative products in our industry. At ASML I will contribute more directly than before to the development and improvement of technologically high-quality products. I am given the task, together with a number of colleagues, to raise the quality of development and testing within ASML. For me the chance to discover a new world with all the experience that I have been able to gain during my career.

Polteq has played a crucial role in my life since 2004: challenging projects, developing and providing training, speaking at conferences around the world as well as various roles within Polteq itself. This enabled me to develop myself and discover what my strengths (and weaknesses) are. To me, it will always be very special that I was given the opportunity to work closely with Martin Pol and Marjolein Steyerberg – the founders of Polteq – and thereby was able to contribute directly to Polteq. I truly enjoyed that and I am incredibly proud of that.

Why leave? Because the time has come – after being a contractor for 29 years and 2 months – to join a different type of organisation full of new challenges.

Agile or not?

One of my first projects when I returned to the testing ranks, was an Agile/SCRUM multi-project program. Goal was to implement a future proof solution that supports continuity and enables continuous innovation of customer services (“any time, any place, anywhere”). My assignment: align the company’s generic test process with the project approach and manage the testing within the program accordingly. Major challenges: the generic software (test) process is not tuned to Agile/SCRUM (yet) and the (lack of) Agility of the project.

A generic software (test) process gives you a head start when moving towards Agile

The organization that hired me, put a lot of effort in improving the software development process, including testing. Supported by several external consultancy firms, they developed and implemented a broad set of tools, templates, guidelines and training for their people. The generic process covers every discipline: from project management to operations, from design to development, from architecture to testing. It was great to learn that testing was one of the driving forces behind this initiative. Therefore, it’s not a surprise to see that the organization highly appreciates testing. Turned out that they created a great starting point for becoming more Agile.

Adapt instead of abandon

Although several teams applied Agile/SCRUM, the generic software development approach was not aligned with that yet. Despite the willingness to become more Agile, some people were hesitant to move in that direction. Turned out that fear of losing what they achieved was the reason for that. By tuning the generic process to Agile/SCRUM instead of abandoning it, our project showed that the essential parts of the software development process remained intact and valuable. No reason to abandon (test) design or tooling. And a well-oiled DSAP-approach for test environments is great in an Agile/SCRUM project as well! Even some of the mandatory reporting turned out to be effective in our Agile/SCRUM project as well. To me, the biweekly reports about the status of the projects forced me to get in the helicopter and get a bird’s eye view instead of looking at the project alone.  Simply by requiring a substantiation (facts) of the status you report, asking you to look beyond the borders of the sprint (product) and tracking completeness of the deliverables, including testware. It enabled me to provide helpful input for the retrospectives and ensured that my actions were more based on facts and less on emotions.

Contract negotiation over supplier collaboration?

It was the program manager’s decision to use Agile/SCRUM. And for the right reasons. The project start architecture formed a perfect start for the project. It created clear, coherent themes for the development teams and ensured they had the necessary freedom required to start designing, developing, testing and delivering in sprints.

It was the program manager’s decision to hire suppliers based on a fixed price fixed date contract. Alas, for the worst of all reasons: that’s how we do things here… For example, one of the SCRUM-masters created a high-level draft schedule for the first 8 sprints, based on the information available. Unfortunately, it was used against him and his team: the program manager did not allow changes to this schedule. Alas, again for the worst of all reasons: the project manager did not want to write an exception report and since he has communicated the schedule to the steering committee, he doesn’t want to tell them we’re deviating from it…

Collaboration over contract negotiation!

Turning point was a program retrospective. The program manager organized it, the SCRUM-masters and the test manager (me) initiated it and we made sure that, next to a part of the project team, several members of the steering committee participated in this meeting. In their opinion, a major improvement would be more collaboration. They wondered why, in an Agile/SCRUM setting, this was clearly lacking…
The new contract form chosen was simple: time and material, with a twist of commitment (make it beneficial for all parties involved to deliver high quality products). True collaboration and the ability to respond to change, were the logical consequence. And it enabled the teams to focus on what is important: delivering working software and increase velocity.

Test automation played a major role

At those moments where we were disappointed in the way we were (not) moving on in the project, the one thing that motivated me and my test colleagues, was test automation. Together, we took the time to update and extend the existing test automation solution to cover the new functionality. Worth another blogpost.

Being a tester (again)

It has always been my ambition to be manager of a (testing) company. And I achieved that a couple of years ago. So far so good… But then something strange happened: I gradually realised that I wasn’t happy being a manager and, of course, people started noticing it.

Why I wasn’t happy? Although we discussed testing most of the time, testing slipped away from me and I started to miss the actual testing in my daily work. And I felt left out: everyone at my company was testing, except me!

Time for a drastic career change and go back to being a tester again!

Walk the talk (again)

Those that have worked with me for (quite) a while – without exception – told me it was great that I joined the testing ranks again and never questioned my new/old position or decision. Some even said that I should have never left the ranks in the first place… Of course, there were others that immediately asked questions like : “When are you leaving?”  “Have you found a new job?” “Doesn’t it feel like a demotion and major set-back?”.

My answer was and is simple: there is no reason (yet) to leave. My company is offering me what I want: challenges in testing. By stepping up to those challenges and “walk the talk (again)”,  the questions automatically seem to stop…

Several colleagues and (test) friends suggested – independent from each other – that it could make a nice blog series writing about a “successful journey” from manager to tester. It is going to be fun, they claimed, since most “successful journeys” cover the opposite direction.

So, I decided to give it a go. Reason: my last blog post is several months old, the journey is worth writing about (I think) and I hope I can inspire some of you to do what you need to do: make sure that you keep doing – or go back to – what you like, what you’re good at and what gives you energy!

One stop shop

Luckily, I never completely let testing go, so I was aware that a tester needs to be a one-stop-test-shop for the projects and the teams you participate in.

You have to make sure that you quickly contribute – in any context – the best way you can to your team or project.  Whoever your customer is, you need to be able to help them, support them and ensure they get the test service they need.

By the way, tester or test service to me is a broad concept: as long as it covers an aspect or service slightly related to testing, I tend to consider it a tester’s job. Well, at least supporting it…

Turns out that my experience enables me to “get in the helicopter” every now and then and by doing so, I get a quick view of the “world” and am able to give advice that is in line with the facts.

So far so good, I kicked off a new category in my blog. Hope you’ll enjoy reading my blog as much as I enjoy writing them. Feel free to provide feedback (positive, negative, thums-up, thums-down, smileys, frowneys,.. )! I really like being a tester again, no regrets whatsoever. Since I have constant access to the magic of testing, there is inspiration enough for new posts! Next one might be about test automation, testing in Agile/SCRUM teams or both.

La plus perdue de toutes les journées est celle où l’on n’a pas testé.
(paraphrased  – Sebastian Roch Nicolas Chamfort, Maximes et Pensées)

TI – any place, anytime, anywhere

I was invited to be the opening keynote speaker at the first EuroSTAR roadshow in Warsaw. And if I would like to discuss “Test Improvement” please. This is a synopsis of some of the points I made and the audience liked.

I want my team to improve, but they are not allowed to change a thing…

Don’t think I’m joking, I’ve been given this “additional” assignment several times.

It might be based on the natural human fear of change, maybe it’s because it’s hard to leave your “comfort zone”.
People could be uncertain about their own role, now and in the future. In fact resistance usually originates from the fear of the social impact of the change: does it impact my career? does it affect my relation with my coworkers? And what about my career?

Participation

Getting the people involved in the change participate in making it, is an important step in overcoming this. Participation cannot simply be switched on or bought in a store. Be aware that it’s not based on being invited to meetings, discussion sessions and workgroups, it’s based on the feeling that people have about their role and influence on the changes at hand.

Helping

And yes, of course you can help them. Crucial is that helping people is NOT forcing them to do it “your way”. It is all about enabling them to do the best they can in their context. In other words: give them the means they can use in their situation.

Wherever you put the emphasis, “means they can use” has to be true.

• The MEANS have to enable them to achieve what they want or need to achieve.
• THEY have to use it, so it has to be in line with their skills.
• Whether they CAN use it depends on their abilities and if they are allowed to use it.
• If people choose to USE something it’s because they think/know/feel/hope/expect/… that using it enables them to do it better, faster and/or cheaper.

Reference

If you’re interested in the full presentation, you can see a video coverage of it at Test Huddle. You can also view it on SlideShare.

Final thought

EuroSTAR 2015 Opening

I promised to share the opening of EuroSTAR 2015 with you. That’s why I’ve created a subset of slide images and share them in this image-blog.

Thanks for the many positive reactions I have received on the 2015 event.
Cheers
Ruud

PS As soon as I’ve recovered, I’ll share more insights and thoughts on the EuroSTAR 2015 conference through this blog.

Test Improvement is choosing

For me personally, one of the most difficult things in life is choosing. Especially because I don’t know what would have happened if I had chosen differently… And in the back of my mind, that “what if” game is constantly played. 

So my expert advice in test improvement is to choose wisely and consider several (all?) options. Don’t make choosing an everlasting process, just make sure you choose what helps. To be able to choose, knowledge about and experience with several approaches and models is necessary. Try to find a way that enables you to compare them and find the combination that suits the context and the purpose.

What usually helps is looking at aspects like:

• Testing only?
  Is the scope of the model testing-only or does it consider a broader area?
• Training?
  Is training available to ensure that everyone involved has the opportunity to learn about it.
• Staged or continuous?
  Is the model prescribing a standard sequence of improvements in (big) steps or does it give free choice in the sequence of improvements in (small) steps.
• SPI related?
  Is the approach or model linked to a Software (Process) Improvement model or approach that is already used within the organization or linked to a SPI approach that would help to improve more than just testing.
• Scalability?
  Can you apply it to help a single project? A team? A department? A complete organization?
• Are improvement suggestions “included”?
  Some models and approaches focus on finding the current status of testing and the required status; several include concrete improvement actions and highlight enablers
• Required budget / investment?
  How much time/effort is required for preparation, execution and implementation of the implementation. What is the claim on the organization? Especially since you take into account that Business as Usual will always need to continu. And that the required budget needs to be available; in other words, your approach has to be fit for budget as well.
• Top down? Bottom up? A mix?
• Alignment with current test and development approaches, e.g. waterfall, iterative, incremental, Agile, Scrum, DevOps.
• Culture
  Align your improvement approach with the culture of the organization and its peopke; although this is not typically something that is defined in a SMART way, this is key to success.
• People
  Do the people in the organization like the improvement approach and are they willing to change in line with it. Again not typically something that can be defined in a SMART manner…

When I choose based on these – or similar – aspects, I tend to organize a walkthrough of my approach with experienced fellow testers. For me, it softens the “what if” voice in the back of my head.

Thank you Cyanide & Happiness!

The models and approaches to choose from are plentiful, probably worth (yet) another blog post.

Back to test improvement – for now

Last week I organized a meeting with several colleagues who have, just like me, several years of experience in the improvement field. I’ll try to cover the topics we discussed in a number of posts, this is the first. The meeting turned into a great discussion about test improvement and we discovered – despite working in the same industry – so many different stories to tell. Any situation is different, isn’t it?

Which underlines the most important lesson to learn in successful improvement: there is no such thing as “one tool to rule them all”, sometimes not even within one organization or department.

This is the what differentiates a manager from a leader: a manager sticks to his model of choice and tries to fit the context in his model; a leader uses what is “fit for context and purpose”. And in many cases, that’s more than one model or no model at all.

So, if you really want to improve, one of the things you have to keep in mind is that you need to focus on improving and not on the model you want to use.

The characteristics of each test improvement model define its added value and its limitations. And that is why the foundation for succes needs to be laid at the start of any improvement initiative: what model(s), if any, will you use?

To be able to decide, you’re first step needs to be defining the objectives of your initiative:

• Reduce time
• Save cost
• Improve the quality of the delivered product
• Improve the quality of the test process
• Improve the skills & knowledge of the team
• ……

Scope comes second:

• what area will you consider
• what is the size of the group to consider
• is only testing considered or do you include other disciplines like development and operations as well

Third is taking into account the resources that are available for the initiative:

• Budget
• Time
• Skills
• ……

Culture of the organization is fourth: are they open to change, eager to improve. Context is fifth: what approaches and tooling are already being used. And sixth is your best guess of the current maturity: the more mature an organization is, the easier it is to implement improvements and ensure that current “good” practices remain in place.

In other words, current & future test improvement has to be:

The next step is to come up with an approach that suits all of the above. Worth another post, I guess.

The Awful Truth About Test Estimation – discussed

At EuroStar I faced the challenge of discussing estimation and the awful truth about it. To make my live a bit easier I decided to use Prezi. If you Google-Fu, you’ll end up with all kind of stories (I’m deliberately not referring to them as presentations). Some of them are really great and if you look at them, they take you by the hand and tell you a story. Hopefully the story that the creator wanted to tell you…

Why mention it in this blog? Because the challenge was not in telling the story but in estimating the time required to tell my story. My ‘historical data’ is based on PowerPoint-slides, including animation, that help me tell my story step by step. Including details. Prezi is different, it’s like the red line. Not too many details, just some info to support the story. Mainly graphics, not text. So, I know how much time a PowerPoint deck takes, based on the number of slides. Now translate that into how much time a Prezi- flow takes!

I guess it was a combination of experience, knowing what I wanted to tell, trial runs, experience and luck that made it happen exactly as I estimated: after 30 minutes, it was up to the audience and my moderation skills, to have a good discussion about… estimation! Which lasted – as planned – the rest of the available time.

My point is that, even if you change the “tool” (from PowerPoint to Prezi), and the context (from a talk into a question session), you can use your experience and understanding of the “new context” to come up with an estimate. Also for your next test assignment.

Effort estimation is a necessary evil and will never go away…

The reason for any tester to get sucked into test estimation is a simple one. Your manager at some point in your career, will ask you how much time it will take to test the next release, project, product. You’ll probably do what I did: check what is – planned to be – in it, compare it to the previous thing you tested and come up with a number. A bit less than before if you’re more comfortable with the way of working and the system under test. A bit more if you’re feeling unsure about the quality. As a result, your manager won’t like it and you’ll probably get less than you’re asking for. Maybe, because it is an experienced manager that tricks you into believing this release is less complex or critical. Maybe because you’re not able to substantiate your estimate. Call it character, call it ambition, when this happened to me, I was not happy so I started my “quest” for a more reliable effort estimate. This quest is still going on…

I do realize that there is a discussion “against” test effort estimation in the traditional form, maybe even in “any” form. The arguments are valid as many start with “I don’t know…” (see one of my previous posts). On the dots you can put any item: my team, the product, the development team, the business, the business case, the target, the aspects to tests, where the defects are, what the deadlines are, … This is a non-limitative list and I know you can come up with many more items.

The awful truth is that we have to come up with an estimate anyway. Either we need to tell our manager what time we need to test or we need to show him what we can do within the available time. Including a high-level description of how, when, what, …. Whether we want to or not, because if we don’t do it, he will. Since I believe that craftsman are able to better estimate their work and to discuss, argument and agree about it, I think testers should do it themselves. It’s better than being told and sigh…

Especially for a tester, an estimate is an approximation, nothing more or less. We know that all kinds of things will happen that will disrupt progress, require additional time to investigate or explore, lead to a major change in the scope of the project and/or the product. I know we don’t control most of the factors that influence the required effort. But that doesn’t mean we can’t take them into account! In my opinion, we need to “collect” these factors and show the impact on the test estimate.

And by the way, against all current discussions and opinions, I do believe there is a “best practice” in estimation: you need to discuss, monitor, update, change and adapt your effort estimate continuously.

Why improvement usually fails…

If you want to improve you have to invest time and money. What usually happens, is that you look for what everyone at that moment considers to be the right thing to do. So you tend to adopt the test approach, including tools and templates, that is “hot”.

Let’s say, you decide to implement risk based testing. You learn how to translate business risks into an appropriate test approach, including techniques, and create a beautiful plan and strategy.  And then? Your (pilot) project starts and fails…
Why, you wonder. You applied everything you learned, went to conferences, listened to the gurus, read the books, hired expensive consultants, …

As a test consultant I had the privilege to work in different organisations and projects and discovered a trend in “failures” and “successes”.  It boils down to one common denominator: did you or didn’t you involve the people who do the “real thing” in the test process? Yes, the ones that do the actual work a.k.a. the testers.  Somehow, they do not get involved in the early stages of test improvement.

We tend to start with test management, make the test organisation more professional, develop a test strategy, follow a lifecycle (plan, prepare and execute), use the right tools, select adequate test design techniques … According to the book! Don’t we make the same mistake at the start of a project? Forget to involve those same testers in the early stages?  When we define a test strategy and write a test plan, we talk to the business, the developers, all of our stakeholders, …

Don’t forget: the testers are the most important part of your test process and improvements, they have to learn to apply the new methods, tools and templates. In fact, they are the ones that have to change the most!

I found out the hard way that a few days training – even from the guys that wrote the book – is not enough for that senior business tester that has been there for over 10 years and knows the system inside out… And what about the “fire fighters” that have become heroes because they find, fix and retest defects quickly? They will not be convinced, they won ’t even attend!

So it’s all about people and how you involve them.  Take into account the capabilities and potential of your test engineers. Listen to what they have to say.  On the short term, you might end up at a maturity level you do not like. Or your approach is not exactly what you want it to be. But you know what, if you help them manage the transition, they will reward you with the highest level possible in the shortest possible time.  You will become truly successful.

It’s not your test management approach that decides how successful you are, your test engineers make it or break it. TMap? ISTQB? RST? As long as your people are able and capable to apply it in their day to day work.