Software Testers Cannot Catch Everything

· 4 min read

This blog was originally posted on Medium.

The Bloggers Club-Sprint 6: What’s something that seems obvious within your profession, but the general public seems to misunderstand?

It’s taken me until Sprint 6 of the Bloggers Club to finally get back to blogging.

The Possibilities

I still consider myself to be in the software testing/QA industry by the way! I’ve thought a lot about this one. Some common presumptions I have heard are

  • Software testers just press buttons
  • QA/Software testers are failed developers
  • You need a degree to be a software tester
  • Or the counter to that, anyone can be a software tester if we just tell them they have to be
  • A software tester cannot use someone else’s application without being critical of it

The Focus

Finally, I settled on “Software testers cannot catch everything” after hearing some comments at home about buggy apps.

Picture the scene; you open an app, you’re midway through the signup process and you hit a bug. What’s your initial thought?…….“How did they miss this?” You’re thinking “Dammit sign up is where I decide whether I’m going to use this or not. How could they not have caught this bug that I’ve found.”

Zoom out from that thinking now.

How big is the app? How many user journeys or user flows are possible in the entire system? Hundreds? Thousands? Millions? More? I’m not here to make excuses for teams I have no involvement in, just to make you think more about this.

The Story

I was previously on a team where there were 6 full-time developers, a UX designer and me, the lone tester. We had a LOT of ways that a user could be on-boarded with our product. There were so many from the UX perspective that there were flowcharts. So.many.flowcharts.

The UX designer was a well clued in guy, we worked closely figuring out alternative flows a user might take around the flows he had in the charts. I drew out my own big A3 flow diagrams, an individual one for each step on his larger one. We thought about the user that locks their phone midway through on-boarding process 1 at step C etc. In summary, we believed we had thought of everything or at least we believed we had thought of enough to cover the vast majority of cases with some interesting edge cases.

The dev team took a stab at the on-boarding process from different browsers, devices and approaches once I had tested it. We were all sure we had covered all the scenarios we could think of. Confident, we pushed the changes to production. Go team, another successful release! Everyone testing the whole way from design to release and testing on production. High fives people.

And then, 15 minutes after the code was live, we got a support ticket. There was a user trying to sign up and they met a showstopper bug. That is what we call a bug that completely prevents the user from continuing what they were doing. Of course, the first question was “How did this happen?” Support followed up with the customer to see what steps they had taken to get to this point. Our developers frantically checked the logs. There was a user journey we had completely forgotten about. We hadn’t planned for it in UX, development or testing.

Of course with the benefit of insight into this issue, you can see how this happened. You can see how this wasn’t any one person’s fault. Many discussions had happened. The team was human and forgot one thing. It happens.

The Conclusion

Bugs happen. They might be small. They might be massive. A bug, usually, passes through many hands before it ever gets to the customer discovering it.

There are many reasons or ways this might happen. The example above is just one. Software testers, software developers, UX designers etc can’t catch everything but they can try to catch the worst before it gets to the user.