Software testing. What is it?

In the digital age and connected world, software testing is even more important than ever - our Test Engineer, Mike Keir, explains why.

A software tester walks into a bar, then runs into a bar, then strolls into a bar, then gallops into a bar, then saunters into a bar…Ok, terrible software tester jokes aside. Onto the serious matter of software testing!

In the digital age, as humans become increasingly intertwined with technology, is it relevant to test digital solutions?

The obvious and simple answer is yes!

Ok, I won’t leave you hanging there either. Testing is a serious business after all. The reputational hit a company can face, because of an obscure bug going live in their solution can result in years’ worth of bad press and financial losses. Look at the recent British Airways IT failure. The final financial figure for that will surely run into the billions, after ticket refunds, cargo and goods not being sent, fines from regulators and clients etc.

The more you know

So, now that we know you should be testing your software before going ‘live’ with it, what is the best way to approach it?

This is a somewhat vague question and could be answered in a multitude of ways. Unfortunately, there is no one-size-fits-all remedy. The answer is directly related to the project and the result of what will be delivered. Take, for example, a piece of medical software that controls an MRI machine in a hospital. This will need rigorous testing and to be checked constantly throughout the full lifespan of the machine. Whereas, an app that counts how many steps you take per day will have a far less serious consequence if a malfunction occurs.

You really want to test your software as early as possible and do it often. This saves the project, and by extension, your client - money. It adds value to the project by giving confidence that it will do what it was designed for, and potentially brings in repeat business when you deliver a stable and easy to maintain product.

Testing can be done by anyone, but is best left to the professionals. If you don’t have any testers currently, why not?! We’re great people to have around! We like to think outside the box and try to break everything! Nah, just kidding, but seriously, we like to try and find all the little nooks and crannies that the solution has and see what will and will not work. Does that tick box do what it’s supposed to do? Is this feature what the client wants from this part of the system? What happens when I enter text in a number field?

Testers add value to projects and give confidence that a digital solution bridges what the developers write, and what the client wants.

There is no magic wand

There is no way to 100% guarantee a fully bug free digital solution. It’s impossible. The reason this is impossible is simple. Money. There are so many different fringe cases and not enough money to plough into testing. Companies will never justify spending money on the fringe cases to be able to account for the one person in one million that still uses Windows 95 and Internet Explorer 5.5, so you must aim for the most coverage that the project budget will allow for. Most companies try to cover as broad a range as they can, which usually misses a small percentile at each end of the coverage scale. The most used devices and the most used operating systems are the 5%-95% coverage companies shoot for. Again however, this comes down to the requirements of the project. If you are developing an iPhone app, you won’t want to test it on an Android phone…

Have a testing game plan

When approaching testing – you want the best coverage project money will allow and test as often as possible, this is the utopian dream! If you have a short project and few testers, perhaps covering the critical systems would be enough for an initial launch, then follow this up with some more testing of the moderate parts of the system once the solution is ‘live’.

A large project or a project with far reaching consequences if something goes wrong, would be best with continual testing from as early as possible right through the entire lifespan of the project to ensure maximum coverage.

Have independent testing. Have another dev from a different team come over and check the code. Have project managers go through all the expected paths. Get some testers to try and break the solution. Sit random people from around the office down in front of the solution and ask them to do X/Y/Z and observe if they can do exactly that. Did they manage it easily? Note down what issues they found, what they had trouble with.

But I don’t know what to test?

The most critical parts of the solution are a must. Anything that involves money is a good start also. So, anything that could potentially lose your client money should be high priority. If the client loses cash, they will not be a happy bunny! List out the risks of the project. What would happen if X part doesn’t work when it goes ‘live’? Once you know the risks, you can then come up with a plan to try and counter that. Knowledge is power in the IT game.

Test, test, and then test some more…

A major failing of companies, is that they test something once and assume that it is then flawless and needs no further testing. This is most definitely, the wrong approach. ANY update to the solution could, and probably will, introduce more issues. Things that developers have not foreseen at the time of coding, or could not predict would happen after X deployment.

Regression testing is the act of retesting things that worked the last time you checked. I can hear you ask, “But why? If it worked before, I shouldn’t need to test it again and spend more money going over the same things???”. Well, let me tell you why you should!

A lot of issues in a digital solution are introduced as a direct result of a new patch or update to a different part of the system. Regression testing is an integral part of the testing process. Regression testing is to software development, what butter is to bread when making a sandwich. One cannot exist without the other* (*You may not like butter on sandwiches, I won’t hold that against you.)

Conclusion time

To summarise:

  • You should ALWAYS test your stuff. Even just a little bit.
  • Developers should always be encouraged to test their own code, but the more independent testing you can get, the better. It has a direct financial gain to the project. The quicker and sooner issues are found, the cheaper it is on the project. If a foundation shaking issue is found two weeks before launch, it will be much more expensive to have the solution ripped apart and re-developed, than finding it nearer the start.
  • Have testers involved as early as possible. Get their opinions. They have a weird thinking on things and sometimes come up with questions nobody else has thought of. (Weird in a good way, honest.)
  • Always test the critical parts of the solution as a bare minimum. But, as best practise, test as much as you can afford to.
  • Test, test, and then test again. The more testing gets done, the more robust and stable the solution will be. Regression testing is your friend!
  • If all else fails, take a deep breath and repeat the software tester/Pokémon trainer mantra: You gotta catch them all!