The case for Automated Testing

In Roman times, the engineers of a legion, upon having finished building a new bridge, had to stand under the arches as the whole legion marched across it for the first time. This formidable (and most likely apocryphal) performance was supposed to make sure that the builders would take extreme care in the design and implementation of the bridge.

While the software systems we build today lack the corporality of real-world buildings, the parallels can easily be drawn: we too, are generally trying to build a new piece of an engineering marvel, preferably on firm footing and ready to withstand a lot of stress. Even the language we use to describe our codebase borrows heavily from civic engineering – we often talk about architecture, pipelines, structure, integrity, stress tests, etc.

However, the reliability and final quality of the software systems that we end up building is far from those of our roman counterparts. I believe this is partly because we ignore – either from ignorance or well-meaning but ultimately misguided prioritization – the best tools and best practices of our discipline, which are absolutely necessary to the success of any project. In this writing, I shall try to elucidate one such tool, one such best practice, that is often ignored: namely, automated testing.

The theoretical problem

Back in the early 20th century – back when “programmer” was not a job title yet – the very first professionals who were fulfilling this very role were mathematicians. When the question of proving whether the code they wrote works correctly, they naturally thought it would be a similar process to proving a mathematical theorem – and in fact, some tried to do so, attempting to use the tool of mathematical induction (proving that it works for N, then proving it works for N+1 as well) to definitively prove that their code works. Needless to say, everyone quickly realized that this is a very laborious, time-expensive and tedious undertaking, and is thus not a feasible solution. Thankfully, another method was at hand, ready to be utilized: the scientific method.

In science, we cannot “prove” a law exists, but by carefully creating falsifiable hypotheses about various phenomena, and then testing said hypotheses to our utmost capabilities, we can conclude that to the best of our knowledge – unless proven otherwise in the future -, we can assume that something works according to our hypothesis. Simply put, if you try really hard to break something, but ultimately can’t, you may conclude that it’s solidly built and will perform reliably in the future.

Notice that the above implies that without extensive testing, we cannot purport to claim anything about how well our product is functioning. All we can say is that the last time we checked, some of it seems to have worked. And God help us if anything goes wrong.

Changes to the code can introduce bugs, break or modify functionality

Software can easily be changed, adapted, molded. This malleable nature is precisely why it can be used to so quickly achieve some truly amazing things. Because the codebase is not a static object, it can be conceptualized as an almost living, breathing, evolving thing: it expands, contracts, it changes internally, can grow new limbs, and so forth. This is a double edged sword – it is precisely because of this ever changing nature of code that most of our problems stem from.

Whenever such a change occurs – be that however miniscule – there’s a risk that we might break something that was earlier working perfectly fine, or introduce some bugs to the code. However hard we might attempt to mitigate this risk, it can never be completely eliminated. This is why regression testing – basically re-running all tests and verifying that everything still works – is so imperative. The only way to cost-effectively achieve this is by automation.

Tremendously helps with refactoring

Refactoring is a critical part of software development, but it can be challenging to do without introducing new bugs. Regression tests can provide a safety net for refactoring efforts, allowing developers to make changes with confidence and ensuring that any changes do not break existing functionality.

Tests serve as specification and documentation

The great thing about tests is that they also serve other functions, besides making sure that future changes are not breaking any functionality.

First, if you think about it, they are quite literally the specification for how your code ought to work. In order to solve any problem with code, you need to first describe what the desired behavior of your software will be. Writing unit tests inadvertently forces developers to think about the intended outcome of their code, helping to clarify requirements and ensure that the software meets user needs. This even cuts down on bugs, because a lot of “bugs” are just unspecified and unhandled behaviors in certain edge cases. By having both the developers and the business think and talk about these cases beforehand, we can cut down on after-release patches.

This is also why TDD – Test Driven Development, the method of writing tests before and during the process of actually implementing the solution – is such a strong tool: you will need to understand the specification to begin with anyway, so by writing the test cases first, you are not doing any “extra” work, you are just making that work more useful. (While this is outside the scope of this article, note that TDD should not be conceptualized as a framework for corporations, one that everyone needs to adapt (like SAFe for example) – each individual can start using TDD by themselves for the most part if they so desire, without completely changing the working environment and getting everyone on board).

Second, they also double as a form of documentation for the software. In fact, I would argue that it is the only trustworthy documentation that can exist, because anything else, no matter how extensive or well maintained, will not be able to keep up with the ever-changing nature of code.

Personal benefits

It is worth mentioning the personal benefits of automated testing.
While your mileage might vary, I happen to think the development process of writing and running unit tests can be characterized as a gameplay loop that many can find genuine joy in, and it can easily trigger a flow state. There is something oddly satisfying about turning a red “FAILED” cross to a green “PASSED” tick in itself – who doesn’t love a little dopamine hit, after all? And people who love working on something will generally produce better results as well.

Furthermore, even if you disagree with the fun part of that assessment, automated testing has become widely regarded as a best practice, and by mastering it, you will be able to further expand your ”software developer’s toolbox”, undoubtedly making you a better professional in your field.

Forces you to write testable code

To write effective tests, developers need to write code that is easily mockable, modular, and loosely coupled – in other words, testable. (Think: dependency injections/inversions.) This emphasis on writing testable code inadvertently leads to better software architecture and a more maintainable codebase.

Ultimately speeds up the dev process

While writing automated tests may seem like an additional step in the development process, it can actually help speed up development in the long run. Discussing various common objections regarding testing would be an excellent future topic, but it is outside the scope of this article. However, it’s important to note that by catching bugs early, providing a safety net for refactoring, and improving developer confidence, unit testing can ultimately lead to faster development times and definitely higher-quality software.

In conclusion, I hope to have shown you that automated testing in general is not just a useful tool, but an essential part of the software development process that provides many practical benefits. From catching bugs early to improving developer confidence and ultimately speeding up development times, automated testing is critical for the success of any software development effort. By recognizing this, we can – and should – use it to the best of our abilities, for it is a prerequisite to ensuring that our software works well. Although as programmers, we cannot perform a statement as literal and inspiring as the Roman engineers have, we can still benefit a lot from aiming at similarly high standards. I believe our industry, our clients, and ultimately, our end users will thank us for this.