The Importance of a Tight Testing Loop

A tight testing loop (the time taken to test any change you make) is an incredibly important goal in any software project and one that is often overlooked and not understood by management or clients, not that either generally understand the point of spending time on improving technical quality.

The time it takes to test your app is directly proportional to the time taken to develop new features. Every time you build something new, you need to test in some way whether its clicking around the app or running a set of unit tests and the longer this process takes the longer development takes.

Some of you may think I'm taking this too seriously but allow me to use some examples from my career to demonstrate why this is a point that needs making.

Click Testing

I have worked on several systems where the idea of unit testing is a strange and alien concept but deadlines are still a reality and this necessitates a quick run through the feature you are trying to implement, with nothing more than will and caffeine for support, every time you change or add something. This is an imperfect system but many developers manage to write squeeze out viable, if difficult to maintain, systems this way.

This process is made significantly more difficult if red tape and bureaucracy get in the way. For example, in one system I worked on we were forbidden to create a local copy of the database for testing. This was ostensibly to ensure the security of the personal information contained in this database and this is indeed a valid concern but it makes testing difficult as all the developers on the team were expected to share a development database. This situation lead to many problems such as changes to the data model having to be pushed to the whole team as soon as they were made and data discrepancies while testing. When someone deletes an entity that another person just created it tends to create some homicidal thoughts…and sets back testing by however long that entity took to create.

This problem could have been somewhat alleviated by creating separate databases for each developer on the test server but the pile of paperwork and sign-offs and arguments this would require was daunting.

Still, we are in the business of solutions to problems and this one can be quite easily solved via one of several approaches:

Masking

When creating a local copy of the database for testing a script could be written to mask out any personally identifiable information in the database. This approach is not as scary as it seems at first, you do not need to block out all of the data, just the parts which identify an entity. For example, you do not need to mask out account balances if you cannot tell who the account belongs to.

Test Data

Another approach is simply to generate test data. This is usually easier if the scripts to generate this data are built along with the entities but it can become time consuming if you are dealing with a large existing application with many entities. This approach has been made much easier with the advent of tools such as those provided by Entity Framework which can generate test data automatically.

In-Memory Repository

If default repositories can be easily overridden or replaced using the dependency injector, a simple in-memory repository can be used to facilitate quick, hassle-free testing without the need to external storage and test data can be added to it on creation in code.

Unit Testing

A comprehensive suite of unit tests is the best way to ensure the integrity of your application when developing new features. Being able to check that your existing application logic is not adversely affected by new changes at the click of a button is invaluable but unit tests are not immune from the evils of slow testing.

Ideally you should be able to run all of your unit tests in, at most, a few minutes. I have worked with a system in which the unit test project took between thirty minutes and an hour to run because nearly all the tests were running the full DI container and going all the way down to the database. The problem with this situation is that while the tests are still very useful when run by the build server it is impractical to run them often while developing. This leads to a situation in which the tests are only run, and fixed, after the task has been completed and this removes a large part of the advantage which unit testing provides.

In my opinion a good suite of unit tests should cover upwards of 80% of the core domain of the application (we don’t need to test property assignments and that sort of thing) and should be able to be run in a couple of minutes at most, preferably less.

The Cost of a Slow Testing Loop

A slow testing loop slows down development significantly as the time taken for testing is multiplied by the number of times a developer needs to test their code…which is a lot.

To get management and clients to understand this concept, and the fact that time should be spent making testing faster, can be very difficult, it’s best to speak to them in a language they understand: Deadlines. More time required for testing equals more time required for development which equals less work done in more time.