Sunday, 19 February 2012

Integration tests, how much are they needed?

I few days ago I was required to verify a custom Jenkins plugin was developed correctly and would be running as expected. The plugin in question was covered with proper unit tests and part of a big solution project including  some other modules using the plugin to achieve a job build task.  As it was a plugin, unit tests were not guaranteeing that it was ready to be used for functional requirements test.The question is how you can be confident with your module that it is ready for functional requirements tests?

It looks like today’s agile software development practitioners are generally focusing only on unit tests and omitting integration-tests, Maybe this routine is due to the fact that it’s generally difficult to test a module without isolating/mocking some external dependencies, that’s why a code base passing unit tests is generally treated ready for functional testing stage. In reality there is a certain need to verify that a module as a whole is tested before going further in the development pipeline. I think instead of omitting the integration-test phase all together, how much integration tests coverage  is needed can be asked. In most cases number of integration test cases would not surpass unit tests. Depending on project factors, usability and risk assessment, a percentage of integration-test coverage can be set for projects.

I would rather prefer  deferring heavy cost/slow tests to the end of development pipeline as much as possible, this would give more time to achieve a certain level of unit and integration tests coverage, which will ensure to check against a number of common pitfalls.
Ideally, if a TDD style development is followed  with concise user stories and clearly highlighted  feature requests, the unit test level coverage of the project can easily hit above 90% and integration test coverage could be set to a lesser target while functional tests coverage would be the least, only verifying high level user stories.

CD Flow