Wednesday, January 21, 2009

Half baked versions are good for you!

Often, we find ourselves in a situation in which a feature is ready but the version as a whole is not: It can take anywhere between a day or two to a week until all the developers have finished working on a build and all integration tests are concluded, but during this time some features can be tested. So we came up with a method for speeding up the testing process: testing incomplete builds.

The idea is simple. When a feature is ready, we get an incomplete version. This version is installed on a separate, local, system and only the specific feature (or features) is tested. We don't test other components of the application and when bugs are found there we ignore them. This way, by focusing on specific, new, features, we are able to detect bugs before the release candidate version is released to the QA. Once we get the official version, we are more familiar with the new components and they are usually more robust as the most glaring bugs were purged earlier.

Testing in this method helps us test more and more often, but one must bear in mind that testing this way can't replace thorough testing later when the full version is tested. Some bugs only rear their ugly heads in more complex environments and need to be detected separately.