How We Test - Overview
Before releasing a new version we go through an exhaustive test procedure. Whereas most companies test their products we feel that this is a bit nice. Our aim is to kill ImageGlue using any method possible. We try to break it so that you won't.
As well as testing in-house we also test on our own live web server. Many component vendors don't have a live demonstration of their components. Others have a very limited range of capabilities on display.
We don't see how you can possibly produce quality components unless you have a live, comprehensive demonstration on your own test server. To do less would be to expect your customers to be the first to test your software live.
We run all our tests on multiple platforms and on both single and multiprocessor machines.
How We Test - Detail
First we run over a hundred different scripts, each of which is designed to test a particular feature of ImageGlue. Every script produces a set of output images which can be compared against reference copies.
Secondly we stress test using standard browsers running on a number of different machines running against a client server. The method we use (auto-refresh) places a high load on the web server while allowing us to keep track of what a client visiting the pages might see.
Thirdly we stress test using the Microsoft Web Application Stress Tool. This allows us to place a vast load on the web server testing a variety of different aspects of ImageGlue and see how the system performs. We leave this test running while we monitor memory and processor use.
Only if all these tests have been passed we release the candidate as a beta. We install it on our live web server and place it on our web site as a beta download. Because of the advanced error handling methods we use, any errors that occur will automatically be saved in our web server's Application Event Log.
If after a defined period of time no errors have been reported by beta testers or recorded on our web server then we release the beta. If the candidate fails at any point it's back to the start.