There is quite a buzz around the idea of testing in production these days. Mostly, this proves costly for the service oriented companies. However, with the product based companies like Microsoft, this could be a step backward to the conventional testing methodologies. Moreover, testing in production involves less involvement from the QA team, which leads to a mess around the bugs.
For example, working on a service-oriented product like IBM WebSphere could prove to be tricky, especially when you create more than one application server on a single WebSphere Application Server infrastructure. By creating a test server and production server on a single base will disarray them if you wish to upgrade or modify the Software Development Kit (SDK). In order to avoid any conflicts between the two application servers, you need to configure their ports. However, there logs will be shared.
In such a situation it is required to update the test system more often than not, while the production system cannot bear frequent disruption. Hence, a repeated upgrade of the SDK would thwart the production system, affecting a number of users.
This sounds pretty unstable and is not acceptable with the IBM WebSphere Server. In order to make things simple and stable, there are few recommendations by the WebSphere SWAT.
The best way is to isolate the test system that is identical to the production system. Once you separate the test system, you can carry out the stress and load test of the application on the test system, before finally shifting it to the production system. A separate test system would allow the application to work seamlessly into the production system. In addition, it would have less interference from the frequent test activities. A separate test platform would be ideal for performance and integrity tests prior to any major enhancement. Moreover, you can replicate the production issues and test the fixes.
The idea is quite simple. However, it is little costly to create an identical production data center for testing. For anyone, who cannot afford to build a complete production replica, they can rather try to replicate the expected load through the load generators. By creating a diminutive environment, it would be easy to control and simulate the environment.
The basis of this article so far, lies in the fact that the testing in production is absolutely right, unless you make it wrong. Testing in production is an impressive way to uncover various distinctness of the real world. In addition, it overcomes various limitations of the lab based testing. By creating a separate identical production environment, gives you a great opportunity to explore more bugs, which was otherwise not possible with the in-lab testing.
You can still use in-lab testing method if it is identical to your production environment. However, you need to use the real data, workflows, and resources from the production. In case the lab is scaled-down, the use of real production data could take you much close to testing in production. In this way you can reduce the production users’ risk.
The risk involved in the IBM WebSphere was quite similar to this, which could have been avoided by isolating the test and the production systems. Many companies like Netflix do this by deploying the APIs on different server clusters. This minimizes the chance of services being destabilized by millions of users.
Risks can also be reduced by the process of exposure control, a way to reveal the new code to a few users. The advantage of having real users cannot be ignored; however, this allows you to manage any impact on the test code.
