We believe that testing is an extremely important function to the success of an Agile software development product; and that this function becomes even more critical in Distributed Agile software development approaches. This section discusses our learnings from our long experience in testing in a Distributed Agile context. Our successes our testimony to the fact that we have consistently created a positive and cohesive mind-set that is focussed on enabling our clients to achieve their goals. At all times, we have maximized communication bandwidths with an emphasis on collaboration. We also make it a point to invest in the right kind of toolsets, and pay attention to the computing environments that play a critical role in each project's success. For more information on our knowledge areas, take a look at our Best Practices section.
In a Distributed Agile context, testing teams have to constantly balance the need for the face time imposed by Agile, with the cost arbitrage and time zone advantages offered by the distributed model. Deciding on activities needing colocation v/s activities that can be safely parked in a distant location forms the crux of the 'Distributed Agile Testing Dilemma'.
There is no single answer to what exactly is the optimal mix in the composition of a testing team with respect to colocated and distributed resources. There is no consensus either on the activities that must be done at client location vis-à-vis activities that can be distributed. Based on BSI's experience, the following tables provide a snapshot of some of the activities that can be executed in a distributed context:
From our collective experience with multiple clients, test automation, metrics and defects management are some important aspects to think about in the distributed context.
Given the need to complete testing effectively in the quickest possible manner, test automation becomes the most integral part of an agile testing project. Our efforts to automate regression testing take into consideration scenarios that pertain to individual sprints, as well as scenarios relevant to the entire product, i.e. Sprint Level Regression & End to End Regression Testing. Sprint level regressions are focused on testing new functionalities that have been incorporated since the last production release and End to End regressions incorporate all core functionalities. Mature agile organizations with continuous integration processes would have automated most of end to end regression test cases so that it can be executed in a very short time window
A robust measurement system is required to articulate the ROI of testing. One of the key activities of a testing team is to publish a test metrics dash board. BSI has developed automated metrics dash boards for this purpose. The measurement criteria for testing should go beyond defect density and examine the business results of testing in terms of quicker time to market, cost of rework dollars saved, etc. Some key metrics that we recommend are:--
Quality of defects needs to be high. It is important that the reported defects articulate the exact issues in the best possible way and identify the correct parameters' severity, priority, injected / fixed version, etc., so that the analysis also provides meaningful information to improve upon. At BSI, our seasoned teams have been trained along these lines to spot defects and prioritize them on criticality; in order to ensure a glitch-free Agile software development process.
Turn the page to learn about some of the flavours of the testing function in Distributed Agile scenarios
The decision of whether testing should be colocated or distributed must be based on portfolio level considerations rather than on individual project level considerations. Some projects may have 100% of the testing teams located at client sites, whereas some others may have 80% - 100% of testing from distributed locations. As long as the portfolio has a healthy mix of colocated and distributed resources, the customer can enjoy the benefits of agility and cost arbitrage.
The importance of rigorous regression testing cannot be stressed enough to ensure a high quality product. Up-to-date maintenance of the regression bed is also crucial. However in reality, these activities are often ignored especially in agile testing projects simply because of the overwhelming time & resource constraints. The results are show stoppers during integration that put the entire project in jeopardy.
Having an independent testing team that leverages time-zone differences to focus solely on regression execution and bed maintenance has proved beneficial for us in many projects.
Automation brings out the "speed" and "coverage" in the regression testing required by the Agile Model. While other teams focus on feature testing, independent automation teams develop regression test scripts for already developed features. The execution of the regression suite can be scheduled accordingly along with automated build process, i.e., the suite runs after the automated build is successful. These activities can be planned to not interfere with one another and other testing activities.
The biggest advantage of this model is early identification and analyses of regression defects. Feature teams would immediately learn of the defects caused by the features that they had developed as recently as the previous day.
Flavours of Distributed Agile Testing ( contd... )
Typically integration testing would be conducted at those locations where the necessary integration environments reside. In such scenarios, all other testing activities can be carried out by other independent teams. The model works out better in a situation where support for upkeep of the environments is limited. This reduces the downtime that the teams may possibly encounter in following cases:
This would be a best fit if the development teams and the test environments are at a single location. The benefit of the model is faster resolution of technical issues. The downtime of testing teams would be minimized by quick resolution of build and deployment related issues. Also direct communication with the development teams for clarifications regarding reported defects will bring down the communication times required to address such defects. Also the teams would be able to report the end-to-end status of the stories and features developed at that location.
Want to know more? You can take a look at our best practices section, or contact us directly to learn more about our Agile software development and testing projects for other clients; whose success we have enabled using our tried and tested Star Agile Framework.