The content about xtreme testing that i have stumbled across focuses and emphasizes only on the software development process with little or no emphasis of how a testing team fits into an extreme programming project or how the process needs to be tailored to accommodate testing teams and the activities initiated by them to test software.I have also attempted to outline methodologies that can be adopted for testing teams to undertake the principles of extreme programming when the corresponding development is undertaken using conventional software development paradigms.
Extreme testing derives from the concepts outlined for extreme programming (XP) and distributed extreme programming (DXP) and tailored to be applicable to onshore-offshore model of software development, single & multi vendor model of software development. This approach retains the classical concepts of eXtreme Programming like simplicity, discipline, communication, quality, and flexibility
Introduction
Extreme testing (XT) derives from the XP and DXP agile development methodology and implements its core practices like communication, simplicity, feedback and courage. The following section outlines how most of the 12 practices of DXP can be applied to conventional software testing methodology namely,
- Planning Game
- Small Releases
- Metaphor
- Simple Design
- Testing,
- Refactoring
- Pair Programming
- Collective Ownership
- Continuous Integration
- 40-hour Week
- On-Site Customer
- Coding Standards
The principals from above will be adopted into extreme testing on a strict need to have basis and should be aggressively tailored to appreciate the fact that the process was initially designed for software development by keeping testing as a back end activity.
The methodologies of extreme testing (XT) which will be detailed in later sections will appreciate the core problem areas that current DXP methodology addresses namely,
quick and lean response to changing customer requirements from a testing perspective. How an extreme testing (XT) flavor can be added as a plug-in to a conventional development project.
The following are the practices from eXtreme Programming that are adopted for eXtreme Testing.
1. Planning Game
The “planning game” concept of typical Distributed eXtreme Programming (DXP) address the following key decisions:
What will be accomplished by the due date?
What to do next?
The extreme testing (XT) approach will also answer these 2 questions. The tailored form of it being:
What deliverables will be tested by the due date?
What deliverables will be accepted into QA?
Conventional DXP works in iterations of 2 weeks. Where the planning game is re-envisioned and re-factored at the end of every 2 weeks. The 2 key planning steps involved in the planning game are:
1.QA Test planning:
What modules would be tested over the next 2 weeks? This forms a very critical part of steering of the QA activities since QA provides a customer commitment to provide complete test results of a named list of modules. The number of modules to be incorporated in a 2 week burst has to be kept at an optimum since the XT model of testing would aggressively test the modules in QA. Hence, the number of bugs reported for the module would be higher in the 2 week burst and if QA test more than an optimum number of modules the development team may be bogged down by defects reported and fail to make a defect fix delivery for all the modules.
* The core of this activity lies in the fact that
All modules that can be tested in parallel SHALL NOT be accepted into QA even if a build is available for deployment
The focus of eXtreme Testing(XT) is to aggressively test a select list of modules in a burst and NOT lukewarm testing of multiple modules
- Advantages of this model of planning
The number of defects reported at any given point will cluster around a select list of modules.
The tracking of bugs is less tedious.
The development team can focus on the clean-up of a restricted number of modules.
The modules that will be tested are prioritized by the customer.
The customer perceived value addition in terms of a bug free roll-out of his high priority modules will be enhanced.
The completeness of testing is inherently transparent to the customer since the team is entirely focused on a select list of high priority modules.
2.Iteration Planning
This is the stage of planning where the QA team gets an alignment with customer regarding the next modules that needs to go into QA on a priority basis. Depending on the extent of bug fixes delivered and tested in the previous 2 week burst the QA team can factor what number of modules can be accepted into QA and still retain team focus on testing of the new modules and re-testing of builds.
The advantage of this type of planning from conventional test execution planning is that in conventional planning, periods and dates are tagged to modules to move into QA at the envisioning and planning phase. The is no room to accommodate scenarios where extensive cycle times are required or where the number of bug fixes are high which would require increased regression testing efforts. The end result of such planning is an overlap of the QA efforts in testing new modules and regression testing of old modules. This often results in mayhem with respect to tracking and closure.
In iteration planning of eXtreme Testing(XT) QA can take a call to ramp down the number of new modules hitting QA at the end of every 2 weeks. Thus they can retain their focus and efforts on select modules thus ensuring better delivery.
2.Simple Design
This feature of XP has been tailored to suit testing efforts by laying emphasis on the design of test plans for a module to be tested to be kept simple. This would be the guiding principle of the entire test planning phase where the inclusion of a test case into a test plan will be on a strict “need to” basis. Rather than design and document n number of test cases for a scenario (which does not aid testing anyway) the testing team would study and arrive at a complete list of all possible scenarios for a module (this could be a derivative of use cases if implemented for the project). This activity shall be the precursor to writing test cases. From the list of these possible scenarios, the team would then identify scenarios that that be clubbed and tested in one shot. The team would then proceed to write the minimum number of test cases for each of such “clubbed scenarios”.The alternative approach for simplifying test design phase of (XT) for project which neither implement use cases nor do scenario based testing is outlined below:
1.List out the comprehensive set of functionalities that can/ need to be tested as part of the module under test.
2.Begin writing the first test case.
3.Analyze if the test case written address some supporting functionality/ dependency/ requirement.
4.If yes, analyze if the test case completely address the supporting functionality/ dependency/ requirement.
5.IF it addresses it completely knock off that requirement from the set of functionalities that you need to test since it has already been covered and a further test case would be superfluous.
6.IF it addresses it only partially THEN attempt to re-work the scope of the written test case to encompass this requirement too. IF you succeed then execute step ‘e’. ELSE continue writing test cases.
7.After completion of writing all test cases re-analyze the test cases written in the light of steps ‘b’ to ‘f’.
3.Pair Testing
This concept has been derived from the classical XP concept of “pair programming”. Pair testing is however, not a mandatory requirement to implement eXtreme Testing (XT). The concept of pair testing can be adopted when the team is extremely focused on testing a very few modules in parallel, that too extremely aggressively. The test pair formed should then resort to classical testing concepts like “exploratory testing” and “intuitive testing” since 2 resources are deployed for testing 1 module the productivity of bugs reported to testing efforts should not be compromised.
This scenario is applicable to a highly mature team of tenured testers who have a comprehensive grasp of the system functionalities and can perform real time prioritizing of the test cases planned to be executed by spotting the defect density in a module and zeroing in on testing those functionalities early in the testing cycle.
The benefits of pair testing are:
Early reporting of a large number of defects.
Enhance the customer perception that the test team is highly efficient in spotting defects.
The paired team has to be dismantled as soon as the defect reporting stage reaches its “critical mass”. This methodology therefore assumes importance when fresh modules get deployed into QA. This would not be necessary for regression testing efforts where the number of bug fixes delivered is less.
4.Metaphor
This is an approach where the testing team evolves a common and complete understanding of what the system is intended to do at a macro level and what each piece of it coming into QA during a 2 week burst is supposed to address. The implementation of this concept is done through brain storming sessions prior to starting a 2 week testing burst where the team will review and strategize the activities planned for the week to arrive at a common understanding of the modules coming in and how it integrates to the already certified modules. This can help test teams evolve their understanding of the system and will help at advanced testing stages like end-to-end or clean pass testing.
5.Refactoring
This is a critical concept of eXtreme Testing (XT) where the initial test design documents, test strategy documents and test plans are treated as “live documents” and the team is free to enhance their test suite, augment the test coverage in real time during test execution or during the look ahead meeting they would have every 2 weeks to evolve the common metaphor (discussed in the above segment). These documents can then be updated on a “need to” basis. This flexibility would allow the testing teams to actively document their test efficiency improvements during the entire lifecycle of the project.
6.Collective Ownership
Collective ownership of all QA deliverables is a mandatory part of eXtreme Testing (XT). This means that all artifacts and plans can be reviewed and changes proposed at any point of time.
The benefits of implementing collective ownership:
The QA deliverables get the attention of the entire team.
Cross dependencies and redundancies of test strategies and test cases can be caught by any member of this team.
The probability of code segments and snippets of automation scripts developed being re-used is greatly enhanced.
The testing activities will hence become further lean and the bulk of test cases and LOC’s involved in automation scripts would be reduced because of extensive re-use or re-scoping them to be used for multiple scenarios. This involves test cases and automation scripts used across modules being re-scoped or re-used to produce a lean mode of operation.
Collective ownership will ensure that there is cross expertise in the team. This would help is de-risking the project by being heavily reliant on a single person to ensure delivery.
7.Continuous Integration
Continuous integration requests will be done for to the development team. This is so that instead of testing individual components till the fag end of testing or till end-to-end testing the QA team will be testing new modules by integrating them into the bug free modules already existing in QA. The benefit of this mode of operation is that integration does not become a one shot activity where multiple interfaces can fail OR integration leads to collateral damage on a large scale. This could make the task of defect analysis extremely tedious. Hence the focus in eXtreme Testing is on a continuous bottom up integration in QA. This would ensure that during end-to-end testing the entire system as a whole can be tested without being bogged down by minor interfacing issues being raised and fixed at the end of the testing effort.
8.40-Hour Week
This is a mandate of eXtreme Programming (XP) that is retained as is in eXtreme Testing (XT). Since teams adopting eXtreme Testing would endure long periods of concentrated efforts it is mandatory that they are not to work overtime. It is deemed in this paper that the necessity of adhering to this and the impact of not adhering to it need not be explained.
“XP teams are in it to win, not to die”
If you enjoyed this post, make sure you subscribe to my RSS feed!
