A question always asked is “How much should we test?” and a lot of schools of thought follow on with answers. However, there is another question which id like to pose.
“What level of documentation is required?”
The norm for the above question that we practice is “Document till good lord is squeezed out of your test case”. The reason why this question is prominent is that a tester may take 10 minutes to test a scenario and spend 20 minutes generating documentation and backing up log files. The base duty of a tester is to provide quality assurance. When a test case passes the quality is assured for that snippet. Everything beyond that point is what id call an overhead.
So the question is how much overhead are you willing to bear. The funny reason being that a testing document seems capture a very short attention span. For example, My team spends a fortnight preparing documentation of a testing activity. The purpose of it being that it gets send out to “N” stakeholders who are too busy to look into the nitty gritty’s there and the document gets a 1/2 hour window where it gets “reviewed” at a very high level.
Under the circumstances, to be realistic i believe that a stake holder is not interested in what message was logged when a test case failed. He would be more interested in metrics that say “100 defects were reported at Phase 1. 98 were fixed, retested and verified and 2 low sev defects were deferred.”
So the guiding principal in documentation would be “DO NOT document the nitty gritty’s that would lose importance beyond the realms of your desktop. Invest time in collating metrics.” If that be done we would not be sending out spread sheets of test reports that have 183493 rows of backup log messages and a green column indicating it has passed. The industry has to reach a level of maturity where they trust a tester who says “Test case # 2322 has passed. “
One point to make is that:
“NO ONE IS INTERESTED IN A PASSED TEST CASE. ITS ONLY THE DEFECTS THAT MATTER.”
Realistically if there is contention regarding the “pass” status of a test case, that should be re-verifiable by reverting to a pre-defined snapshot of the test environment and re-testing the scenario. Cos even if you have dumps and documentation for a contentious test case you would still have to do the above thingie….. So then whats the point in the whole idea.
Ideally testing is a “Defect targeted activity” AND NOT “Documentation generation activity”.
“For those who believe, no explanation is necessary. For those who do not believe, no explanation is possible.”
If you enjoyed this post, make sure you subscribe to my RSS feed!
