A question always asked is “How much should we test?” and a lot of schools of thought follow on with answers. However, there is another question which id like to pose.
“What level of documentation is required?”
The norm for the above question that we practice is “Document till good lord is squeezed out of your test case”. The reason why this question is prominent is that a tester may take 10 minutes to test a scenario and spend 20 minutes generating documentation and backing up log files. The base duty of a tester is to provide quality assurance. When a test case passes the quality is assured for that snippet. Everything beyond that point is what id call an overhead.
So the question is how much overhead are you willing to bear. The funny reason being that a testing document seems capture a very short attention span. For example, My team spends a fortnight preparing documentation of a testing activity. The purpose of it being that it gets send out to “N” stakeholders who are too busy to look into the nitty gritty’s there and the document gets a 1/2 hour window where it gets “reviewed” at a very high level.
Under the circumstances, to be realistic i believe that a stake holder is not interested in what message was logged when a test case failed. He would be more interested in metrics that say “100 defects were reported at Phase 1. 98 were fixed, retested and verified and 2 low sev defects were deferred.”
So the guiding principal in documentation would be “DO NOT document the nitty gritty’s that would lose importance beyond the realms of your desktop. Invest time in collating metrics.” If that be done we would not be sending out spread sheets of test reports that have 183493 rows of backup log messages and a green column indicating it has passed. The industry has to reach a level of maturity where they trust a tester who says “Test case # 2322 has passed. “
One point to make is that:
“NO ONE IS INTERESTED IN A PASSED TEST CASE. ITS ONLY THE DEFECTS THAT MATTER.”
Realistically if there is contention regarding the “pass” status of a test case, that should be re-verifiable by reverting to a pre-defined snapshot of the test environment and re-testing the scenario. Cos even if you have dumps and documentation for a contentious test case you would still have to do the above thingie….. So then whats the point in the whole idea.
Ideally testing is a “Defect targeted activity” AND NOT “Documentation generation activity”.
“For those who believe, no explanation is necessary. For those who do not believe, no explanation is possible.”
Recently i was talking to my fiancé (another prolific tester) on her drive down to her office. The topic of conversation being the bad quality of development at her workplace. So these are a few thought that came outta it and we decided to jot it down
There are many “maturity levels” that a development practice undergoes. Here is a skeletal framework of that:
Skeletal Development ( Writing & Compiling code)
This is the sort of code that should ideally be seen written by new grads from college or out of High School. Where a snippet of code is deemed to have the pinnacles of quality if it just compiles right without any errors. This is the first stage to being a good developer. Unfortunately this is a “coding standard” that i have found among a vast number of “experienced” developers in the so-called CMMi (deprecated IT standard) level 5 companies in India. This is best described as the infancy of a hacker.
Unit testing
This is when you take that first baby step towards QA. (Yes and we thought that is the height of Software Testing expected of a developer right??? Loooooong way to go…….. ) The practice is pretty prevalent now that the heading is self explanatory. Your write code and you write some unit test cases and then you execute them.
This is where the real game starts ( big boys are found here)…… This is where your code is not just Unit tested but it adheres to the coding standards defined for the programming language used. This is important because one of the aspects of Quality code is not just that it performs well and correctly…. but also that it is highly maintainable.
The relevance of this derived from the fact that 70% of the IT world out there are involved not in developing fresh code but in maintenance and enhancement of legacy code. So lack of standards in legacy code makes them un-interpretable and the enhancement code tends to be more buggy.
This is the stage of coding that is often left untouched. This stage comes into picture where your code has been developed, reviewed. Once the review is complete, the developer and reviewer has a sit-in to refactor the code. Now for those developers who are going “Huhh, whats refactoring” check out the following link below:
“halleluia to agile community for driving this practice…….”
Preemptive Functional testing
Preemptive functional testing is once of the final qualities of a legend class developer. Where he does all the above mentioned stuff and then sits down to do a bit of functional QA himself. He executes “black box” grade critical test cases and ensures that they pass.
The outcome of adopting the above standards are:
Your build “will never break”
Your QA will tear out the hair trying to find a bug
A rookie out of college testing you app will come out with zero defects.
Testing has to reach a new height to beat your code.
A legend class developer is born.
For all those testers reading this send this to your developer friends out there……..
With the advent and maturity of Web 2.0 concepts and feature rich dev tools, the impetus to develop desktop solutions for your computing needs should be seeing a decline. I was mulling over the “problem of choice” ( Matrix fame)…… What would be the factors to consider when deciding whether an application/system should be served as a standalone installer for a desktop app OR a surf-n-use web solution. These are some of the factors that favored going online with our apps.
Cross compatibility: This is one BIG advantage where you would’nt have to create Linux, Mac and Windows distro’s for the same app. Bad news for those companies that made a neat packet out of “application porting”.
Universal Upgrade: Web applications reduce the pain of distributing patches, service packs and version updates. This needs to be done only in the centralized application server and it trickles to all users.
Easy application support: This will come as a great sigh of relief to companies maintaining extensive service/ support personnel. The logic lies in the fact that a stable release of a software is done after extensive QA where trained testers try to break the code. So most support queries issues are with respect to system specific compatibility issues, missing support utilities and the problems due to OS localization etc etc etc… These are cleaned out in one sweep when you go online.
Heightened security and reliablity: Service continuity is enhanced if the application servers are robust and outages will not happen on an ad-hoc basis. If the application is processor intensive then high end servers can do a better job and heighten throughput as compared to a desktop.
Will be building on this in the coming days. For the moment just could’nt resist sharing this with you…..
Feliz Navidad
Feliz Navidad
Feliz Navidad
Prospero año y Felicidad
I want to wish you a Merry Christmas
I want to wish you a Merry Christmas
I want to wish you a Merry Christmas
From the bottom of my heart
As a continuum of my post on test approach best fit vs first fit, thought id present Heuristic testing as a means of getting this done….
Question is “What is heuristics”
Well, a heuristic is a rule of thumb solution to a problem that may qualify as the solution but need not necessarily be so….. This a terminology where emphasis lies on the potential of a thought, approach or method to be come the solution for a problem.
How does this help testing…… Well QA and testing has reached very appreciable maturity levels the worldover (barring India ) that for any given problem posed you have a sizeable set of solutions as compared to what you would have 10 years back. Let say i want to do funtional testing. There are umpteen number of options at hand like the following:
Sticking to age old manual testing.
An automated solution ( just because a testing can be automated it need not be)
The possibility of using my own custom scripts.
Possibility of using COTS products ( high industry affinity)
Now heuristic testing is a methodology where the set of n possible solutions are all given equal importance initially. Then begins the probing stage where the validity of accepting any of these n solutions is questioned and a process of elimination gets kicked off. Darwin says “The fittest will survive”. Well if thats true for the cosmos then why should not hold true for Testing Methodologies ?
The biggest truth is “Winrunner need not be the best solutions for your functinal testing requirements just because your QA Consultant know only that !!!!! ”
Sorry if that came across too hard…… but then in many cases i have seen the following classic case “My carpenter knows how to use the only the hammer and i want him to put in a screw. So he bangs the bejesus out of the screw till it goes in….. End result i have a wall that looks like its survived world war II……… and we give him an award for ontime service delivery but failed to realize that the next day a painter and a mason will have to come in for damage control post deployment………” Now i be that sounds like a familiar story…….
This is where we do Heuristic testing where the primary focus is not on getting A single solution but to ascertain what options we have. Then we get around to finding our best option after careful analysis by questioning the authenticity of each of our options and let the fittest solutions emerge as the winner….
My friend Kenney recently introduced me to the concept of stumbling. This a concept where the Internet goes democratic. You as user of the internet now have the right to vote for content/ information on the internet that you think is more precise and relevant than other websites that talk of the same thing. I believe that this brings us one more step closer to harnessing the collective intelligence of users on the internet. This is a great place to start stumbling:
Download the stumble toolbar for your Mozilla or IE.
Visit the sites you like and vote for them by pressing the “I like it” button.
You can also register for types of information that generally interests you and when you click the stumble! button on the toolbar it will fetch a random page from the interests you have registered. Trust me it makes for some cool reading. You can also invite friends to join this network and stumble on pages that your friends have voted for.
It was a really welcome sight to see stumble votes affecting google listings also. As a stumbler my mouse pointer drifts to google results that have a “stumble thumb” next to it
So what are you waiting for…… “Get Stumbling NOW !!!!!!”
Recently I was brainstorming with a highly competent tester based in Bangalore, India [a rare breed out here] and we stumbled on a startling fact. Requirement gathering and tracking methods have remained largely static for a long time. The delivery models in India with respect to QA and process are reminiscent of the late 90’s.
These are some of the facts that we ascertained:
The most prolific and commonly used requirementtracking tool in the Indian IT service sector is…….. “Microsoft Word”.
Every stakeholder/developer/tester in a project has a personal copy of the requirement document resident in his inbox.
Requirement changes are often not centrally held/ distributed.
Requirement discussions/ freeze is effected after multiple rounds of talks with the onsite customer/coordinator.
Software systems today have reached a stage of “critical complexity” where it is imperative that we hold requirements in a central repository. There core reason for no advances in this segment is because our internet non-savvy IT community is oblivious to the concepts of web2.0 and the “collaborative internet”.
The version of the internet as we have it today has moved towards distributed applications with added focus on communities,collaboration and harnessing collective intelligence.
The benefits this can provide to requirement tracking are the following:
Use Web 2.0 principles to build requirement tracking tools on the “internet platform”.
Provide features within the app like forums [to start discussion threads], chat enabling for real time gap resolution, wikis for public bulletin boards.
The feature rich UI capabilities of internet based applications can provide “mind map” like interfaces for provide a structural organization for requirements.
This would provide the benefit of greater usability and clarity compared to “referring to section 1.1.3 (a) or 1.2.4 (b)” to track requirements.
Requirements are the most critical pieces of information in the IT sector…. Information management is a science that Internet gurus are trying to perfect and they are the forerunners. So its only logical to adopt their practices in “managing” our information rather than stick to “Pubic library model of requirement tracking and analysis”.
I was extended and invitation to attend and present at the first barcamp in Kerala, India. Its a great feeling to be part of this technology collaboration stride being held at technopark, Kerala. This is the first of its kind to be held in this southern state of India.
For more details and to register yourself for this event visit: Barcamp Kerala
Pass on this link/info to your colleagues/ friends out there who might be interested to be a part of this.
For insights into what the concept of barcamps are all about, you can read my fellow blogger kenneys post about the event
Looking forward to attending this event and sharing my experiences with you in future posts…..
A picture is worth a thousand words…… Presenting the synopsis of what i mean
The nut bolt analogy would give u a small idea of how this is relevant to testing. There is always a set of N possible ways to test and application/product. This is the “superset of all solutions”. Out of this a small subset can be derived by using filters like
Feasibility
Compatibility
Efficiency
Capability
Time
Once this subset has been derived, we reach the set of n possible testing solutions. Now in most delivery models that i have encountered, the focus of the testing group is to find out “a” solution rather than “the” solutions. Especially in the Indian IT scenario testers are highly susceptible to the “tunnel vision syndrome” where the first solutions that rings a bell is pursued.
What do I do about it?
Well for starts here are a few pointers:
To start creating an overall strategy, have the best testers on-board work in silos.
DO NOT rely on the collective intelligence of the team at this stage. That will come in later.
DO NOT interfere with the independent thought process of the the “silo’ed testers”.
Once individual approaches are derived, bring the team to the table.
Battle mode: This is where “collective intelligence” begins. Discuss the various approaches on table.
Survival of the fittest: Darwinian theory will do the rest for you. The best will emerge as none of the “silo approaches” but as the common maximum of the best approaches put forth by each tester.
This is how you can ascertain the “best fit” test approach and NOT pursue the first stray thought.
Testing principle: Excellence is always instilled never inbuilt.
