Ex - Vice President, RBS
Sanjay is having 19+ years of experience in the areas of Project / Program / Delivery Management ... more>>
The tester has to find bugs without any proper planning and documentation solely based on just his intuition. If carried out by a skilled tester, it can often find problems that are not caught in regular testing cycle. Sometimes, if testing occurs very late in the development cycle, this will be the only kind of testing that can be performed.
When I was a tester, I was told to focus on the functionality assigned to me and make sure that customer should not find any issues while using the functionality, the term called "FUNCTIONALITY OWNER". But I wanted to do something different as I got bore to perform the same activities for every product testing along with their release cycle, so I started doing testing of the entire product instead of testing only those functionality assigned to me just after 2 rounds of System Integration Testing using my knowledge and without using any test cases. The result was excellent. I found good issues which were with the product from the very beginning. After analyzing those issues it was found that those information was missing in the requirement document.
Based upon my findings, I approached to my Test Manager along with my test lead and try to convince him that if the entire testing team will perform one round of Ad-hoc testing just after finishing the System Integration Testing cycle then there is a chance to get more issue which we might miss during the regular testing cycle. It was very hard to convince him as in the Testing Approach / Strategy document; no information about the Ad-hoc testing cycle was mentioned. We were already completed 2 rounds of System Integration Testing cycle and waiting for the new release / build to perform regression testing followed by Acceptance Testing by Product Management team. Test Manager also wanted to have one round of Ad-hoc testing based upon the issue found by me so he has taken those issues to the Steering Committee meeting and tried to convince the entire stake holders and requested to include one week on Ad-hoc testing. Somehow we got the approval from Steering Committee to include one round of Ad-hoc testing and in this way, we the entire testing team has been asked to perform one round of Ad-hoc testing based upon our testing skills, knowledge of the product and intuition only. After one week of Ad-hoc testing, the result was:
1. 20% Defects (Showstoppers and High priority) of SIT Cycle 1 and 2 has been found.
2. 10% Defects (Medium priority) of SIT Cycle 1 and 2 has been found.
One of the best uses of ad-hoc testing is for discovery. Reading the requirements / specifications Use cases rarely gives you a good sense of how a program actually behaves. Even the user documentation may not capture the "look and feel" of a program. Ad-hoc testing can find holes in your test strategy, and can expose relationships between subsystems that would otherwise not be apparent. In this way, it serves as a tool for checking the completeness of your testing. Missing cases can be found and added to your testing arsenal. Finding new tests in this way can also be a sign that you should perform root cause analysis.
Finding new tests in this way can also be a sign that you should perform root cause analysis. Ask yourself or your test team, "What other tests of this class should we be running?" Defects found while doing ad-hoc testing are often examples of entire classes of forgotten test cases. Another use for ad-hoc testing is to determine the priorities for your other testing activities.
It has also been found that ad-hoc testing can also be used effectively to increase your code coverage. Adding new tests to formal test designs often requires a lot of effort in producing the designs, implementing the tests, and finally determining the improved coverage. A more streamlined approach involves using iterative ad-hoc tests to determine quickly if you are adding to your coverage. If it adds the coverage you're seeking, then you'll probably want to add these scenarios to your existing test cases.
A good ad-hoc tester (Ad-hoc testing can be performed effectively and efficiently by the experienced testers) needs to understand the design goals and requirements for these low-level functions. What choices did the development team make, and what were the weaknesses of those choices? As a tester, we are less concerned with the correct choices made during development. Ad-hoc testing can be done as black box testing. However, this means you must check for all the major design patterns that might have been used. This allows you to narrow the testing to many fewer cases.
An important element of any IT strategy is to ensure deployment of defect free systems. Among other benefits, ad-hoc testing helps minimize significantly the total cost of ownership (TCO) of applications. However, organizations quickly discover that despite their best intentions and efforts, their QA team has hit a ceiling from a defect leakage standpoint. It seems as if an invisible hurdle is preventing the QA team from achieving its true potential - deploying defect free systems.
Now a days ad-hoc testing finds a place during the entire software testing cycle. Early in the project, ad-hoc testing provides breadth to testers understanding of the application and helping them more effective and quality test cases, thus aiding in discovery. In the middle of a project, the data obtained helps set priorities and release schedules of the software program / Application. As a project nears the ship date / acceptance testing by business users / product management team, just after completion all of your testing cycles as mentioned in your Test Approach / Strategy document, ad-hoc testing can be used to examine the quality of the application.
A primary goal of ad-hoc testing is to uncover new defects in the product / specification. In the hands of a skilled tester, it can be highly effective at discovering such problems. Regression tests and ad-hoc tests complement each other in remarkable ways. When you find a defect with a regression test, you'll often need to use ad-hoc methods to analyze and isolate the defect, and to narrow down the steps to reproduce it. You may also want to explore "around" the defect to determine if there are related problems. On the other hand, when you find a defect with an ad-hoc test, you'll probably document it in your defect tracking system, thereby turning it into a regression test, for verification after the defect is fixed.
The major benefits of ad-hoc testing are:
1. No planning and documentation needed. It can be added successfully during beginning of the project, mid or before release of the product.
2. The important bugs are found which helps you in discovering missing test cases - can find holes in your original testing strategy that can be incorporated later.
3. Gives more understanding of how a software application or feature behaves which may not be known by reviewing specification documents or use cases.
4. Gives better understanding of the testing priorities, for example if an ad-hoc test runs very well, you can decide testing in that area can be deferred to the later stage.
5. Easy to start and Implement.
6. It helps to save a lot of precious time. Sometimes its after spending valuable time on preparing and explaining tests that the programs design is changed. With Ad-hoc testing precious time is not wasted on planning and documents.
The success of Ad-hoc testing depends upon the capability of the tester who carries out the test. The tester has to find bugs without any proper planning and documentation solely based on just his / her intuition.
In my testing carrier, wherever I work, I have always given importance of ad-hoc always put a round of Ad-hoc testing just after completion of System Integration Testing cycle if the new version of product is getting tested. I have also used the same before the regular testing cycle for a brand new product (first version of product) followed by performing the same before sending to product to either release or sending for the acceptance testing by product management team just after finishing the regular testing cycle. I always mentioned 2 type of testing in my Test Strategy / Approach document which are Formal and Less Formal Test like:
Last but not the least, if you are planning to establish TMMi framework within your organization, then you must implement a round of ad-hoc testing during PA (process area) 2.1 of Level 2 which is "Test Policy and Strategy" and PA 2.2 which is "Test Planning". I am sure if you implement the ad-hoc testing during Level 2 you will easily achieve Level 3 - "Defined" to Level - 5 "Optimization".
Experts on QA
Test Automation Lead
NBN Co Limited