THE SCOPE & DEPTH OF TESTING: TIPS

Now that you’ve decided to outsource work to a testing service organization, how do you accurately describe the scope of testing you need? Before you sign the contract, it’s critical that the anticipated charges match the reality of your testing needs for scope, depth and coverage. How do you accurately describe your testing needs and avoid additional charges? After all, the intention is to save money while getting additional testing completed and improving application quality.

It’s important for the business relationship to submit an accurate description of precisely what testing you want and far you want testing to go. If you don’t describe the scope of the desired testing effectively, you run the risk of generating additional charges for work not previously disclosed. Be certain you know and describe what you want to pay for and how much testing you expect in return.

In order to prepare an accurate evaluation of test scope and depth, pull together a small group of development team members including members from both Development and QA. It doesn’t matter if the role is Director, Manager, Lead, or individual contributor - simply pick the people who understand what is currently tested and how, in detail - preferably the people doing the actual testing work and creating the code.

The more specific you describe the testing effort, the more accurate the valuation, scheduling, and resource implementation. When planning testing scope, keep the following in mind and determine both the plan and result for each:

• Test environment support (if off normal business hours).
• Precisely define what type of testing is functional and how much test coverage you want.

For example, smoke testing is generally defined as a superficial testing ensuring install and page flow. However, some testing groups define their functional regression testing as smoke testing. In the same manner, integration testing can be as simple as verifying API connections or it can involves data verification workflows across a complex system with 20 or more connection applications.

A second option is organizing your testing by priority and risk. Defining the risk of each application area helps define the type and amount of test coverage you need. Start a risk analysis by documenting the areas of your application and listing them in order of execution priority.

• Select which test scripts or test suites are being executed and the number of tests you need. One key to success in providing test cases or scripts is understanding the test documentation is a living document similar to requirements. It’s important to make sure the testing service is given the latest updated version of valid tests.

For a more organized testing effort, include a Test Strategy that outlines exactly which test cases to execute in what order.

If you don’t have existing test cases, perhaps send the online Help menu documentation and have the team test using it. Again, verify the documentation is the latest version. The testing service team tests to verify the application works as described in the Help.

No documentation? Consider creating user personas and writing in depth user stories that describe the functionality and the business intent. You’ll need a user story for each area or workflow within the application.

Keep in mind the less precise the documentation you supply, the more likely you’ll incur additional costs. Why? Because ambiguity is expensive. Not supplying accurate test documentation results in more questions, issues, false defects, and an increased need to to keep the team engaged and productive.

Implement and confirm the team is allowed access to the network as well as testing tools if any. Plan for support in case the test environment goes down during hours when the internal team is not available. You don’t want to pay them not to be able to work. Nearly everyone prefers working productively.

Finally, verify your Test Strategy document or contract, define your testing needs scope and depth specifically, so the team knows where they are in the process and when testing is considered complete.