Most e-business projects entail high pressure and tight timescales coupled with risky project foundation. Typically, products need to be launched, marketing activity ramped up, and subscription numbers increased rapidly to obtain the next round of venture capital. Where the founders of a company are looking to attract investment, or perhaps even sell out, they must also maximize the attractiveness of their assets. Management may prefer to enhance or expand its product offerings before they actually work. The pressure to deliver may override the pressure to get it right. As a result, the testers of modern systems face considerable challenges. They are required to-
- Assess software product risks. Identify and assess, through consultation, the major product risks of concern and propose tests to address those risks.
- Estimate overall test effort. Estimate, based on the nature and scope of the proposed tests and on experience, how expensive and time consuming the testing will be.
- Gain consensus on the amount of testing. Achieve, through consensus, the right coverage, balance, and emphasis of testing.
- Acquire budget to do enough testing. Convince management to allocate enough of its budget to testing that will address the risks of concern.
- Plan, implement and execute tests. Specify and run tests that address the risks of greatest concern.
- Provide information for a risk-based decision on release. Perhaps the most important task of all is to provide information as the major deliverable of all testing.
Risk and Testing: The Correlation?
There are three types of software risk:
- Project risk– These risks relate to the project in its own context. Projects usually have external dependencies, such as the availability of skills, reliance on suppliers, constraints like fixed-price contracts, or fixed deadlines. External dependencies are project management responsibilities.
- Process risk– These risks primarily relate to the project internally its planning, monitoring, and control come under scrutiny. Typical risks here include the underestimation of the project’s complexity or the level of effort or skills required. The internal management of a project, such as good planning, progress monitoring, and control, is a project management responsibility.
- Product risk– These risks relate to the definition of a product, the stability (or lack) of requirements, the complexity of the product and the fault-proneness of the technology, which could lead to a failure to meet requirements. Product risk is the testers’main area of concern.
The purpose of structured test methodologies tailored to the development activities in risk-based testing is to reduce risk by detecting faults in project deliverables as early as possible. Finding faults early, rather than late, in a project reduces the reworking necessary, costs and amount of time lost.
An ideal risk-based testing strategy addresses four key objectives-
- To detect faults in software products (documentation and software) so that they can be corrected;
- To demonstrate and build confidence that stated (and unstated) requirements have been met;
- To provide evidence that the business benefits required from systems are available;
- To provide data on the risk of release (and use) of the system under test.
The preparation of the risk-based test process has five stages:
- Risk identification;
- Risk analysis;
- Risk response;
- Test scoping;
- Test process definition
Stage 1: Risk Identification – An inventory of potential failure modes is prepared. These are derived from existing checklists of failure modes (most commonly) and generic risk lists that can be used to seed the discussions in a risk workshop. Developers, users, technical support staff, and the testers are probably best placed to generate the initial list of failure modes. The tester should compile the inventory of risks from practitionersinput, schedule the risk workshop, and copy the risk inventory to the attendees.
Stage 2: Risk Analysis– At this stage, the risk workshop is convened. This should involve technically qualified representatives from the business, development, technical support, and testing communities. The workshop should involve some more senior managers who are able to see the bigger picture. Ideally, the project manager, development manager, and business manager should be present.
Stage 3: Risk Response– When the candidate risks have been agreed on and the workshop is over, the tester takes each risk in turn and considers whether it is testable. If possible, the tester then specifies a test activity or technique that should meet the test objective. Typical techniques include requirements or design reviews, inspections or static analysis of code or components, or integration, system or acceptance tests.
Stage 4: Test Scoping– Scoping the test process is the review activity that requires the involvement of all of stakeholders and staff with technical knowledge. At this point, the major decisions about what is in and out of scope for testing are made; it is, therefore, essential that the staff in the meeting have the authority to make these decisions on behalf of the business, the project management and technical support.
Stage 5: Test Process Definition– At this point, the scope of the testing has been agreed on, with test objectives, responsibilities and stages in the overall project plan decided. It is now possible to compile the test objectives, assumptions, dependencies and estimates for each test stage and publish a definition for each stage in the test process.
A major benefit of risk-based testing is that it gives a big incentive to let you conduct the risk assessment at the initial stage. It’s an ideal choice for top-level management to assess risk at early stages.