Testing - Estimation Techniques

Software Testing - Estimation Techniques

Estimating the effort required for software testing is a critical task in project planning. Accurate estimates help in resource allocation, timeline management, and budget control, ultimately contributing to effective test coverage and software quality. This section describes various techniques for test estimation, ranging from traditional methods to modern agile approaches.

It’s important to remember that estimation is not an exact science and should be revisited and refined throughout the project lifecycle.

Traditional Estimation Techniques

Function Point Analysis (FPA)

Function Point Analysis is a structured technique to measure software size based on its functionality from the user’s perspective. It categorizes system functions into types such as:

• External Inputs (EIs): Data entering the system.
• External Outputs (EOs): Data leaving the system (reports, screens).
• External Inquiries (EQs): User requests for specific information.
• Internal Logical Files (ILFs): Data maintained within the system.
• External Interface Files (EIFs): Data referenced by the system but maintained elsewhere.

Each function type is assigned a complexity rating (low, average, high) and weighted. The sum of weighted function points gives a measure of software size, which can then be used with historical productivity data to estimate testing effort. IFPUG (International Function Point Users Group) provides standardized guidelines for FPA.

Test Point Analysis (TPA)

Test Point Analysis is an extension of FPA, specifically tailored for estimating black-box or acceptance testing effort. It considers elements like the size (derived from function points), complexity of functions, system interfaces, uniformity of functions, and the chosen test strategy. Dynamic points related to test levels and types are also factored in to calculate test points, which then translate to effort.

Mark-II Method

The Mark-II (or MkII) FPA method is another variation of function point analysis that also estimates software size from a functional perspective. The procedure involves:

• Defining the estimation viewpoint, purpose, and scope (boundary).
• Identifying logical transactions (processes triggered by an event).
• Identifying and categorizing data entity types involved in transactions.
• Counting input data element types for each transaction.
• Calculating the functional size based on these components.

Like FPA, the resulting size estimate is used with productivity factors to derive effort.

Experience-Based and Other Techniques

Analogy-Based Estimation

This technique involves estimating effort based on the actual effort from similar past projects or tasks. The accuracy depends on the similarity between the current and past projects and the availability of reliable historical data. It requires careful judgment to adjust for differences.

Expert Judgment (e.g., Delphi Technique)

Expert judgment relies on the intuition and experience of one or more experts. The Delphi Technique is a more structured form where multiple experts provide estimates anonymously. A facilitator collates these, shares a summary, and experts then revise their estimates. This iterative process continues until a consensus or stable range is reached, reducing individual bias.

Test Case Enumeration Based Estimation

This method estimates the total number of test cases to be designed and executed. Then, an average effort per test case (for design, execution, reporting) is applied, based on historical data or expert judgment, to calculate the total testing effort.

Work Breakdown Structure (WBS) / Task-Based Estimation

The testing project is broken down into smaller, manageable tasks (e.g., test planning, test design, test environment setup, test execution per module, reporting). Effort is then estimated for each individual task, and these estimates are summed up to get the total project effort. This is a bottom-up approach.

Three-Point Estimation (PERT-like)

For each task, three estimates are provided: Optimistic (O), Most Likely (M), and Pessimistic (P). The final estimate (E) can be calculated using a weighted average, often E = (O + 4M + P) / 6. This technique acknowledges uncertainty and provides a more realistic range.

Agile Estimation Techniques

Agile methodologies emphasize adaptive planning and collaborative estimation. Testing effort is often estimated as part of overall user story or feature estimation.

Story Points

Story points are a relative unit of measure for estimating the overall effort required to implement a user story, including all development and testing activities. Teams estimate complexity, uncertainty, and volume of work, rather than time directly. The team’s historical ‘velocity’ (story points completed per iteration) helps in forecasting.

Planning Poker

Planning Poker is a consensus-based technique often used for estimating story points. Team members (including testers) privately select a card representing their estimate. All cards are revealed simultaneously, and differences are discussed until the team converges on an estimate. This encourages participation and shared understanding.

T-Shirt Sizing

This is a quick, high-level estimation technique, often used for large backlogs or early-stage planning. Items are categorized into relative sizes like XS, S, M, L, XL. These sizes can later be mapped to story points or ideal days if needed.

Best Practices in Test Estimation

• Involve the whole team (including testers, developers, and product owners) in the estimation process.
• Break down large tasks into smaller, more manageable units.
• Use historical data and metrics from past projects whenever possible.
• Clearly document assumptions made during estimation.
• Account for non-obvious tasks like test environment setup, test data creation, meetings, and defect retesting.
• Revisit and refine estimates regularly as more information becomes available or project scope changes.
• Consider risk and contingency; complex or uncertain tasks may require a buffer.
• Choose estimation techniques appropriate for the project context, methodology, and available information.