Software Quality Metrics

Software metrics are quantitative measures used to understand, manage, and improve software products, processes, and projects. Selecting the right metrics, aligned with specific goals (recall GQM), is crucial for driving meaningful improvements.

Metrics can be broadly categorized:

Product metrics: Describe characteristics of the software itself (e.g., size, complexity, reliability, performance).
Process metrics: Measure characteristics of the development, testing, and maintenance processes (e.g., efficiency, effectiveness, adherence to standards).
Project metrics: Describe project attributes and execution (e.g., cost, schedule, effort, team productivity).

Software quality metrics specifically focus on aspects related to the quality attributes of the product and the effectiveness of the processes in achieving that quality. They often overlap between Product and Process categories.

Key categories of software quality metrics include:

Code Quality Metrics
Testing Quality Metrics
Process Efficiency & Flow Metrics
Operational & Reliability Metrics
Customer Satisfaction Metrics
Maintenance Quality Metrics

Code Quality Metrics

These metrics assess the internal quality of the source code, often acting as leading indicators for maintainability, reliability, and security.

Cyclomatic Complexity: Measures the complexity of control flow. High complexity often correlates with higher defect rates and difficulty in testing/understanding.
Cognitive Complexity: Aims to measure how difficult code is for a human to understand.
Code Duplication: Percentage of code that is duplicated. High duplication increases maintenance effort and risk.
Maintainability Index: A composite score (often 0-100) calculated from metrics like complexity, LOC, and Halstead volume (though specific formulas vary). Higher scores indicate better maintainability.
Code Churn: Frequency of changes to a piece of code. High churn can indicate instability or problematic areas.
Static Analysis Violations: Counts of issues found by static analysis tools (e.g., potential bugs, security vulnerabilities, style violations). Tracking trends is important.

Testing Quality Metrics

These metrics evaluate the effectiveness and efficiency of testing activities.

Test Coverage: Percentage of code (lines, branches, methods) executed by automated tests (Unit, Integration). Provides insight into test suite thoroughness but doesn’t guarantee test quality.
Defect Density (During Testing): Number of defects found per unit of size (KLOC, Function Points) during formal testing phases. Useful for comparing releases or components.
Defect Detection Percentage (DDP) / Phase Containment: Percentage of defects found in a specific phase (e.g., Unit Testing) compared to the total defects found for that code later.
Test Execution Status: Number/percentage of planned tests executed, passed, failed, blocked.
Automated vs. Manual Test Effort: Ratio of effort spent on automated vs. manual testing activities.

Process Efficiency & Flow Metrics

These metrics, often associated with Agile and DevOps, measure the speed and efficiency of the development and delivery process.

Lead Time: Time from a requirement being identified/accepted to its delivery to production.
Cycle Time: Time from starting work on an item to its completion (e.g., ‘In Progress’ to ‘Done’).
Deployment Frequency: How often code is successfully deployed to production. Higher frequency often indicates a more mature process.
Change Failure Rate: Percentage of deployments that result in a production failure requiring remediation (e.g., rollback, hotfix). A key DORA metric.
Mean Time To Recover (MTTR): Average time taken to restore service after a production failure. Another key DORA metric.

Operational & Reliability Metrics

These metrics measure the quality of the software once it is running in production.

Mean Time Between Failures (MTBF): Average time between system failures. Primarily used for hardware but conceptually applied to critical software systems.
Mean Time To Failure (MTTF): Average time a non-repairable system (or component) operates before failing.
Availability: Percentage of time the system is operational and available to users (e.g., 99.9%, ‘three nines’).
Application Performance Metrics: Response times, throughput, error rates, resource utilization (CPU, memory) under various load conditions.
Production Defect Density: Number of defects reported by users post-release, often normalized by user base size or usage duration.

Customer Satisfaction Metrics

These metrics gauge how users perceive the quality and value of the software.

Customer Satisfaction Score (CSAT): Measured via surveys, often using a scale (e.g., 1-5, Very Dissatisfied to Very Satisfied). Metrics derived include ’% Satisfied’, ’% Dissatisfied’.
Net Promoter Score (NPS): Measures customer loyalty based on the question ‘How likely are you to recommend this product/service?’.
Customer Effort Score (CES): Measures how easy customers find it to interact with the product/service.
Task Success Rate: Percentage of users who successfully complete a specific task using the software (often measured in usability testing).
Customer Support Tickets: Volume and severity of issues reported to support channels.

Customer Problems Metric (PUM - Historical Context): This measured total problems (defects and non-defects) per user-month. Formula:

PUM = Total Customer Reported Problems / Total License Months

While the concept of tracking customer issues is vital, PUM itself is less commonly used now compared to metrics like CSAT, NPS, and raw counts/severity of support tickets.

Maintenance Quality Metrics

During the maintenance phase, focus shifts to efficiently and effectively addressing reported issues.

Fix Backlog: Number of open/unresolved reported defects, often tracked over time.
Backlog Management Index (BMI): Ratio of closed issues to newly reported issues in a period. $$BMI = (Number : of : problems : closed / Number : of : problems : arrived) \times 100%$$ A BMI > 100 indicates the backlog is shrinking.
Mean Time to Resolution (MTTR - for defects): Average time taken to fix a defect from reporting to deployment of the fix.
Fix Response Time: Average time taken from reporting to initial response or acknowledgement.
Percent Delinquent Fixes: Percentage of fixes that exceed agreed-upon resolution times (often based on severity). $$Percent :Delinquent = (Number : of : overdue : fixes / Total : fixes : delivered) \times 100%$$
Fix Quality / Bad Fix Rate: Percentage of fixes that are incorrect (don’t solve the problem) or introduce new regressions. Aim should be close to 0%.

Effective quality management relies on selecting a balanced set of metrics across these categories, visualizing them (e.g., dashboards), and using them to drive targeted improvements.