Software Testing - Overview

Software Testing - An Overview

What is Software Testing?

Software testing is the process of evaluating a software application or system to determine if it meets specified requirements and to identify any defects (gaps, errors, or missing functionalities). It involves executing the software under controlled conditions to verify its behavior, performance, security, and other quality attributes. The ultimate goal is to provide stakeholders with information about the quality of the product and the risks of releasing it.

According to the ANSI/IEEE 1059 standard, testing is defined as: “A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item.” Modern testing also emphasizes preventing defects by involving testers early in the lifecycle.

Who Performs Testing?

Testing is not solely the responsibility of a dedicated test team. In modern software development, quality is a shared responsibility. Various roles contribute to testing:

• Software Developers: Perform unit testing and often contribute to integration and component testing. In DevOps cultures, developers take more ownership of testing their code.
• Software Testers / QA Engineers: Design and execute tests at various levels (integration, system, acceptance), develop test plans, report defects, and often manage test automation.
• Software Development Engineers in Test (SDETs): Specialized engineers who combine development and testing skills, often focusing on building test automation frameworks and tools.
• Test Leads / Test Managers: Plan, manage, and coordinate testing efforts, define test strategies, and report on quality.
• Product Owners / Business Analysts: Participate in acceptance testing, ensuring the software meets business needs and user requirements.
• End Users: Involved in User Acceptance Testing (UAT), beta testing, and providing real-world feedback.

The ‘whole team approach’ to quality is prevalent in Agile methodologies, where everyone contributes to delivering a high-quality product.

When to Start Testing?

The principle of ‘Shift Left’ advocates starting testing activities as early as possible in the Software Development Life Cycle (SDLC). Early testing helps detect defects when they are easier and cheaper to fix. Testing activities can occur in every phase:

• Requirements Phase: Reviewing and analyzing requirements for clarity, completeness, and testability.
• Design Phase: Reviewing architectural and detailed designs to identify potential flaws.
• Development (Coding) Phase: Developers perform unit tests. Static analysis tools can check code quality.
• Testing Phase (Dedicated): Formal execution of integration, system, and acceptance tests.
• Maintenance Phase: Regression testing after modifications or bug fixes.

In iterative models like Agile, testing is a continuous activity performed in each iteration or sprint, ensuring constant feedback and quality.

When to Stop Testing?

Deciding when to stop testing is a critical and often challenging decision, as exhaustive testing is usually impossible. The decision is typically based on a combination of factors:

• Project Deadlines and Budget Constraints: Practical limitations on time and resources.
• Test Case Completion: Execution of all planned test cases or achieving a predefined percentage.
• Coverage Goals: Reaching target levels for requirements coverage, code coverage, or risk coverage.
• Defect Metrics: When the rate of finding new defects falls below a certain threshold, and no critical or high-priority defects remain open.
• Risk Assessment: When the residual risk of releasing the software is deemed acceptable by stakeholders.
• Business Value: When the cost of further testing outweighs the potential benefits or the risk of undiscovered defects.
• Management Decision: Ultimately, a decision based on all available information and project context.

Verification vs. Validation

Verification and Validation (V&V) are two fundamental concepts in quality management, often confused but distinct:

Aspect	Verification	Validation
Question Answered	”Are we building the product right?"	"Are we building the right product?”
Focus	Checks if the software conforms to its design and specifications. Ensures internal correctness.	Checks if the software meets the user’s needs and intended use. Ensures fitness for purpose.
Timing	Typically performed throughout the development process, often before validation.	Typically performed towards the end of the development process, on the built product.
Activities	Static activities like reviews, inspections, walkthroughs of documents (requirements, design, code). Also includes some dynamic checks like unit tests verifying specific logic.	Dynamic activities like system testing, acceptance testing, executing the software to see if it works as users expect in their environment.
Example	Reviewing a design document to ensure it meets architectural standards. A unit test checking if a function correctly calculates a value as per its specification.	User Acceptance Testing where end-users perform real tasks with the software. System testing to ensure all features work together as intended for a user journey.
Nature	More objective, checking against documented standards and specifications.	More subjective, evaluating if the product is fit for its intended purpose from the user’s viewpoint.

Guiding Principles: The Test Pyramid and Continuous Testing

The Test Pyramid is a widely adopted model illustrating a recommended proportion of tests at different levels. It suggests having a large base of fast, inexpensive Unit Tests, a smaller layer of Integration Tests, and an even smaller layer of slower, more expensive End-to-End (UI) Tests. This structure promotes faster feedback and more stable test suites. For further information, search for ‘Mike Cohn Test Pyramid’.

Continuous Testing is a practice in DevOps where testing occurs continuously throughout the software delivery pipeline. Automated tests are integrated into CI/CD processes, providing rapid feedback on every change, enabling faster and more reliable releases.