Software Testing - Types of Testing

Software Testing - Execution Methods: Manual and Automation

This section focuses on the primary methods of executing software tests: Manual Testing and Automation Testing. While other chapters discuss what to test (e.g., functionality, performance - covered in ‘Levels and Types’) or how to approach test design (e.g., black-box, white-box - covered in ‘Core Methods’), this chapter explores how those tests are actually carried out.

Manual Testing

Manual testing involves a human tester executing test cases without the use of any automation tools or scripts. The tester plays the role of an end-user, interacting with the application, providing inputs, and verifying outputs against expected results. This method is used across various testing levels, including unit (by developers), integration, system, and user acceptance testing.

Testers typically follow documented test plans and test cases but also employ techniques like Exploratory Testing. Exploratory testing is a valuable manual approach where the tester’s learning, test design, and test execution are concurrent activities. It relies on the tester’s skill, intuition, and experience to discover defects that scripted tests might miss. Manual testing is essential for usability testing, ad-hoc testing, and scenarios requiring human judgment and observation.

Automation Testing

Automation testing, or Test Automation, uses specialized software tools to execute pre-scripted tests on an application. This involves writing code (test scripts) that automatically performs test actions, compares actual results with expected results, and reports the outcomes. The primary goal is to run tests more quickly, efficiently, and repeatedly than is possible with manual testing.

Conceptually, test automation involves a test tool interacting with the Application Under Test (AUT). The tool drives the AUT through its interfaces (UI or API), performs actions defined in the test script, captures results, and compares them against predefined assertions or expected outcomes. This allows for unattended execution of large test suites.

Key benefits include increased test coverage, improved accuracy (by reducing human error), faster feedback cycles (especially in CI/CD), and cost savings in the long run for repetitive tests like regression testing. It’s also indispensable for performance, load, and stress testing.

What to Automate?

Not all tests are suitable or cost-effective to automate. Good candidates for automation include:

• Repetitive tests: Such as regression test suites that are run frequently.
• High-risk, critical functionalities: Ensuring core features always work.
• Data-driven tests: Tests that need to be run with multiple data sets (e.g., checking a login form with many valid/invalid credentials).
• Tests requiring precision and consistency: Where human error is a risk.
• Tests across multiple platforms/browsers/devices.
• API tests: Often more stable and faster to automate than UI tests.
• Performance, load, and stress tests.
• Stable functionalities: Requirements that are not expected to change frequently.

The Test Automation Pyramid (emphasizing more automation at unit and API levels, less at the UI level) is a good guideline for prioritizing automation efforts.

When to Automate?

Consider test automation when:

• The project is large, complex, or long-term, justifying the initial investment.
• The same test scenarios need to be executed repeatedly.
• The application is relatively stable; automating highly volatile features can lead to high maintenance.
• There’s a need for rapid feedback in a CI/CD pipeline.
• Manual testing becomes a bottleneck for frequent releases.
• Return on Investment (ROI) is clear: benefits outweigh the costs of development and maintenance of automation scripts.

How to Automate?

A typical test automation process involves:

• Defining the Scope of Automation: Identifying suitable test cases.
• Selecting the Right Automation Tool(s) and Framework: Based on application technology, team skills, and budget.
• Developing Test Scripts: Writing code in languages like Python, Java, JavaScript, C# using chosen tools/frameworks.
• Designing a Test Automation Framework: Creating a structured environment for efficient script development, execution, and reporting (e.g., Page Object Model, Keyword-Driven, BDD).
• Executing Test Scripts: Running automated tests, often integrated into CI/CD pipelines.
• Analyzing Results and Reporting: Generating reports, identifying failures, and logging defects.
• Maintaining Test Scripts: Updating scripts as the application evolves.

Common Software Testing Tools & Frameworks

The landscape of testing tools is vast and constantly evolving. Here are some popular examples categorized by their primary use:

UI Automation:

• Selenium: Open-source, supports multiple browsers and languages (Java, Python, C#, JavaScript). A de-facto standard.
• Cypress: JavaScript-based, modern end-to-end testing framework, known for speed and developer-friendliness.
• Playwright (Microsoft): Node.js library for browser automation (Chromium, Firefox, WebKit), supports multiple languages.
• Appium: Open-source, for automating mobile native, hybrid, and web applications (iOS, Android).
• Micro Focus UFT (formerly QTP): Commercial tool, supports a wide range of technologies.

API Testing:

• Postman: Popular tool for designing, building, testing, and documenting APIs.
• Rest Assured (Java): Java library for testing RESTful APIs.
• Karate DSL: Open-source test automation framework built on top of Cucumber, specialized for API testing.
• SoapUI: Widely used for testing SOAP web services and REST APIs.

Performance Testing:

• Apache JMeter: Open-source, Java-based load testing tool.
• K6 (Grafana Labs): Modern load testing tool for developers and testers, scriptable in JavaScript.
• Gatling: Open-source, Scala-based load testing tool, known for high performance.
• Micro Focus LoadRunner: Commercial, comprehensive performance testing solution.

Unit Testing Frameworks (often used by developers):

• JUnit (Java), TestNG (Java)
• NUnit (.NET), MSTest (.NET)
• PyTest (Python), unittest (Python)
• Jest (JavaScript), Mocha (JavaScript)

Test Management Tools:

• Jira (with plugins like Xray, Zephyr Scale)
• TestRail
• qTest
• Azure Test Plans (part of Azure DevOps)

Choosing the right tools depends on project needs, team expertise, application technology, and budget. Often, a combination of tools is used to achieve comprehensive test coverage.