Jenkins - Automated Testing

Jenkins - Automated Testing Strategies

Automated testing is a cornerstone of Continuous Integration. Jenkins excels at integrating various types of automated tests into the build pipeline, providing fast feedback on code quality.

The goal is to objectively verify build quality at different stages. Jenkins doesn’t perform testing itself but orchestrates the execution of test tools and frameworks, then collects and presents the results.

Integrating Common Test Types

1. Unit Tests

These test individual components (classes, methods) in isolation. They are typically fast and run frequently.

• Tools: JUnit (Java), TestNG (Java), NUnit (.NET), pytest (Python), Jest (JavaScript), Go testing package.
• Integration: Use build tools (Maven, Gradle, npm, pip, go test) to execute unit tests as part of the build stage.
• Jenkins: Use the build tool integration (e.g., mvn test, gradle test, npm test) within a Pipeline step. Publish results using steps like junit (for JUnit/TestNG XML reports) to display trends and detailed reports in Jenkins. See the ‘Unit Testing Integration’ chapter for details.

2. Integration Tests

These verify interactions between different components or systems (e.g., application code and a database, communication between microservices).

• Tools: Often use the same frameworks as unit tests but require setting up external dependencies (databases, APIs). Tools like Testcontainers can manage dependencies in Docker.
• Integration: Typically run after unit tests pass. May require a dedicated stage in the Pipeline.
• Jenkins: Execute tests via build tools. The junit step can publish results. Ensure the Jenkins agent environment has access to necessary services or use tools like Testcontainers within the build.

3. UI / End-to-End (E2E) Tests

These simulate real user interactions by driving the application through its user interface (web browser, mobile app).

• Tools: Selenium WebDriver, Cypress, Playwright (for web), Appium (for mobile).
• Integration: Usually run after the application is built and potentially deployed to a test environment. Often require a dedicated, more complex stage.
• Jenkins:
• • Execute tests using build tools (e.g., mvn failsafe:integration-test for Selenium tests in Maven, npx cypress run for Cypress).
• • Environment Setup: The Jenkins agent needs access to web browsers (e.g., Chrome, Firefox) and their corresponding drivers (ChromeDriver, GeckoDriver). This can be complex to manage directly.
• • Headless Execution: Run browsers in headless mode on the agent to avoid needing a display.
• • Docker Solution: Run browsers inside Docker containers (e.g., Selenium Grid Docker images) for easier environment management.
• • Cloud Testing Services: Integrate with services like Sauce Labs, BrowserStack, or LambdaTest to run tests on various browsers/OS combinations in the cloud. These services often have Jenkins plugins.
• • Publish Results: Use the junit step if your framework produces compatible XML reports. Publish HTML reports (e.g., from Cypress) using the publishHTML step.

Note: The old approach using the ‘Hudson Selenium Plugin’ to run HTML suites directly is outdated. Modern practice uses WebDriver bindings within standard test frameworks.

Example Pipeline Stage for Selenium Tests (using Maven)

This assumes you have Selenium WebDriver tests written using JUnit/TestNG and configured to run via the Maven Failsafe plugin.

stage('E2E Tests') {
    // Agent needs access to browser drivers or use a Docker agent with browsers
    agent { label 'selenium-agent' } // Or use a Docker agent
    steps {
        echo 'Running Selenium E2E Tests...'
        // Assumes application is deployed or accessible at WEBAPP_URL
        sh 'mvn verify -Dwebdriver.base.url=$WEBAPP_URL'
    }
    post {
        always {
            // Publish test results regardless of outcome
            junit '**/target/failsafe-reports/*.xml'
            // Optionally publish HTML reports if generated
            // publishHTML(target: [allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: 'target/reports', reportFiles: 'my-report.html', reportName: 'E2E Test Report'])
        }
    }
}

Best Practices

• Run tests appropriate for the stage: Fast unit tests early, slower E2E tests later.
• Fail fast: Stop the pipeline immediately if critical tests fail.
• Isolate test environments: Use Docker or dedicated agents to prevent test interference.
• Report effectively: Use plugins (junit, publishHTML, Warnings NG for static analysis) to make results visible and track trends.
• Manage test data: Ensure tests have reliable and consistent data.
• Parallelize tests: Speed up execution by running tests in parallel where possible (often handled by the test framework or build tool).

Consult the documentation for your specific testing frameworks and build tools for the best ways to execute them and generate reports compatible with Jenkins.