- provide nice, compact overview
- include screenshot of a moment of failure
Fortunately I found ReportNG after that. The default design might not be so fancy but it is still very good and it fits well into TestNG and our tests.
First we had to add necessary dependencies to our maven POM:
<dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.8.8</version> </dependency> <dependency> <groupId>org.uncommons</groupId> <artifactId>reportng</artifactId> <version>1.1.4</version> <exclusions> <exclusion> <groupId>org.testng</groupId> <artifactId>testng</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>com.google.inject</groupId> <artifactId>guice</artifactId> <version>3.0</version> </dependency>
TestNG has several interfaces to hook into the test processing, the most interesting probably are ITestListener, IConfigurationListener, and sometimes IMethodInterceptor. ReportNG add class HTMLReporter to that.
To add a screenshot to the report, we need to save it ITestListener.onContextFailure() and pick it up
in a custom ReportNGUtils -- for customization we need to provide custom Velocity context by overriding createContext() and passing custom ReportNGUtils implementation.
import org.apache.commons.io.FileUtils; import org.apache.velocity.VelocityContext; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; import org.testng.IConfigurationListener; import org.testng.ITestContext; import org.testng.ITestListener; import org.testng.ITestResult; import org.uncommons.reportng.HTMLReporter; import java.io.File; import java.net.URL; public class TestListener extends HTMLReporter implements ITestListener, IConfigurationListener { protected static final CustomReportNgUtils REPORT_NG_UTILS = new CustomReportNgUtils(); @Override
protected VelocityContext createContext() { VelocityContext context = super.createContext(); // VelocityContext has three properties: meta, utils, messages // - see AbstractReporter.createContext() context.put("utils", REPORT_NG_UTILS); return context; } /** Invoked when test method (method with annotation @Test) fails. */ @Override public void onTestFailure(ITestResult testResult) { if (getWebDriver(testResult) != null) { File scrFile = ((TakesScreenshot) getWebDriver(testResult)) .getScreenshotAs(OutputType.FILE); String screenshotName = createScreenshotName(testResult); File targetFile = new File(screenshotName); FileUtils.copyFile(scrFile, targetFile);
URL scrUrl = new URL(getDriver(testResult).getCurrentUrl());
Screenshot screenshot = new Screenshot(targetFile, srcUrl ); testResult.setAttribute(Screenshot.KEY, screenshot); } } // ... }
Class Screenshot is a custom class holding screenshot-related data, bare bones version could looke as this:
class Screenshot { /* Name of {@link ITestResult} attribute for Screenshot. */ static final String KEY = "screenshot"; /** File in which is the screenshot stored. */ File file; /** URL of a web application's page the screenshot captures. */ URL url; }
Now we need to add the custom ReportNGUtils implementation which picks up contextual information (Screenshot instance in our case) and uses it to modify the report output.
import java.util.List; import org.testng.ITestResult; import org.uncommons.reportng.ReportNGUtils; class CustomReportNgUtils extends ReportNGUtils { public List<String> getTestOutput(ITestResult testResult) { List<String> output = super.getTestOutput(testResult); if ( testResult.getAttribute(Screenshot.KEY) != null ) { Screenshot screenshot = (Screenshot) testResult.getAttribute(Screenshot.KEY); String screenshotFileName = screenshot.getFile().getName(); if (screenshot != null) { String url = (String) testResult.getAttribute("screenshotUrl"); output.add(String.format("screenshot for %s %s <br/><img src='../screenshots/%s'>", testResult.getName(), url, screenshotFileName) ); } } return output; } }
The final step is to register test listener in the plugin executing the tests. We use failsafe, the configuration for surefire is similar if you desire to use it.
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <configuration> <systemPropertyVariables> <org.uncommons.reportng.escape-output>
false
</org.uncommons.reportng.escape-output> </systemPropertyVariables> <summaryFile>${project.build.directory}/failsafe-reports/failsafe-summary.xml</summaryFile>
<testClassesDirectory>${project.build.directory}/classes</testClassesDirectory> <properties> <property> <name>usedefaultlisteners</name> <value>false</value> </property> <property> <name>listener</name> <value> org.bithill.test.testng.TestListener </value> </property> </properties> <suiteXmlFiles> <suiteXmlFile>src/main/resources/suiteX.xml</suiteXmlFile> </suiteXmlFiles> </configuration> <executions> <execution> <id>integration-test</id>
<phase>integration-test</phase>
<goals> <goal>integration-test</goal> </goals> </execution> <execution> <id>verify</id>
<phase>verify</phase>
<goals> <goal>verify</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
And that's all - when you run 'mvn failsafe:integration-test' the tests are run, then you follow with 'mvn failsafe:verify', which processes the results of the integration tests and generates a report and sets proper build result. Note that set the testClassesDirectory si crucial if you have it different than the expected 'test-classes'.
