TestNG Reporting Customization

We have introduced TestNG default report html and xml files in article TestNG Report Example. But you may find it is not enough for you. Because the original reports generated by TestNG has some inconvenience, it is so basic. TestNG provide IReporter interface which you can implement it to create a test context listener. Then you can customize report in it’s generateReport() method. This example will show you how to customize TestNG report step by step.

Default TestNG Generated Reports

We will customize the default report generated in article TestNG Report Example. Below is the testng.xml and test java files used in that article.

  1. main-suite.xml
  2. suite1.xml
  3. suite2.xml
  4. TestCaseClass1.java
  5. TestCaseClass2.java
  6. TestCaseClass3.java
  7. TestCaseClass4.java
  8. TestCaseClass5.java
  9. TestCaseClass6.java

original testng report example testng.xml and java files

Default TestNG Emailable Html Report

default testng emailable report

Above is the default TestNG emailable report, it has below disadvantages.

  1. Time column is not user friendly.
  2. Do not provide test start time and end time.

We will customize it to below report. It has below improvements.

  1. Add browser, start time and end time for suite summary report.
  2. Add start time, parameter, reporter message and exception message column for test method summary report.
  3. Convert execution time to more human readable format.

customized testng emailable html report

Customize TestNG Report Steps

  1. customize-emailable-report-template.html : This is the template html for customize reports. It contains html layout and three customize TestNG report element place holder.
      <body>
        <table>
          <tr><center><b>$TestNG_Custom_Report_Title$</b></center></tr>
          <thead>
            <tr>
              <th>Test Name</th>
              <th># Total Method</th>
              <th># Passed</th>
              <th># Skipped</th>
              <th># Failed</th>
              <th>Browser</th>
              <th>Start Time</th>
              <th>End Time</th>
              <th>Execute Time (mm:hh:ss:ms)</th>
              <th>Included Groups</th>
              <th>Excluded Groups</th>
            </tr>
           </thead> 
           $Test_Case_Summary$
        </table>
        <table id="summary">
          <thead>
            <tr>
              <th>Class</th>
              <th>Method</th>
              <th>Start Time</th>
              <th>Execution Time (mm:hh:ss:ms)</th>
              <th>Parameter</th>
              <th>Reporter Message</th>
              <th>Exception Message</th>
            </tr>
          </thead> 
          $Test_Case_Detail$
        </table>
      </body>

    customize emailable report template html

  2. main-suite.xml : Add test listener in this TestNG suite xml. We will introduce the test listener later.
    <listeners>
    	<listener class-name="com.dev2qa.example.testng.report.custom.CustomTestNGReporter" />
      </listeners>

    add testng test listener in main-suite.xml

  3. CustomTestNGReporter.java : This is the java class that implement IReporter interface. It just need to override IReporter interface generateReport() method. You can see below code comments for more explanation.
    Package : com.dev2qa.example.testng.report.custom

    public class CustomTestNGReporter implements IReporter {
    	
    	//This is the customize emailabel report template file path.
    	private static final String emailableReportTemplateFile = "C:/WorkSpace/dev2qa.com/TestNGExampleProject/src/com/dev2qa/example/testng/report/custom/customize-emailable-report-template.html";
    	
    	@Override
    	public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites, String outputDirectory) {
    		
    		try
    		{
    			// Get content data in TestNG report template file.
    			String customReportTemplateStr = this.readEmailabelReportTemplate();
    			
    			// Create custom report title.
    			String customReportTitle = this.getCustomReportTitle("Custom TestNG Report");
    			
    			// Create test suite summary data.
    			String customSuiteSummary = this.getTestSuiteSummary(suites);
    			
    			// Create test methods summary data.
    			String customTestMethodSummary = this.getTestMehodSummary(suites);
    			
    			// Replace report title place holder with custom title.
    			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$TestNG_Custom_Report_Title\\$", customReportTitle);
    			
    			// Replace test suite place holder with custom test suite summary.
    			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$Test_Case_Summary\\$", customSuiteSummary);
    			
    			// Replace test methods place holder with custom test method summary.
    			customReportTemplateStr = customReportTemplateStr.replaceAll("\\$Test_Case_Detail\\$", customTestMethodSummary);
    			
    			// Write replaced test report content to custom-emailable-report.html.
    			File targetFile = new File(outputDirectory + "/custom-emailable-report.html");
    			FileWriter fw = new FileWriter(targetFile);
    			fw.write(customReportTemplateStr);
    			fw.flush();
    			fw.close();
    			
    		}catch(Exception ex)
    		{
    			ex.printStackTrace();
    		}
    	}
    	
    	/* Read template content. */
    	private String readEmailabelReportTemplate()
    	{
    		StringBuffer retBuf = new StringBuffer();
    		
    		try {
    		
    			File file = new File(this.emailableReportTemplateFile);
    			FileReader fr = new FileReader(file);
    			BufferedReader br = new BufferedReader(fr);
    			
    			String line = br.readLine();
    			while(line!=null)
    			{
    				retBuf.append(line);
    				line = br.readLine();
    			}
    			
    		} catch (FileNotFoundException ex) {
    			ex.printStackTrace();
    		}finally
    		{
    			return retBuf.toString();
    		}
    	}
    	
    	/* Build custom report title. */
    	private String getCustomReportTitle(String title)
    	{
    		StringBuffer retBuf = new StringBuffer();
    		retBuf.append(title + " " + this.getDateInStringFormat(new Date()));
    		return retBuf.toString();
    	}
    	
    	/* Build test suite summary data. */
    	private String getTestSuiteSummary(List<ISuite> suites)
    	{
    		StringBuffer retBuf = new StringBuffer();
    		
    		try
    		{
    			int totalTestCount = 0;
    			int totalTestPassed = 0;
    			int totalTestFailed = 0;
    			int totalTestSkipped = 0;
    			
    			for(ISuite tempSuite: suites)
    			{
    				retBuf.append("<tr><td colspan=11><center><b>" + tempSuite.getName() + "</b></center></td></tr>");
    				
    				Map<String, ISuiteResult> testResults = tempSuite.getResults();
    				
    				for (ISuiteResult result : testResults.values()) {
    					
    					retBuf.append("<tr>");
    					
    					ITestContext testObj = result.getTestContext();
    					
    					totalTestPassed = testObj.getPassedTests().getAllMethods().size();
    					totalTestSkipped = testObj.getSkippedTests().getAllMethods().size();
    					totalTestFailed = testObj.getFailedTests().getAllMethods().size();
    					
    					totalTestCount = totalTestPassed + totalTestSkipped + totalTestFailed;
    					
    					/* Test name. */
    					retBuf.append("<td>");
    					retBuf.append(testObj.getName());
    					retBuf.append("</td>");
    					
    					/* Total method count. */
    					retBuf.append("<td>");
    					retBuf.append(totalTestCount);
    					retBuf.append("</td>");
    					
    					/* Passed method count. */
    					retBuf.append("<td bgcolor=green>");
    					retBuf.append(totalTestPassed);
    					retBuf.append("</td>");
    					
    					/* Skipped method count. */
    					retBuf.append("<td bgcolor=yellow>");
    					retBuf.append(totalTestSkipped);
    					retBuf.append("</td>");
    					
    					/* Failed method count. */
    					retBuf.append("<td bgcolor=red>");
    					retBuf.append(totalTestFailed);
    					retBuf.append("</td>");
    					
    					/* Get browser type. */
    					String browserType = tempSuite.getParameter("browserType");
    					if(browserType==null || browserType.trim().length()==0)
    					{
    						browserType = "Chrome";
    					}
    					
    					/* Append browser type. */
    					retBuf.append("<td>");
    					retBuf.append(browserType);
    					retBuf.append("</td>");
    					
    					/* Start Date*/
    					Date startDate = testObj.getStartDate();
    					retBuf.append("<td>");
    					retBuf.append(this.getDateInStringFormat(startDate));
    					retBuf.append("</td>");
    					
    					/* End Date*/
    					Date endDate = testObj.getEndDate();
    					retBuf.append("<td>");
    					retBuf.append(this.getDateInStringFormat(endDate));
    					retBuf.append("</td>");
    					
    					/* Execute Time */
    					long deltaTime = endDate.getTime() - startDate.getTime();
    					String deltaTimeStr = this.convertDeltaTimeToString(deltaTime);
    					retBuf.append("<td>");
    					retBuf.append(deltaTimeStr);
    					retBuf.append("</td>");
    					
    					/* Include groups. */
    					retBuf.append("<td>");
    					retBuf.append(this.stringArrayToString(testObj.getIncludedGroups()));
    					retBuf.append("</td>");
    					
    					/* Exclude groups. */
    					retBuf.append("<td>");
    					retBuf.append(this.stringArrayToString(testObj.getExcludedGroups()));
    					retBuf.append("</td>");
    					
    					retBuf.append("</tr>");
    				}
    			}
    		}catch(Exception ex)
    		{
    			ex.printStackTrace();
    		}finally
    		{
    			return retBuf.toString();
    		}
    	}
    
    	/* Get date string format value. */
    	private String getDateInStringFormat(Date date)
    	{
    		StringBuffer retBuf = new StringBuffer();
    		if(date==null)
    		{
    			date = new Date();
    		}
    		DateFormat df = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
    		retBuf.append(df.format(date));
    		return retBuf.toString();
    	}
    	
    	/* Convert long type deltaTime to format hh:mm:ss:mi. */
    	private String convertDeltaTimeToString(long deltaTime)
    	{
    		StringBuffer retBuf = new StringBuffer();
    		
    		long milli = deltaTime;
    		
    		long seconds = deltaTime / 1000;
    		
    		long minutes = seconds / 60;
    		
    		long hours = minutes / 60;
    		
    		retBuf.append(hours + ":" + minutes + ":" + seconds + ":" + milli);
    		
    		return retBuf.toString();
    	}
    	
    	/* Get test method summary info. */
    	private String getTestMehodSummary(List<ISuite> suites)
    	{
    		StringBuffer retBuf = new StringBuffer();
    		
    		try
    		{
    			for(ISuite tempSuite: suites)
    			{
    				retBuf.append("<tr><td colspan=7><center><b>" + tempSuite.getName() + "</b></center></td></tr>");
    				
    				Map<String, ISuiteResult> testResults = tempSuite.getResults();
    				
    				for (ISuiteResult result : testResults.values()) {
    					
    					ITestContext testObj = result.getTestContext();
    
    					String testName = testObj.getName();
    					
    					/* Get failed test method related data. */
    					IResultMap testFailedResult = testObj.getFailedTests();
    					String failedTestMethodInfo = this.getTestMethodReport(testName, testFailedResult, false, false);
    					retBuf.append(failedTestMethodInfo);
    					
    					/* Get skipped test method related data. */
    					IResultMap testSkippedResult = testObj.getSkippedTests();
    					String skippedTestMethodInfo = this.getTestMethodReport(testName, testSkippedResult, false, true);
    					retBuf.append(skippedTestMethodInfo);
    					
    					/* Get passed test method related data. */
    					IResultMap testPassedResult = testObj.getPassedTests();
    					String passedTestMethodInfo = this.getTestMethodReport(testName, testPassedResult, true, false);
    					retBuf.append(passedTestMethodInfo);
    				}
    			}
    		}catch(Exception ex)
    		{
    			ex.printStackTrace();
    		}finally
    		{
    			return retBuf.toString();
    		}
    	}
    	
    	/* Get failed, passed or skipped test methods report. */
    	private String getTestMethodReport(String testName, IResultMap testResultMap, boolean passedReault, boolean skippedResult)
    	{
    		StringBuffer retStrBuf = new StringBuffer();
    		
    		String resultTitle = testName;
    		
    		String color = "green";
    		
    		if(skippedResult)
    		{
    			resultTitle += " - Skipped ";
    			color = "yellow";
    		}else
    		{
    			if(!passedReault)
    			{
    				resultTitle += " - Failed ";
    				color = "red";
    			}else
    			{
    				resultTitle += " - Passed ";
    				color = "green";
    			}
    		}
    		
    		retStrBuf.append("<tr bgcolor=" + color + "><td colspan=7><center><b>" + resultTitle + "</b></center></td></tr>");
    			
    		Set<ITestResult> testResultSet = testResultMap.getAllResults();
    			
    		for(ITestResult testResult : testResultSet)
    		{
    			String testClassName = "";
    			String testMethodName = "";
    			String startDateStr = "";
    			String executeTimeStr = "";
    			String paramStr = "";
    			String reporterMessage = "";
    			String exceptionMessage = "";
    			
    			//Get testClassName
    			testClassName = testResult.getTestClass().getName();
    				
    			//Get testMethodName
    			testMethodName = testResult.getMethod().getMethodName();
    				
    			//Get startDateStr
    			long startTimeMillis = testResult.getStartMillis();
    			startDateStr = this.getDateInStringFormat(new Date(startTimeMillis));
    				
    			//Get Execute time.
    			long deltaMillis = testResult.getEndMillis() - testResult.getStartMillis();
    			executeTimeStr = this.convertDeltaTimeToString(deltaMillis);
    				
    			//Get parameter list.
    			Object paramObjArr[] = testResult.getParameters();
    			for(Object paramObj : paramObjArr)
    			{
    				paramStr += (String)paramObj;
    				paramStr += " ";
    			}
    				
    			//Get reporter message list.
    			List<String> repoterMessageList = Reporter.getOutput(testResult);
    			for(String tmpMsg : repoterMessageList)				
    			{
    				reporterMessage += tmpMsg;
    				reporterMessage += " ";
    			}
    				
    			//Get exception message.
    			Throwable exception = testResult.getThrowable();
    			if(exception!=null)
    			{
    				StringWriter sw = new StringWriter();
    				PrintWriter pw = new PrintWriter(sw);
    				exception.printStackTrace(pw);
    				
    				exceptionMessage = sw.toString();
    			}
    			
    			retStrBuf.append("<tr bgcolor=" + color + ">");
    			
    			/* Add test class name. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(testClassName);
    			retStrBuf.append("</td>");
    			
    			/* Add test method name. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(testMethodName);
    			retStrBuf.append("</td>");
    			
    			/* Add start time. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(startDateStr);
    			retStrBuf.append("</td>");
    			
    			/* Add execution time. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(executeTimeStr);
    			retStrBuf.append("</td>");
    			
    			/* Add parameter. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(paramStr);
    			retStrBuf.append("</td>");
    			
    			/* Add reporter message. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(reporterMessage);
    			retStrBuf.append("</td>");
    			
    			/* Add exception message. */
    			retStrBuf.append("<td>");
    			retStrBuf.append(exceptionMessage);
    			retStrBuf.append("</td>");
    			
    			retStrBuf.append("</tr>");
    
    		}
    		
    		return retStrBuf.toString();
    	}
    	
    	/* Convert a string array elements to a string. */
    	private String stringArrayToString(String strArr[])
    	{
    		StringBuffer retStrBuf = new StringBuffer();
    		if(strArr!=null)
    		{
    			for(String str : strArr)
    			{
    				retStrBuf.append(str);
    				retStrBuf.append(" ");
    			}
    		}
    		return retStrBuf.toString();
    	}
    
    }

    CustomTestNGReporter implements IReporter

  4. Right click main-suite.xml, click ” Run As —> TestNG Suite ”
    run testng main-suite.xml
  5. After execution, you can see custom-emailable-report.html in left panel list under test-output folder. Open it in your favorite web browser, you can see the customized TestNG reports.
    custom emailabel testng report

Download “TestNGReportingCustomizationExample.zip” TestNGReportingCustomizationExample.zip – Downloaded 349 times – 120 KB

(Visited 2,131 times, 15 visits today)
READ :   TestNG Parameterized Testing Via DataProvider Or Testng.xml

8 Comments


  1. Nice Article…Very Helpful..:)

    Reply

  2. unable to automatically create custom emailable report html file , can you help ? I have followed every steps right

    Reply

    1. What error have you encountered? Can you show the error messages?

      Reply

  3. please share the import statements

    i am getting lot of errors

    Reply

    1. import java.awt.List;
      import java.io.BufferedReader;
      import java.io.File;
      import java.io.FileNotFoundException;
      import java.io.FileReader;
      import java.io.FileWriter;
      import java.io.PrintWriter;
      import java.io.StringWriter;
      import java.text.DateFormat;
      import java.text.SimpleDateFormat;
      import java.util.Date;
      import java.util.Map;
      import java.util.Set;

      import org.testng.IReporter;
      import org.testng.IResultMap;
      import org.testng.ISuite;
      import org.testng.ISuiteResult;
      import org.testng.ITestContext;
      import org.testng.ITestResult;
      import org.testng.Reporter;

      what imports are missing here?

      Reply

      1. finally i fixed all the errors

        running the testng suite , not generating the customer report

        Please help

        Reply

  4. @Senthil, You can import statements below:

    I am getting Customized report successfully for every run.Hope it helps for you.

    import java.io.BufferedReader;
    import java.io.File;
    import java.io.FileNotFoundException;
    import java.io.FileReader;
    import java.io.FileWriter;
    import java.io.PrintWriter;
    import java.io.StringWriter;
    import java.text.DateFormat;
    import java.text.SimpleDateFormat;
    import java.util.Date;
    import java.util.List;
    import java.util.Map;
    import java.util.Set;

    import org.apache.batik.bridge.UserAgent;
    import org.openqa.selenium.Capabilities;
    import org.openqa.selenium.JavascriptExecutor;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.remote.BrowserType;
    import org.openqa.selenium.remote.RemoteWebDriver;
    import org.testng.IReporter;
    import org.testng.IResultMap;
    import org.testng.ISuite;
    import org.testng.ISuiteResult;
    import org.testng.ITestContext;
    import org.testng.ITestResult;
    import org.testng.Reporter;
    import org.testng.xml.XmlSuite;

    Reply

  5. Hi,

    Thank you so much for code.It works fine for all test cases passed.

    Some of the points identified while working on above script:

    + I am unable to get the customized template report when test case failed.
    + If any exception raises at script level, which doesn’t show in customized report.
    + It doesn’t show total count of methods in the particular project/product

    Could you please help us in above points.

    Thanks in Advance.

    Best Regards,
    Murali V.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *