TestNG Report Example

When you create and run a TestNG project, it will generate a lot of html or xml report files. The report files include huge amount of useful information that can help you to check the test case status, debug test case steps and show test case execution results to your manager. So it is very important to know all the key reports that TestNG generated for you.

What You Will Learn

  1. Create a TestNG project with one main suite, the main suite will contain two child suite.
  2. Create some pass and fail test in above suite. Some has data provider and some are grouped.
  3. Generate TestNG report files.
  4. Introduce each html and xml reports generated.

Create TestNG Suite Project

You can read TestNG Eclipse PluginTestNG Tutorial to learn how to crate TestNG project. Below we only show you how to add main and sub suite xml in it.

  1. Right click the project name, click ” New —> Others…”, choose “TestNG class” in the popup dialog, click Next.
  2. Browse the source folder to your project src folder. Input the package name and class name. In the xml suite file input box, input the file path relative to src folder as below picture.
    testng suite xml generation
  3. After click Finish button, you can see main-suite.xml exist under the report folder in left panel.
    main-suite.xml location
  4. Create other test case class and suite1.xml and suite2.xml as below.
    main-suite.xml suite1.xml and suite2.xml

Code Example Files Content

  1. main-suite.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <suite name="MainSuite" parallel="false">
      <suite-files>
      	<suite-file path="./suite1.xml" />
      	<suite-file path="./suite2.xml" />
      </suite-files>
      <test name="MainSuiteTest">
        <classes>
          <class name="com.dev2qa.example.testng.report.TestCaseClass1"/>
          <class name="com.dev2qa.example.testng.report.TestCaseClass2"/>
        </classes>
      </test> <!-- MainSuiteTest -->
    </suite> <!-- MainSuite -->
  2. TestCaseClass1.java
    public class TestCaseClass1 {
      
    	@Test /* This method will pass. */
    	public void tc1m1() {
    	  
    	}
    	
    	@Test /* This method will fail. */
    	public void tc1m2() {
    	  String str = null;
    	  System.out.println(str.length());
    	}
    	
    	@Test(enabled=false) /* This method will ignore. */
    	public void tc1m3() {
    
    	}
    }
  3. TestCaseClass2.java
    public class TestCaseClass2 {
    
    	  /* tcc2m1 and tcc2m2 are all in group1. */
    	  @Test(groups="group1")
    	  public void tcc2m1() {
    		  Assert.assertTrue(false);
    	  }
    	  
    	  @Test(groups="group1")
    	  public void tcc2m2() {
    		  Assert.assertTrue(true);
    	  }
    	  
    	  /* tcc2m3 in group2*/
    	  @Test(groups="group2")
    	  public void tcc2m3() {
    		  Assert.assertTrue(true);
    	  }
    	  
    	  @Test
    	  public void tcc2m4() {
    		  Assert.assertTrue(true);
    	  }
    
    	  /* tcc2m5 has a data provider. */
    	  @Test(dataProvider="exampleDP")
    	  public void tcc2m5(String param) {
    	  }
    	  
    	  @DataProvider
    	  private Object[][] exampleDP() {
    		  return new Object[][]{{"Java"}, {"Selenium"}};
    	  }
    }
  4. suite1.xml
    <suite name="Suite1" parallel="false">
      <test name="Suite1Test1">
        <classes>
          <class name="com.dev2qa.example.testng.report.TestCaseClass1"/>
          <class name="com.dev2qa.example.testng.report.TestCaseClass3"/>
        </classes>
      </test>
      <test name="Suite1Test2">
        <classes>
          <class name="com.dev2qa.example.testng.report.TestCaseClass4"/>
        </classes>
      </test>
    </suite>
  5. TestCaseClass3.java
    public class TestCaseClass3 {
      @Test(groups="group3")
      public void tcc3m1() {
    	  /* Reporter will log the message in TestNG Reporter output. */
    	  Reporter.log("This method is TestCaseClass3.tcc3m1");
      }
      
      @Test(groups="group3")
      public void tcc3m2() {
    	  
      }
    }
  6. TestCaseClass4.java
    public class TestCaseClass4 {
    
    	  @Test
    	  public void tcc4m1() {
    		  
    	  }
    	  
    	  @Test
    	  public void tcc4m2() {
    		  Reporter.log("This method is TestCaseClass4.tcc4m2");
    	  }
    }
  7. suite2.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <suite name="Suite2" parallel="false">
      <test name="Suite2Test1">
        <classes>
          <class name="com.dev2qa.example.testng.report.TestCaseClass5"/>
          <class name="com.dev2qa.example.testng.report.TestCaseClass6"/>
        </classes>
      </test>
    </suite>
  8. TestCaseClass5.java
    public class TestCaseClass5 {
      @Test
      public void tcc5m1() {
    	  Reporter.log("This method is TestCaseClass5.tcc5m1");
      }
    }
  9. TestCaseClass6.java
    public class TestCaseClass6 {
    	  @Test
    	  public void tcc6m1() {
    		  Reporter.log("This method is TestCaseClass6.tcc6m1");
    	  }
    }
    

Generate TestNG Reports

  1. Right click main-suite.xml file, Click ” Run As —> TestNG Suite ” in popup menu.
    run testng main-suite.xml
  2. When execution complete, you can see below output in console.
    test suite execute result in console
  3. Right click the project name, click ” Refresh ” in popup menu. Then you can see the test-output folder. There are many html and xml reports file there.
    output folder for testng test report
READ :   Run Selenium Test Case Use TestNG

Report Files Introduction

test-output / index.html

This is the html report enter page. Open it in a web browser, you can see below page. Left panel list all the three suite and related links. Click the link, right panel will show the detail information.
index.html enter page for testng report

  1. Each suite has a Info and Results section.
    Info — Information about the suite. Such as main-suite.xml, test status, group status link.
    Results — Test result link, pass methods, fail methods etc.
  2. Info section primary links intro:
    main-suite.xml — The content of suite xml file.
    Reporter output — Click this link to show Reporter.log() logged messages in right panel. Reporter.log() method can be used in any test method to log custom message.
    custom reporter log information
    Ignored methods — List all ignored test methods such as the method with @Test(enabled=false) annotation.
    Chronological view – List test methods by their execution time in chronological order.
  3. Detail Info Panel
    Which ever link you click in left panel. The right panel will show the detail information related to it. For example, If you click the fail method name, the detail panel will show the detail error information about the failure.

test-output / emailable-report.html

This report file is generally used to send to other team member by email. It includes all test suite and test method status information. It has been divided into three sections.

  1. The test suite summary list all the test executed in this suite and sub suite.
    testng emailable report test suite summary section
  2. When you click any test name in above list, it will link you to detail execution status section.
    testng report test method detail status information
  3. When you click any method name in above list, it will show the detail execution information include error message if exist.
    testng test method detail execution message
READ :   TestNG First Example

Other Important Report Files Under test-output Folder. 

old / index.html : This is old style TestNG reports.

testng-failed.xml : This file include only failed test class and methods. You can easily rerun these failed methods again by right click this xml file and then click ” Run As —> TestNG Suite “.

junitreports folder : Junit format report xml files for all test class.

testng-results.xml : Xml format file for the results report.

Suite1, Sutie2 folder: Include individual html and xml reports for each test case.

Download “TestNGReportExample.zip” TestNGReportExample.zip – Downloaded 82 times – 105 KB

(Visited 1,066 times, 2 visits today)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.