Selenium testng.xml and emailable report in detail

To get started with testNG framework, refer testNG framework If we want to execute the testng class, we can do one by one in eclipse by clicking F5 or right click run as,

what if we want to run multiple classes, then we can take use of testng.xml.

tetng.xml helps to execute one or more testNG classes at a time and also gives an option to maintain the sequence of execution of java classes and also it provides various ways to control the execution.

Let’s discuss them one by one.

Let’s have 3 testNG tests in 3 different class files

Pass -1, Fail -1 & Skip -1

package testpkg;
import org.testng.annotations.Test;
public class tc1 {
    @Test
    public void method1(){
        System.out.println("tc1: testcase1");
    }
}

package testpkg;
import org.testng.annotations.Test;
public class tc2 {
    @Test
    public void method1(){
        int i = 5/0;
        System.out.println(i+"testcase2");
    }
}

package testpkg;
import org.testng.SkipException;
import org.testng.annotations.Test;
public class tc3 {
    @Test
    public void method1(){
        throw new SkipException("skipped: testcase3");
    }
}

For this above classes, we will see the basic testng.xml execution:
Note: we can have some other name than testng.xml, like can have testng_regression.xml

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="MySuite" parallel="none">
    <test name="Regression" preserve-order="true">
        <classes>
            <class name="testNG.testpkg.tc1">
                <methods>
                    <include name="method1" />
                </methods>
            </class>
            <class name="testNG.testpkg.tc2"/>
        </classes>
    </test>
    <test name="UAT">
        <classes>
            <class name="testNG.testpkg.tc3"/>
        </classes>
    </test>
</suite>

Let’s understand the above xml before running….

• suite name can be anything.
• parallel attribute is set to none, as we want to run this suite once, when we want to run multiple times in different browser, we can set this attribute. please refer cross browser testing.
• test name can be anything, enter a name such that it relates to your purpose.
• preserve-order is set to false, if we want to run the mentioned classes in an unpredictable manner, if want to run in a sequence they are written, update the value to true.
• in one test tag, we can have multiple classes, and each class can have multiple methods, if no method tag present under class tag, then all the methods of that class will be executed.
• class name should be exactly same as  packageName.className

If we want to run all the classes of a package, no need to mention all the classes of that package under test tag, can have a package tag as mentioned below:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="MySuite" parallel="none">
<test name="Regression">
   <packages>
      <package name="testpkg" />
   </packages>
</test>
</suite>

Run any one of the above testng.xml [Refer my prev blog, for different ways of running testng.xml]

Output will be:

tc1: testcase1


java.lang.ArithmeticException: / by zero

	at testNG.testpkg.tc2.method1(tc2.java:7)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:134)
	at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:597)
	at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:173)
	at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    ...


Test ignored.

===============================================
MySuite
Total tests run: 3, Passes: 1, Failures: 1, Skips: 1
===============================================

The output clearly says that, total 3 tests run in which 1 skipped, 1 failed, and 1 successfully executed.
To get more detailed report about the test execution, let’s see the test-output folder.
(As created under project folder, if not appears, right click on the project and refresh)
test-output > index.html [open in browser]

Note – what to do if test-output folder not created

Edit configuration > Configuration > Listeners > Check the “Use default reporters” checkbox & rerun the testng.xml file

Another report called as (emailable-report.html), which you might be interested to look at.

test-output > emailable-report.html [open in browser]

Hope this helps!

Let’s get to know more about testNG, that is testNG annotations