JUnit 5 Dynamic Tests

By Arvind Rai, April 21, 2018

This page will walk through JUnit 5 dynamic tests example. Dynamic test is generated at runtime by a factory method that is annotated with @TestFactory. Dynamic test has been introduced in JUnit 5 Jupiter. @TestFactory method differs with @Test method in such a way that @TestFactory method is not itself a test case but rather a factory for test cases. So we can say that dynamic test is the product of a factory method. @TestFactory method must not be private or static and must return a Stream, Collection, Iterable, or Iterator of DynamicNode instances. DynamicNode is an abstract class and it has two subclasses DynamicTest and DynamicContainer that can be instantiated. DynamicTest is a test case generated at run time and DynamicContainer is a container generated at runtime. DynamicTest instances will run lazily. In the dynamic test lifecycle @BeforeEach and @AfterEach methods are executed for the @TestFactory method but not for each dynamic test. Dynamic tests are useful when we need to run the same set of tests on many different input values. Now find the complete example of JUnit dynamic tests step by step.

Technologies Used
Gradle Build File
DynamicNode
Instantiate DynamicTest
@TestFactory Annotation
Dynamic Tests with Stream
Dynamic Tests with Collection
Dynamic Tests with Iterable
Dynamic Tests with Iterator
Dynamic Tests with DynamicContainer
Dynamic Tests with Interface Default Method
@BeforeEach and @AfterEach with @TestFactory
Complete Example
References
Download Source Code

Technologies Used

Find the technologies being used in our example.
1. Java 9
2. JUnit 5
3. Gradle 4.3.1
4. Eclipse Oxygen.3a

Gradle Build File

Find the Gradle build file to resolve the JUnit 5 JAR dependencies.
build.gradle

apply plugin: 'java'
apply plugin: 'eclipse'
archivesBaseName = 'concretepage'
version = '1' 
repositories {
    mavenCentral()
}
dependencies {
    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-api', version: '5.1.1'
    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-engine', version: '5.1.1'
    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-params', version: '5.1.1'
    testCompile group: 'org.junit.platform', name: 'junit-platform-launcher', version: '1.1.1'
}

DynamicNode

DynamicNode is the abstract base class for a container or a test case generated at run time. DynamicNode has two subclasses DynamicTest and DynamicContainer that can be instantiated.
DynamicTest: It is a test case generated at run time. It is created using display name and JUnit Executable functional interface. DynamicTest instances will be executed lazily. Find the sample code to instantiate DynamicTest.

DynamicTest.dynamicTest("Test",
            		() -> assertTrue(number % 2 == 0, "Output will be true"))

DynamicContainer: It is a container generated at run time. It is created using display name and Iterable or Stream of instances of DynamicNode. Find the sample code to instantiate DynamicContainer.

DynamicContainer.dynamicContainer("Container", Stream.of(
      DynamicTest.dynamicTest("Not Null", () -> assertNotNull(utility.getAllUsers())),
      DynamicTest.dynamicTest("Not Empty", () -> assertTrue(utility.getAllUsers().isEmpty()))
))

Instantiate DynamicTest

DynamicTest has a factory method as dynamicTest to create DynamicTest instances.
Find the structure of dynamicTest method from JUnit doc.

public static DynamicTest dynamicTest(String displayName, Executable executable)

We can see that we need to pass display name and instance of JUnit Executable functional interface in above factory method.
Find the Executable abstract method structure from JUnit doc.

void execute() throws Throwable;

The above method has no return type and no arguments. We can instantiate Executable functional interface using lambda expressions, method references, or constructor references.
Find the sample instantiation of Executable functional interface using lambda expression.

org.junit.jupiter.api.function.Executable exeTest = () -> { 
       	 assertTrue(7 > 5, "Result will be true");
};

The above lambda expressions has no return type and no arguments to match execute method structure of JUnit Executable functional interface. Now DynamicTest can be instantiated as following.

DynamicTest dyTest = DynamicTest.dynamicTest("Test", exeTest);

We can also directly pass lambda expression to our dynamicTest method as following.

DynamicTest dyTest = DynamicTest.dynamicTest("Test ", () -> { 
    assertTrue(7 > 5, "Result will be true");
});

Now find the sample dynamic test code in which we initialize DynamicTest using lambda expression and method reference.
a. Dynamic test using lambda expression

@TestFactory
@DisplayName("Dynamic Test using Lambda Expression")
Stream<DynamicTest> evenNumberDynamicTestWithStream() {
	return IntStream.of(2, 4, 6)
		.mapToObj(number -> DynamicTest.dynamicTest("Number " + number,
				() -> assertTrue(number % 2 == 0, "Output will be true")));
}

b. Dynamic test using method reference

@TestFactory
@DisplayName("Dynamic Test using Method Reference")
Collection<DynamicTest> oddEvenDynamicTestWithCollection() {
	return Arrays.asList(
		DynamicTest.dynamicTest("Even Number Test", TestUtility::evenNumverTest),
		DynamicTest.dynamicTest("Odd Number Test", TestUtility::oddNumberTest)
	 );
}

@TestFactory Annotation

@TestFactory denotes that the method is test factory for dynamic tests. @TestFactory method differs with @Test method in such a way that @TestFactory method is not itself a test case but rather a factory for test cases. So we can say that dynamic test is the product of a factory method. @TestFactory method must not be private or static and must return a Stream, Collection, Iterable, or Iterator of DynamicNode instances. Find a sample example of @TestFactory returning Stream.

@TestFactory
Stream<DynamicTest> dataLengthDynamicTest() {
	return Stream.of("ABC", "PQRS", "XYZ")
		.map(el -> DynamicTest.dynamicTest("Test: " + el, () -> assertTrue(el.length() > 2)));
}

As we know that DynamicNode has the subclasses as DynamicTest and DynamicContainer. We create our test cases by creating the instances of DynamicTest. In the above test factory method we are returning a Stream of instances of DynamicTest. These test cases as instances of DynamicTest will execute lazily.

Dynamic Tests with Stream

Stream has been introduced in Java 8. It is a sequence of elements that supports sequential and parallel aggregate operations. In JUnit dynamic tests, we can use Stream to contain our all DynamicTest instances. The @TestFactory method needs to return Stream of DynamicTest type. Find the example.

@TestFactory
@DisplayName("Dynamic Test with Stream")
Stream<DynamicTest> userAvailabilityDynamicTestWithStream() {
	TestUtility utility = new TestUtility();
	return Stream.of("Krishn", "Arjun", "Bheem")
		.map(user -> DynamicTest.dynamicTest("User Test: " + user, () -> { 
			assertNotNull(utility.getAllUsers());
			utility.addUser(user);
			assertTrue(utility.getAllUsers().contains(user), "User will exist");
			utility.deleteUser(user);
			assertFalse(utility.getAllUsers().contains(user), "User will not exist");            	
		 }));
}

The number of instances of DynamicTest will be created as many as the number of elements in Stream.

Dynamic Tests with Collection

We can use Java Collection to create JUnit dynamic tests. The @TestFactory method needs to return Collection of DynamicTest type. We need to collect all our DynamicTest instances and return it.

@TestFactory
@DisplayName("Dynamic Test with Collection")
Collection<DynamicTest> mathDynamicTestWithCollection() {
	TestUtility utility = new TestUtility();
	return Arrays.asList(
		DynamicTest.dynamicTest("Addition Test",
			() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
		DynamicTest.dynamicTest("Multiplication Test",
			() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
	 );
}

Dynamic Tests with Iterable

We can use Java Iterable to create JUnit dynamic tests. The @TestFactory method needs to return Iterable of DynamicTest type.

@TestFactory
@DisplayName("Dynamic Test with Iterable")
Iterable<DynamicTest> mathDynamicTestWithIterable() {
	TestUtility utility = new TestUtility();
	return Arrays.asList(
		DynamicTest.dynamicTest("Addition Test",
			() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
		DynamicTest.dynamicTest("Multiplication Test",
			() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
	 );
}

Dynamic Tests with Iterator

We can use Java Iterator to create JUnit dynamic tests. The @TestFactory method needs to return Iterator of DynamicTest type.

@TestFactory
@DisplayName("Dynamic Test with Iterator")
Iterator<DynamicTest> mathDynamicTestWithIterator() {
	TestUtility utility = new TestUtility();
	return Arrays.asList(
		DynamicTest.dynamicTest("Addition Test",
			() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
		DynamicTest.dynamicTest("Multiplication Test",
			() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
	 ).iterator();
}

Dynamic Tests with DynamicContainer

DynamicContainer is the subclass of DynamicNode. DynamicContainer is a container generated at run time. It is created using display name and Iterable or Stream of instances of DynamicNode. Find the example.

@TestFactory
@DisplayName("Dynamic Test with Containers")
Stream<DynamicNode> userAvailabilityDynamicTestWithContainers() {
	TestUtility utility = new TestUtility();
	return Stream.of("Krishn", "Arjun")
		.map(user -> DynamicContainer.dynamicContainer("User Container : " + user, Stream.of(
			DynamicTest.dynamicTest("Not Null", () -> assertNotNull(utility.getAllUsers())),
			DynamicTest.dynamicTest("Not Empty", () -> assertTrue(utility.getAllUsers().isEmpty())),
			DynamicContainer.dynamicContainer("Test Properties", Stream.of(
				DynamicTest.dynamicTest("Add User", () -> {
					   utility.addUser(user);
					   assertTrue(utility.getAllUsers().contains(user), "User will exist");
				}),
				DynamicTest.dynamicTest("Delete User", () -> {
					   utility.deleteUser(user);
					   assertFalse(utility.getAllUsers().contains(user), "User will not exist");
				})
			))
		)));
}

Dynamic Tests with Interface Default Method

We can create dynamic tests with interface defaults methods and can run it by implementing interface in our test class. To create a dynamic test in interface we need to annotate default method with @TestFactory annotation and return Stream, Collection, Iterable, or Iterator of DynamicNode instances. Find the example returning Collection.
DynamicTestWithInterface.java

public interface DynamicTestWithInterface {
    @TestFactory
    @DisplayName("Dynamic Test with Interface Default Method")
    default Collection<DynamicTest> oddEvenDynamicTestWithDefaultMethod() {
        return Arrays.asList(
    		DynamicTest.dynamicTest("Even Number Test", () -> {
    			   int num = 21;
    			   assertTrue(num % 2 == 1, "Result will be true");
    		}),
    		DynamicTest.dynamicTest("Odd Number Test", () -> {
    			   int num = 30;
    			   assertTrue(num % 2 == 0, "Result will be true");
    		})
         );
    }  
}

@BeforeEach and @AfterEach with @TestFactory

In the dynamic test lifecycle @BeforeEach and @AfterEach methods are executed for the @TestFactory method but not for each dynamic test. The @BeforeEach method will execute before execution of @TestFactory method and @AfterEach method will execute after complete execution of @TestFactory method. Find the example.
BeforeEachAndAfterEach.java

package com.concretepage;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.TestInfo;

public class BeforeEachAndAfterEach {
    @BeforeAll
    static void initAll() {
    	System.out.println("---Inside initAll---");
    }
    @BeforeEach
    void init(TestInfo testInfo) {
    	System.out.println("Start..." + testInfo.getDisplayName());
    }	
    @TestFactory
    @DisplayName("Dynamic Test")
    Stream<DynamicTest> oddNumberDynamicTestWithStream() {
        return IntStream.of(3, 5, 7)
            .mapToObj(number -> DynamicTest.dynamicTest("Number " + number,
            	() -> {
            		System.out.println("Test "+ number);
            		assertTrue(number % 2 == 1, "Output will be true");
            	}));
    }    
    @AfterEach
    void tearDown(TestInfo testInfo) {
    	System.out.println("Finished..." + testInfo.getDisplayName());
    } 
    @AfterAll
    static void tearDownAll() {
    	System.out.println("---Inside tearDownAll---");
    }  
}

Output

---Inside initAll---
Start...Dynamic Test
Test 3
Test 5
Test 7
Finished...Dynamic Test
---Inside tearDownAll---

Complete Example

DynamicTestWithInterface.java

package com.concretepage;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.Arrays;
import java.util.Collection;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

public interface DynamicTestWithInterface {
    @TestFactory
    @DisplayName("Dynamic Test with Interface Default Method")
    default Collection<DynamicTest> oddEvenDynamicTestWithDefaultMethod() {
        return Arrays.asList(
    		DynamicTest.dynamicTest("Even Number Test", () -> {
    			   int num = 21;
    			   assertTrue(num % 2 == 1, "Result will be true");
    		}),
    		DynamicTest.dynamicTest("Odd Number Test", () -> {
    			   int num = 30;
    			   assertTrue(num % 2 == 0, "Result will be true");
    		})
         );
    }  
}

LifecycleLogger.java

package com.concretepage;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.TestInfo;

public interface LifecycleLogger {
    @BeforeAll
    static void initAll() {
    	System.out.println("---Inside initAll---");
    }
    @BeforeEach
    default void init(TestInfo testInfo) {
    	System.out.println("Start..." + testInfo.getDisplayName());
    }	
    @AfterEach
    default void tearDown(TestInfo testInfo) {
    	System.out.println("Finished..." + testInfo.getDisplayName());
    } 
    @AfterAll
    static void tearDownAll() {
    	System.out.println("---Inside tearDownAll---");
    }    
}

TestUtility.java

package com.concretepage;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.ArrayList;
import java.util.List;

public class TestUtility {
   List<String> allUsers = new ArrayList<>();
   public List<String> getAllUsers() {
	   return allUsers;
   }
   public void addUser(String user) {
	   allUsers.add(user);
   }
   public void deleteUser(String user) {
	   allUsers.remove(user);
   }
   public static void evenNumverTest() {
	   int num = 30;
	   assertTrue(num % 2 == 0, "Result will be true");
   }
   public static void oddNumberTest() {
	   int num = 21;
	   assertTrue(num % 2 == 1, "Result will be true");
   }   
   public long multiplyNumbers(int num1, int num2) {
	  return num1 * num2;
   }
   public long addNumbers(int num1, int num2) {
	  return num1 + num2;
   }    
}

MyJUnitTests.java

package com.concretepage;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DynamicContainer;
import org.junit.jupiter.api.DynamicNode;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

public class MyJUnitTests implements DynamicTestWithInterface, LifecycleLogger {
    @TestFactory
    @DisplayName("Dynamic Test using Lambda Expression")
    Stream<DynamicTest> evenNumberDynamicTestWithStream() {
        return IntStream.of(2, 4, 6)
            .mapToObj(number -> DynamicTest.dynamicTest("Number " + number,
            		() -> assertTrue(number % 2 == 0, "Output will be true")));
    }
    
    @TestFactory
    @DisplayName("Dynamic Test using Method Reference")
    Collection<DynamicTest> oddEvenDynamicTestWithCollection() {
        return Arrays.asList(
        	DynamicTest.dynamicTest("Even Number Test", TestUtility::evenNumverTest),
        	DynamicTest.dynamicTest("Odd Number Test", TestUtility::oddNumberTest)
         );
    }     
    
    @TestFactory
    @DisplayName("Dynamic Test with Stream")
    Stream<DynamicTest> userAvailabilityDynamicTestWithStream() {
    	TestUtility utility = new TestUtility();
        return Stream.of("Krishn", "Arjun", "Bheem")
            .map(user -> DynamicTest.dynamicTest("User Test: " + user, () -> { 
            	assertNotNull(utility.getAllUsers());
            	utility.addUser(user);
            	assertTrue(utility.getAllUsers().contains(user), "User will exist");
            	utility.deleteUser(user);
            	assertFalse(utility.getAllUsers().contains(user), "User will not exist");            	
             }));
    }

    @TestFactory
    @DisplayName("Dynamic Test with Collection")
    Collection<DynamicTest> mathDynamicTestWithCollection() {
    	TestUtility utility = new TestUtility();
        return Arrays.asList(
        	DynamicTest.dynamicTest("Addition Test",
        		() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
        	DynamicTest.dynamicTest("Multiplication Test",
        		() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
         );
    }

    @TestFactory
    @DisplayName("Dynamic Test with Iterable")
    Iterable<DynamicTest> mathDynamicTestWithIterable() {
    	TestUtility utility = new TestUtility();
        return Arrays.asList(
        	DynamicTest.dynamicTest("Addition Test",
        		() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
        	DynamicTest.dynamicTest("Multiplication Test",
        		() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
         );
    }

    @TestFactory
    @DisplayName("Dynamic Test with Iterator")
    Iterator<DynamicTest> mathDynamicTestWithIterator() {
    	TestUtility utility = new TestUtility();
        return Arrays.asList(
        	DynamicTest.dynamicTest("Addition Test",
        		() -> assertTrue(utility.addNumbers(20, 30) == 50, "Output will be true")),
        	DynamicTest.dynamicTest("Multiplication Test",
        		() -> assertTrue(utility.multiplyNumbers(9, 5) == 45, "Output will be true"))
         ).iterator();
    }       
    
    @TestFactory
    @DisplayName("Dynamic Test with Containers")
    Stream<DynamicNode> userAvailabilityDynamicTestWithContainers() {
    	TestUtility utility = new TestUtility();
        return Stream.of("Krishn", "Arjun")
            .map(user -> DynamicContainer.dynamicContainer("User Container : " + user, Stream.of(
              	DynamicTest.dynamicTest("Not Null", () -> assertNotNull(utility.getAllUsers())),
              	DynamicTest.dynamicTest("Not Empty", () -> assertTrue(utility.getAllUsers().isEmpty())),
              	DynamicContainer.dynamicContainer("Test Properties", Stream.of(
              		DynamicTest.dynamicTest("Add User", () -> {
            	           utility.addUser(user);
            	           assertTrue(utility.getAllUsers().contains(user), "User will exist");
              		}),
              		DynamicTest.dynamicTest("Delete User", () -> {
            	           utility.deleteUser(user);
            	           assertFalse(utility.getAllUsers().contains(user), "User will not exist");
              		})
              	))
            )));
    }    
}

To run the example, download the source code and import into Eclipse Oxygen IDE. To run a test class, right click on the test class -> Run As -> JUnit Test. If we are using JUnit 4 environment to run JUnit 5 examples then we need to annotate our test class with @RunWith(JUnitPlatform.class) annotation. Eclipse Oxygen.1a onwards provides JUnit 5 environment. In our example, we are using Eclipse Oxygen.3a to run test class. Find the output in console.

---Inside initAll---
Start...Dynamic Test with Interface Default Method
Finished...Dynamic Test with Interface Default Method
Start...Dynamic Test with Collection
Finished...Dynamic Test with Collection
Start...Dynamic Test using Lambda Expression
Finished...Dynamic Test using Lambda Expression
Start...Dynamic Test using Method Reference
Finished...Dynamic Test using Method Reference
Start...Dynamic Test with Stream
Finished...Dynamic Test with Stream
Start...Dynamic Test with Containers
Finished...Dynamic Test with Containers
Start...Dynamic Test with Iterable
Finished...Dynamic Test with Iterable
Start...Dynamic Test with Iterator
Finished...Dynamic Test with Iterator
---Inside tearDownAll---

Find the print screen of JUnit run.