Test cases

Test cases are interfaces which implements JUnit 5 Test Factories (tests generators). Each test case is built around one single use case, and is implemented using Java 8’s interfaces (with their default feature), in order to simulate multiple inheritance.

Let’s have a look at the NoOrphanAliasInLayoutsRuleTest test class to get a better understanding. As a linting rule, the NoDuplicateRelationshipsRule rule has to implement at least the createParseTreeListener method, and could override the ones implemented in its parent class. Each of these methods should be tested, and there is a dedicated test case for each of them.

For instance, CreateParseTreeListenerLintingRuleTestCase tests the behavior of the createParseTreeListener method: let’s see how. At first, the test class needs to provide a instance to test, using the @SUT annotation. It can be placed on a field or a method, and the rule will pick the first (alphabetical order) annotated field, or else the first annotated method (alphabetical order), as the SUT instance (System Under Test).

The test case will then look for a specific annotation on the test class (here it will look for the @CreateParseTreeListener). If the annotation is not present, the test case won’t even be triggered.

You can of course combine test cases with the JUnit @Nested feature, in order to provide multiple SUT with multiple configurations.

Below is a list of the existing test cases so far, along with their usage. Feel free to browse the Javadoc of the test bed JARs in order to learn more about them.

Finally, you may want to have a look at the way rules linting messages are tested.

Mock helper design pattern

The mock helper design pattern is a specialization of the builder design pattern and aims to increase the tests readability and maintainability, by factorizing the mock frameworks boiler code.

Let’s study a simple test using the EasyMock mocking framework (the same could be achieved with Mockito):

The following code is a sample from the unit test:

@InjectMocks
class CDataDocumentDecoratorTest {

    static class TestData {

        ...
        static class Export {
        
            static final boolean INDENT = true;

            static final boolean STRICT_VALIDATION = true;

            static final boolean VALIDATE = true;
        }
        
        static class TextNode {
        
            static final String NAME = "someNodeName";
            
            static final Text NODE = MockType.NICE.createMock(Text.class);
        }
        ...
    }

    ...
    @Mock
    Document documentMock;
    ...
    
    @AfterEach
    void tearDown() {
        verify(this.documentMock);
    }
    ...
    @Test
    void testCreateTextNode_shouldDelegateToDecoratedDocument() {

        setUpStrictMock(this.documentMock)
            .toCreateTextNodeGiven(TestData.TextNode.NAME)
                .andReturn(TestData.TextNode.NODE)
                .setUpDone();

        final Text results = this.sut.createTextNode(TestData.TextNode.NAME);
        assertThat(results).isEqualTo(TestData.TextNode.NODE);
    }
    ...
}

You can notice an annotated documentMock field, which will have its value instantiated and injected at runtime thanks to the use of the @InjectMocks JUnit custom extension.

The testCreateTextNode_shouldDelegateToDecoratedDocument focuses on what should the mock do, instead of how. If we should have to rewrite the test without the mock helper pattern, this is what the method would looks like:

    @Test
    void testCreateTextNode_shouldDelegateToDecoratedDocument() {

        EasyMock.resetToDefault(this.documentMock);
        EasyMock.expect(this.documentMock.createTextNode(eq(TestData.TextNode.NAME))).andStubReturn(TestData.TextNode.NODE);
        
        final Text results = this.sut.createTextNode(TestData.TextNode.NAME);
        assertThat(results).isEqualTo(TestData.TextNode.NODE);
    }

While there is nothing fundamentally wrong with this approach, you can see how it distracts you from the goal of the test as you need to understand the framework syntax and focus on the how instead of the what.

Finally, using a builder type design pattern allows you to easily compose you mocks behaviors while factorizing the code into the builder. Don’t you find it weird that as a developer you tend to factorize code as much as possible, and that when it comes to mocking frameworks and writing unit tests, you end up with tons of copy paste between each test method?

Thank you for caring and for using that pattern as much as possible when dealing with mocks.

Extending AssertJ

The C4-PlantUML Maven Plugin makes use of AssertJ. You should be familiar with how to extend the AssertJ library in order to write better unit tests.

Unit testing

There is nothing much to explain there except that we usually extend a MOJO in the test bed to be able to disable Maven’s logging using SLF4J and avoid spamming the build and wasting precious CI minutes.

You can have an example of how to do that with the NoLogLintingMojo.

Integration testing

The integration testing test bed is a JUnit rule, built over Maven’s official test bed. Let’s have a look at the LintingMojoIT integration test.

First we notice the following lines:

...
@Rule
public final MavenVerifierRule<LintingMojoIT> verifierRule = new MavenVerifierRule<>(this);

@OutputDirectory
private Path outputDirectory;

private Verifier verifier;
...

The MavenVerifierRule injects the Verifier object, and the path to the Maven output directory.

Then, we notice the way tests methods are written:

...
@Test
public void shouldNotThrow_givenValidFilesAndActiveRules() {
}

@VerifierConfig(resetStreamsAtTheEnd = false, verifyErrorFreeLogs = false)
@Test
public void shouldThrow_givenDuplicatedRelationship() throws VerificationException {

    this.verifier.verifyTextInLog(TestData.ErrorMessage.DUPLICATED_RELATIONSHIPS);
    this.verifier.verifyTextInLog(TestData.ErrorMessage.BUILD_FAILURE);
    this.verifier.resetStreams();
}
...

The shouldNotThrow_givenValidFilesAndActiveRules test method seems to do nothing. This is because the JUnit rule takes care of everything and fails the test if any error message is reported during the Maven execution.

But how does the rule get to know what to test? It simple. It looks for a directory in the Maven test directory which is named after the test class and the test method: TEST_CLASS_NAME+ ‘_’ + TEST_METHOD_NAME.

So for the shouldNotThrow_givenValidFilesAndActiveRules test method, our directory should be named LintingMojoIT_shouldNotThrow_givenValidFilesAndActiveRules and be found somewhere in target/test-class.

Here is what that directory looks like in src/it/resources/....:

├── pom.xml
└── src
    └── main
        └── resources
            ├── c4l1__valid.puml
            ├── c4l2__valid.puml
            ├── c4l3__valid.puml
            └── empty_dictionary.dict

If we take a look at the pom.xml content, we can see that it simply reproduces a real use case:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.thewonderlemming.c4plantuml</groupId>
        <artifactId>c4plantuml-maven-parent</artifactId>
        <version>${project.version}</version>
    </parent>

    <artifactId>c4plantuml-maven-parent-LintingMojoIT-shouldNotThrow-givenValidFilesAndActiveRules</artifactId>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <itTest.outputDirectory>${project.build.testOutputDirectory}/org/thewonderlemming/c4plantuml/mojo/linting/it/LintingMojoIT_shouldNotThrow_givenValidFilesAndActiveRules/target/classes/</itTest.outputDirectory>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>${project.groupId}</groupId>
                <artifactId>c4plantuml-maven-plugin</artifactId>
                <version>${project.version}</version>
                <executions>
                    <execution>
                        <id>lint-files</id>
                        <goals>
                            <goal>lint</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <aliasesShouldBeListedInDictionary>
                                    <aliasDictionaryFilename>${itTest.outputDirectory}/empty_dictionary.dict</aliasDictionaryFilename>
                                </aliasesShouldBeListedInDictionary>
                                <noDuplicateRelationships>
                                    <isActivated>false</isActivated>
                                </noDuplicateRelationships>
                                <noOrphanAliasInBoundaries>
                                    <isActivated>false</isActivated>
                                </noOrphanAliasInBoundaries>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

With that tool, testing becomes dead-simple! Just beware of the execution time. You should keep these integration tests for non-regression testing when fixing bugs to ensure that they stay fixed amongst releases, and try to test as much as possible using unit tests.

Please also notice how we put Maven properties instead of their real value in that pom.xml file, to ensure that the test will always use the current version of the build no matter what.

We’ve seen how to check that a test does not display any Maven error message. What about passing only if a specific message is displayed instead? Well, this is what we do in the shouldThrow_givenDuplicatedRelationship test method:

static class TestData {

    static class ErrorMessage {

        ...
        static final String BUILD_FAILURE = "MojoFailureException";
        
        static final String DUPLICATED_RELATIONSHIPS = "Duplicate relationship found for <'src' -> 'dest': '\"Test\"'>";
        ...
    }
}
...
@VerifierConfig(resetStreamsAtTheEnd = false, verifyErrorFreeLogs = false)
@Test
public void shouldThrow_givenDuplicatedRelationship() throws VerificationException {

    this.verifier.verifyTextInLog(TestData.ErrorMessage.DUPLICATED_RELATIONSHIPS);
    this.verifier.verifyTextInLog(TestData.ErrorMessage.BUILD_FAILURE);
    this.verifier.resetStreams();
}

The @VerifierConfig annotation gives us the control on the test behavior. The verifyErrorFreeLogs parameter will define if the test should fail if any error message is raised in Maven’s logs, while the resetStreamsAtTheEnd defines if the logs should be cleared automatically.

We can then freely look for error messages using the verifyTextInLog method on the Verifier instance.