## Testing

Testing is a tedious but necessary task when writing software. It requires a lot of time as the tests have to be maintained, in order to be useful. Which implies that they have to be easily updated/fixed or they will become a hassle.

The C4-PlantUML Maven Plugin project provides various testing utility JARs (referred to as test beds), to help you with the testing.

These test beds heavily rely on JUnit for testing (JUnit Jupiter API for the library layer, and JUnit Vintage API for the MOJO layer).

##### What you will learn by reading that page:

- Testing the libraries
  - Test cases
  - Mock helper design pattern
  - Extending AssertJ
- Testing the MOJOs
  - Unit testing
  - Integration testing

### Testing the libraries

The library layer test beds are distributed amongst the following dependencies:

Dependency name | Description
--- | ---
c4plantuml-commons-lib-1.0.0-RC1-testbed.jar | Contains shared testing utility classes
c4plantuml-grammars-lib-1.0.0-RC1-testbed.jar | Contains ANTLR Grammar testing utility classes
c4plantuml-graphml-lib-1.0.0-RC1-testbed.jar | Contains GraphML testing utility classes
c4plantuml-linter-lib-1.0.0-RC1-testbed.jar | Contains Linting testing utility classes
c4plantuml-syntaxchecker-lib-1.0.0-RC1-testbed.jar | Contains Syntax checking testing utility classes
c4plantuml-testingutils-lib-1.0.0-RC1.jar | Contains base classes for testing

We will go through some of their features, below.

#### Test cases

Test cases are interfaces which implements JUnit 5 Test Factories (tests generators). Each test case is built around one single use case, and is implemented using Java 8’s interfaces (with their `default` feature), in order to simulate multiple inheritance.

Let’s have a look at the NoOrphanAliasInLayoutsRuleTest test class to get a better understanding. As a linting rule, the NoDuplicateRelationshipsRule rule has to implement at least the createParseTreeListener method, and could override the ones implemented in its parent class. Each of these methods should be tested, and there is a dedicated test case for each of them.

For instance, CreateParseTreeListenerLintingRuleTestCase tests the behavior of the createParseTreeListener method: let’s see how. At first, the test class needs to provide a instance to test, using the @SUT annotation. It can be placed on a field or a method, and the rule will pick the first (alphabetical order) annotated field, or else the first annotated method (alphabetical order), as the SUT instance (System Under Test).

The test case will then look for a specific annotation on the test class (here it will look for the @CreateParseTreeListener). If the annotation is not present, the test case won’t even be triggered.

You can of course combine test cases with the JUnit `@Nested` feature, in order to provide multiple SUT with multiple configurations.

Below is a list of the existing test cases so far, along with their usage. Feel free to browse the Javadoc of the test bed JARs in order to learn more about them.

Test case name | ArtifactId | Description
--- | --- | ---
AcceptableParsersTypesLintingRuleTestCase | c4plantuml-linter-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to generate tests that verifies that the AbstractLintingRule.acceptableParsersTypes() behaves as expected, by providing test input through the AcceptableParsersTypes annotation, having no extra code to write.
CheckSyntaxTestCase | c4plantuml-syntaxchecker-lib-1.0.0-RC1-testbed.jar | Gathers CheckSyntax annotations on the test class instance and generates a list of DynamicTest to check the syntax of a given file and compare the results to a list of expected messages.
CreateParseTreeListenerLintingRuleTestCase | c4plantuml-linter-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to generate tests that verifies that the AbstractLintingRule.createParseTreeListener(Reporter, Class) behaves as expected, by providing test input through the CreateParseTreeListener annotations, having no extra code to write.
LintSourceFileLintingRuleTestCase | c4plantuml-linter-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to generate tests that verifies that the Linter.lint(Path, Charset) method behaves as expected given the current rule, by providing test input through the LintSourceFile annotations, having no extra code to write.
ParseC4ToGraphMLTestCase | c4plantuml-graphml-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to generate tests that verifies that the C4Graph processes C4 files and exports the content properly, generating valid GraphML that matches the expected one - by providing test input through the ParseC4ToGraphML annotation, having no extra code to write.
SelectParseTreeLintingRuleTestCase | c4plantuml-linter-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to generate tests that verifies that the AbstractLintingRule.selectParseTree(Parser) behaves as expected, by providing test input through the SelectParseTree annotations, having no extra code to write.
ValidateGrammarTestCase | c4plantuml-graphml-lib-1.0.0-RC1-testbed.jar | A JUnit 5 test case to retrieve sample files from a given directory and to validate these files against the given grammar Lexer and Parser.

Finally, you may want to have a look at the way rules linting messages are tested.

#### Mock helper design pattern

The `mock helper` design pattern is a specialization of the `builder` design pattern and aims to increase the tests readability and maintainability, by factorizing the mock frameworks boiler code.

Let’s study a simple test using the EasyMock mocking framework (the same could be achieved with Mockito):

The following code is a sample from the unit test:

    @InjectMocks
    class CDataDocumentDecoratorTest {

        static class TestData {

            ...
            static class Export {

                static final boolean INDENT = true;

                static final boolean STRICT_VALIDATION = true;

                static final boolean VALIDATE = true;
            }

            static class TextNode {

                static final String NAME = "someNodeName";

                static final Text NODE = MockType.NICE.createMock(Text.class);
            }
            ...
        }

        ...
        @Mock
        Document documentMock;
        ...

        @AfterEach
        void tearDown() {
            verify(this.documentMock);
        }
        ...
        @Test
        void testCreateTextNode_shouldDelegateToDecoratedDocument() {

            setUpStrictMock(this.documentMock)
                .toCreateTextNodeGiven(TestData.TextNode.NAME)
                    .andReturn(TestData.TextNode.NODE)
                    .setUpDone();

            final Text results = this.sut.createTextNode(TestData.TextNode.NAME);
            assertThat(results).isEqualTo(TestData.TextNode.NODE);
        }
        ...
    }

You can notice an annotated `documentMock` field, which will have its value instantiated and injected at runtime thanks to the use of the `@InjectMocks` JUnit custom extension.

The `testCreateTextNode_shouldDelegateToDecoratedDocument` focuses on what should the mock do, instead of how. If we should have to rewrite the test without the mock helper pattern, this is what the method would looks like:

        @Test
        void testCreateTextNode_shouldDelegateToDecoratedDocument() {

            EasyMock.resetToDefault(this.documentMock);
            EasyMock.expect(this.documentMock.createTextNode(eq(TestData.TextNode.NAME))).andStubReturn(TestData.TextNode.NODE);

            final Text results = this.sut.createTextNode(TestData.TextNode.NAME);
            assertThat(results).isEqualTo(TestData.TextNode.NODE);
        }

While there is nothing fundamentally wrong with this approach, you can see how it distracts you from the goal of the test as you need to understand the framework syntax and focus on the how instead of the what.

Finally, using a builder type design pattern allows you to easily compose you mocks behaviors while factorizing the code into the builder. Don’t you find it weird that as a developer you tend to factorize code as much as possible, and that when it comes to mocking frameworks and writing unit tests, you end up with tons of copy paste between each test method?

Thank you for caring and for using that pattern as much as possible when dealing with mocks.

#### Extending AssertJ

The C4-PlantUML Maven Plugin makes use of AssertJ. You should be familiar with how to extend the AssertJ library in order to write better unit tests.

### Testing the MOJOs

The MOJOs testing is split over two artifacts:

- org.thewonderlemming.c4plantuml:c4plantuml-maven-plugin
- org.thewonderlemming.c4plantuml:c4plantuml-maven-plugin-it

The first one contains unit tests while the second one contains integration testing.

#### Unit testing

There is nothing much to explain there except that we usually extend a MOJO in the test bed to be able to disable Maven’s logging using SLF4J and avoid spamming the build and wasting precious CI minutes.

You can have an example of how to do that with the NoLogLintingMojo.

#### Integration testing

The integration testing test bed is a JUnit rule, built over Maven’s official test bed. Let’s have a look at the LintingMojoIT integration test.

First we notice the following lines:

    ...
    @Rule
    public final MavenVerifierRule<LintingMojoIT> verifierRule = new MavenVerifierRule<>(this);

    @OutputDirectory
    private Path outputDirectory;

    private Verifier verifier;
    ...

The `MavenVerifierRule` injects the `Verifier` object, and the path to the Maven output directory.

Then, we notice the way tests methods are written:

    ...
    @Test
    public void shouldNotThrow_givenValidFilesAndActiveRules() {
    }

    @VerifierConfig(resetStreamsAtTheEnd = false, verifyErrorFreeLogs = false)
    @Test
    public void shouldThrow_givenDuplicatedRelationship() throws VerificationException {

        this.verifier.verifyTextInLog(TestData.ErrorMessage.DUPLICATED_RELATIONSHIPS);
        this.verifier.verifyTextInLog(TestData.ErrorMessage.BUILD_FAILURE);
        this.verifier.resetStreams();
    }
    ...

The `shouldNotThrow_givenValidFilesAndActiveRules` test method seems to do nothing. This is because the JUnit rule takes care of everything and fails the test if any error message is reported during the Maven execution.

But how does the rule get to know what to test? It simple. It looks for a directory in the Maven test directory which is named after the test class and the test method: `TEST_CLASS_NAME`+ ‘_’ + `TEST_METHOD_NAME`.

So for the `shouldNotThrow_givenValidFilesAndActiveRules` test method, our directory should be named `LintingMojoIT_shouldNotThrow_givenValidFilesAndActiveRules` and be found somewhere in `target/test-class`.

Here is what that directory looks like in `src/it/resources/....`:

    ├── pom.xml
    └── src
        └── main
            └── resources
                ├── c4l1__valid.puml
                ├── c4l2__valid.puml
                ├── c4l3__valid.puml
                └── empty_dictionary.dict

If we take a look at the `pom.xml` content, we can see that it simply reproduces a real use case:

    <project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>

        <parent>
            <groupId>org.thewonderlemming.c4plantuml</groupId>
            <artifactId>c4plantuml-maven-parent</artifactId>
            <version>${project.version}</version>
        </parent>

        <artifactId>c4plantuml-maven-parent-LintingMojoIT-shouldNotThrow-givenValidFilesAndActiveRules</artifactId>

        <properties>
            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
            <itTest.outputDirectory>${project.build.testOutputDirectory}/org/thewonderlemming/c4plantuml/mojo/linting/it/LintingMojoIT_shouldNotThrow_givenValidFilesAndActiveRules/target/classes/</itTest.outputDirectory>
        </properties>

        <build>
            <plugins>
                <plugin>
                    <groupId>${project.groupId}</groupId>
                    <artifactId>c4plantuml-maven-plugin</artifactId>
                    <version>${project.version}</version>
                    <executions>
                        <execution>
                            <id>lint-files</id>
                            <goals>
                                <goal>lint</goal>
                            </goals>
                            <configuration>
                                <rules>
                                    <aliasesShouldBeListedInDictionary>
                                        <aliasDictionaryFilename>${itTest.outputDirectory}/empty_dictionary.dict</aliasDictionaryFilename>
                                    </aliasesShouldBeListedInDictionary>
                                    <noDuplicateRelationships>
                                        <isActivated>false</isActivated>
                                    </noDuplicateRelationships>
                                    <noOrphanAliasInBoundaries>
                                        <isActivated>false</isActivated>
                                    </noOrphanAliasInBoundaries>
                                </rules>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </project>

With that tool, testing becomes dead-simple! Just beware of the execution time. You should keep these integration tests for non-regression testing when fixing bugs to ensure that they stay fixed amongst releases, and try to test as much as possible using unit tests.

Please also notice how we put Maven properties instead of their real value in that `pom.xml` file, to ensure that the test will always use the current version of the build no matter what.

We’ve seen how to check that a test does not display any Maven error message. What about passing only if a specific message is displayed instead? Well, this is what we do in the `shouldThrow_givenDuplicatedRelationship` test method:

    static class TestData {

        static class ErrorMessage {

            ...
            static final String BUILD_FAILURE = "MojoFailureException";

            static final String DUPLICATED_RELATIONSHIPS = "Duplicate relationship found for <'src' -> 'dest': '\"Test\"'>";
            ...
        }
    }
    ...
    @VerifierConfig(resetStreamsAtTheEnd = false, verifyErrorFreeLogs = false)
    @Test
    public void shouldThrow_givenDuplicatedRelationship() throws VerificationException {

        this.verifier.verifyTextInLog(TestData.ErrorMessage.DUPLICATED_RELATIONSHIPS);
        this.verifier.verifyTextInLog(TestData.ErrorMessage.BUILD_FAILURE);
        this.verifier.resetStreams();
    }

The `@VerifierConfig` annotation gives us the control on the test behavior. The `verifyErrorFreeLogs` parameter will define if the test should fail if any error message is raised in Maven’s logs, while the `resetStreamsAtTheEnd` defines if the logs should be cleared automatically.

We can then freely look for error messages using the `verifyTextInLog` method on the `Verifier` instance.