This is all great ; but when you’re in the middle of a task and you write unit-tests, it’s quite tedious to run the build and then open the HTML coverage report just to monitor your code coverage. I spend about 80% of my time for a task in Eclipse : opening and activating tasks with Mylyn, coding, writing unit-tests, running unit-tests and so on. This is why for me it made sense to seek out a way to monitor the code coverage in Eclipse.

After a bit of googling, I found a Java code coverage plug-in for Eclipse : EclEmma. As you can see from its name, it’s based on the EMMA Java code coverage tool. Here’s a list with its main features :

• a coverage mode in which applications launched or unit tests are instrumented and measured
• coverage overview : a coverage view containing a report on the source code coverage values at project-level, package-level and class-level
• source highlighting in the Java code editor using customizable colors
• customizable coverage counters
• multiple coverage sessions and session merging
• importing EMMA coverage data files
• exporting to EMMA coverage data, XML and HTML

The easiest way to install it is through the update site : http://update.eclemma.org. After the installation, you will notice a new launch mode appearing in the Eclipse toolbar, called Coverage, similar to the Run and Debug modes. This new mode allows you to run coverage reports on applications or unit-tests just like you would before run those applications or unit-tests.

In a project I’m currently working on, our server-side unit tests are written using TestNG. From Eclipse, I can run one or multiple TestNG units using the Eclipse TestNG plug-in, so I can easily verify that my code passes the unit tests. I have defined a launch configuration for each server-side project which runs all of the unit-tests. To check out the code coverage for those tests, all I have to do is create a coverage configuration and make sure that I select the source code folders to be instrumented.

I created a sample Java-Maven-TestNG project and added to it a simple class called ShoppingCartImpl along with a TestNG test class. Here’s how it looks :

Source code of a very basic shopping cart

As you can see, this is a very basic class. Now on to configuring the coverage settings ; this is a simple matter of clicking on the Coverage button in Eclipse’s toolbar and selecting the menu option Coverage Configurations…. This opens up the coverage configuration window, as seen below :

Creating a new EclEmma coverage configuration

All I have to do is select the TestNG test suite that I want to run and check the source folders that are relevant to code coverage. I’m using Maven and all my source code is in src/main/java so I only select that folder. I click Apply and then I can finally run the coverage report. I click on the Coverage button and theTestNG configuration is executed and the coverage report is available :

Code coverage report after a first run

In the bottom of the Eclipse window, you get a clear picture of the code coverage. As you can see, the are reports at project, package and class level, which also show up in the package explorer, in the left. To enable the decorators in the package explorer, go to the Eclipse menu and select Preferences -> General -> Appearance -> Label decorations, then make sure that the Java Code Coverage label decoration option is checked.

Another interesting feature of EclEmma is that after the code coverage instrumentation you can actually see the coverage in the source code. As you can see, each line relevant to the coverage report is marked with a color. Green is for 100% branch coverage, yellow is for some branch coverage and red for no coverage at all. The shopping cart has quite a low code coverage so I did shape it up. After a bit of fiddling with the code, I get a 100% code coverage and a very nice report :

The ShoppingCart class with 100% code coverage

As you can see, the architecture is quite simple. We have a layer of service API which is implemented by another layer, a LCDS-based implementation which, in turn, uses an implementation of a DAO layer. In the diagram, I grayed out the DAO layers because they are of no interest to our current subject.
Briefly, the service API only contains the interfaces of the services along with related objects : exceptions and DTOs.

We use our custom business exceptions to signal to service clients any issues encountered during service operations. Each custom exception has a public code which indicates its nature and meaning.

Simple exception mechanism

At first, we decided to simply throw business exceptions from the service implementations. This meant that the Flex client application would receive a fault event, which it had to strip to get to the actual exception.
Here is the Java service exception and method :

public class InvalidCriteriaException extends Exception
{
    ...
    public String getCode()
    {
        return ExceptionCode.INVALID_CRITERIA.toString();
    }
}
 
public SearchResultDto searchById(SearchCriteriaDto criteria) throws InvalidCriteriaException
{
    try {
        // do some processing, use the DAO layer
        ...
    }
    catch (SomeDaoException e)
    {
        // convert the DAO exception into a service exception and then throw the new exception
        ...
    }
    catch (Throwable e)
    {
        // any unexpected exception is caught and a new service exception is created and thrown further
        ...
    }
}

And the corresponding Flex code :

    private function handleException(event:FaultEvent) : void {
        var errorMessage:ErrorMessage = event.message as ErrorMessage;
        Alert.show(errorMessage.rootCause.code);
    }

As you can see, the Java service code is cluttered within a try..catch block, which also contains the details on converting possible exceptions to service exceptions. On top of this, the client-side code is not very clean either, because it uses an untyped object ( rootCause ) on which we make assumptions and assumptions are generally a bad thing to do in your code. The right solution on the client-side would be to take advantage of the properties of the ErrorMessage object.

A better exception mechanism

In order to leverage the properties of the Flex class ErrorMessage, we decided to only throw instances of flex.messaging.MessageException, which is shipped in LCDS (and BlazeDS). To do this in an easy way, we proceeded in making sure that each of our custom exception would inherit MessageException :

public class InvalidCriteriaException extends MessageException
{
    ...
}

The service layer would now throw only MessageException-derived exceptions, which get deserialized on the client-side as instances of ErrorMessage. The Java exception has a field called code, who is translated in the Flex class as the field faultCode. So the only thing left to do is make sure that the code field is set to a proper value on the server side.

Using Spring BlazeDS Integration and a custom exception translator

The solution described in the previous paragraph still comes with some flaws. First of all, we didn’t get rid of the try..catch block in the service code. Uh, ugly.
Second, there is a design issue : the service API layer is exposing the service interfaces along with the exceptions, which are now derived from a LCDS exception. This makes the service API layer dependent on LCDS, which is not what we want. The service API should be clean and free of any implementation-specific dependencies.

These flaws were removed once we switched to using the Spring BlazeDS Integration to expose our Spring-based services as LCDS destinations. Along with other great features, Spring BlazeDS Integration comes with the notion of custom exception translators. The translator makes sure that exceptions thrown from the Spring-exposed LCDS destinations are converted to meaningful exceptions for the client, all of this far away from the service code which will be far more simple and clean.

First of all, we create our own exception translator :

public class ExceptionTranslatorImpl implements ExceptionTranslator
{
    public boolean handles(final Class<?> clazz)
    {
        return Boolean.TRUE;
    }
 
    public MessageException translate(final Throwable throwable)
    {
        final MessageException exception = new MessageException();
        exception.setCode(ExceptionCode.SYSTEM.name());
 
        if (throwable != null)
        {
            if (throwable instanceof BaseCustomException)
            {
                exception.setCode(((BaseCustomException) throwable).getCode().name());
            }
 
            exception.setRootCause(throwable);
            exception.setMessage(throwable.getMessage());
        }
 
        return exception;
    }
}

Then we need to register it in the application context :

  	<bean id="allExceptionTranslator" class="com.adobe.myproject.exception.ExceptionTranslatorImpl" />
 
	 <flex:message-broker services-config-path="/WEB-INF/flex/services-config.xml"  >
        <flex:exception-translator ref="allExceptionTranslator"/>
		<flex:message-service/>
		<flex:secured />
	</flex:message-broker>

We can now remove all the exception conversion logic from the service code and let the exception translator handle all of these ugly details for us. Much better, isn’t it ?

How do I do it ?

First, stop calling every developer test a unit test.

Second, categorizing your formerly-known-as-unit-tests tests means you separate them into different layers based on external dependencies, complexity, time of execution. There are 3 layers of developer tests : unit tests, component tests, system tests.

Categories

Unit tests

Unit tests are run against objects without any external dependencies. Any external dependency is mocked and the unit test concentrates only on the object you wrote. This is why unit tests are usually simpler to write and take less time to execute.

Component tests

Component tests validate multiple objects with external dependencies and their interaction. This means that you deal with databases, file systems, HTTP connections and so on, and with the interaction between your objects (the component) and the external systems. Component tests tend to be more complex than unit tests, they take more time to write and more time to execute.

System tests

When you want to test your application from one end to the other, you write system tests. They verify your application as if an user would use the application, that is they mimic one.

I have my categories, now what ?

Now it’s time to update your build process.

Update the build process

You have 3 categories of developer tests : unit test, component tests and system tests. Because unit tests run fast, you can choose to run them during each build and during each CI build. In contrast, since component tests and system tests take much more time to execute, you can choose to skip them during a regular CI build and run them less often.

Setting up a staged build

In his articles about Continuous Integration, Martin Fowler suggests to keep the build fast by setting up a staged build.
The basic idea is that by setting up a staged build (a 2 phases build) you can achieve a compromise between execution time and testing thoroughness. The developers need a feedback on their commit, so the testing process needs to be as thorough as possible, but the feedback must be delivered as quickly as possible because you don’t want to keep the developers waiting while the build is running.

To accomplish this, you can set up two builds : a commit build – the build that everyone executes before committing code – and a secondary build – a full build.

The commit build

The commit build has to be run by everyone before committing code. During this build, only unit tests are executed, so the build doesn’t take a lot of time ; on the other hand, this also means that the application is not tested at a higher level.
This build is also run by the CI system after each commit.

The secondary build

The secondary build is a full build, it runs all of the tests – unit tests, component tests, system tests. This means it takes a lot longer to complete, but on the other hand it gives you a complete feedback on your application.

The secondary build is run by the CI system. Depending on the time it takes to complete the secondary build, you can have the CI system to run it after each successful commit build or at regular time intervals.

Categorizing tests in Java

I use TestNG to write developer tests for my Java code. The reason for this is that it has some features that I haven’t found elsewhere yet, so I stick to it. One of these features is, coincidentally, the ability to define and use test groups. At its core, it’s all about being able to define groups of tests and to run one or more groups of tests during a testing session or during a build.

The approach I took on a project was to write the tests using TestNG and to divide them into three TestNG test groups, corresponding to the test categories I described earlier. First, I declared some constants for the groups :

public static final String GROUP_UNIT_TEST = "unit-test";
public static final String GROUP_COMPONENT = "component";
public static final String GROUP_SYSTEM = "system";

Then whenever I added a test method, I would assign it to a test group. For instance, unit tests :

@Test(groups = { GROUP_UNIT_TEST })
public void testAdaptNullList() {
...
}
@Test(groups = { GROUP_UNIT_TEST })
public void testAdaptList() {
...
}

Component tests would look very similar :

@Test(groups = { GROUP_COMPONENT })
public void testSqlConnection() {
...
}
@Test(groups = { GROUP_COMPONENT }, dependsOnMethods = { "testSqlConnection" })
public void testGetTargetData() {
...
}

In the code block above, you can notice another nice feature of TestNG : dependent methods. In the test above, the test method testSqlConnection is run first. If it fails, the dependent method testGetTargetData is skipped.
With TestNG’s dependent methods mechanism, you can have some methods from your tests depend on other methods to make sure a certain number of test methods have completed and succeeded before running more test methods. This feature makes sure you don’t waste time running certain tests which will fail for sure if other tests already failed.

And now for some system tests :

@Test(groups = { GROUP_SYSTEM })
public void testGetFilters() {
...
}
 
@Test(groups = { GROUP_SYSTEM }, dependsOnMethods = { "testGetFilters" })
public void testGetFiltersPeriods() {
...
}
 
@Test(groups = { GROUP_SYSTEM }, dependsOnMethods = { "testGetFilters" })
public void testGetFiltersCallCenter() {
...
}

Once the tests are in place, I can configure the builds accordingly. I can configure the commit build, based on Maven, to run only unit tests :

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
      <groups>unit-test</groups>
    </configuration>
  </plugin>

The secondary build, on the other hand, can run all the tests :

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
      <groups>unit-test,component,system</groups>
    </configuration>
  </plugin>

Easy, right ? TestNG is integrated nicely with Maven using the Maven Surefire Plugin.
TestNG is also integrated with Eclipse so that you can run tests or groups of tests from Eclipse while you’re coding. If you’re an Ant guy, check out the TestNG Ant task. And another thing if you’re an Ant guy : switch to Maven.