JUnit best practices - How to write good junit test cases?

Writing unit test cases1 are the part of any development project process. But developers rarely follow process. For us, we wanted to code something and see it to work on what we have fixed and finally move it to production.When it works, we are happy with it. But wait, when it doesn’t work? Then comes a lot of head ache for developer. Ok stories apart, lets get back to work. I’m not going to discuss about how to write unit testcases, but rather discuss how to write some better unit testcases using JUnit and avoid some of the pitfalls that we developers falls. Since JUnit is based on xUnit testing framework, any xUnit based testing framework for the language you use like NUnit, CppUnit, etc.,

1 Write one test-condition per test method

This is first important point. Always, write only one test-condition3 in one test method. Do not group more than one test case in a single test-method. Consider the following method.
public boolean validateZipCode(String zipCode)
{
    Matcher m = Pattern.compile("\d{5}").matcher(zipCode);
    return m.matchers();
}
You can think of following test cases
  1. Input is NULL
  2. Input is Blank
  3. Input is Alpha
and many other test cases etc.,
public void testValidateZipCode()
{
    boolean output = validateZipCode(null);
    assertFalse(output);

    boolean output = validateZipCode("");
    assertFalse(output);

    boolean output = validateZipCode("    ");
    assertFalse(output);
}
When you run the above test case, the JUnit will fail in the first assestion. And it will not proceed to next the next test case. So, only if you fix the 1st failure, you will know the 2nd failure (if any). This is bad, because as humans we try to figure out as much issues during every shot.
public void testValidateZipCode1()
{
    boolean output = validateZipCode(null);
    assertFalse(output);
}

public void testValidateZipCode2()
{
    boolean output = validateZipCode("");
    assertFalse(output);
}

public void testValidateZipCode3()
{
    boolean output = validateZipCode("    ");
    assertFalse(output);
}
Now this helps in identifying all the existing issue in single run. Next, you can measure how many test case you have executed, which helps in how many are success and failed.

2 When testing output of Collections(List, Set) test for Null and Emptiness

Methods returning collections should be tested for more conditions.
  1. Test for NULL
  2. Test for emptiness
  3. Test for list not returning null ( if requirements allow )
  4. Test for object are of same type, if your requirement is so ( applicable for Java 1.4 )

3 Use java reflection for adding new methods into test suites

When you write a separate suite to execute the test cases in order, you might feel pain in the neck when you try to add every testXXX() method to the suite, esp. when your Test has more than 10 test conditions for each method and you are testing a class which has more than 10 methods. A better way to deal with this is to use Reflection.
class ZipCodeTest extend TestSuite
{

    public void testSuite()
    {

    }

}
I would recommend you to read the JUnit FAQ to good a good understanding of JUnit tool

4 Compare the objects after performing an operation

It is recommended to compare the objects after performing an operation.Consider this.
class Engine
{

    int state, fuel = 100, timeToRun;

    public void run(int minutes)
    {
            this.timeToRun = minutes;
            this.start();
            this.fuel -= minutes * 1;
            this.stop();
    }

    public void start()
    {
            this.state = 1;
    }

    public void stop()
    {
            this.state = 0;
    }

    public boolean equals(Object o)
    {
            Engine e = (Engine) o;
            return this.state == e.state && this.fuel == e.fuel;
    }

    public int hashCode()
    {
            return this.fuel;
    }

}

public void testEngineStatusAfterRun()
{
    Engine e = new Engine();
    e.run(5);

    /* less recommended */
    assertTrue(0 == e.state);

    /* recommended */
    Engine expected = new Engine();
    testCasse.state = 0;
    testCasse.fuel = 95;
    assertEquals( expected, e);

}
Why testing the entire objecti is recommended? The main reason is, class methods have ability to modify the class variables internally without your knowdlege or you might miss to see it. So, always compare the entire object to what you expect.

5 Test the exceptions

Most of the developers who write JUnit will forget / fail to test the exceptions thrown from the methods. Right exceptions thrown always convey it's inherit information. Say you have a exception designed as below
class CarEngineException extends Exception
{
    int engineErrorCode;
    String message;

    public void runEngine() throws CarEngineException
    {

        if(someCondition1)
        {
                List l = someOtherProcess();
                if(l.isEmpty())
                {
                    CarEngineException ex = new CarEngineStartException();
                    ex.setMessage("some message for failure");
                    ex.setEngineErrorCode(1);
                }
                else if(l.size() == 1)
                {
                    CarEngineException ex = new CarEngineStartException();
                    ex.setMessage("some message for failure");
                    ex.setEngineErrorCode(1);
                }
         }
    }
}
In the above case, exceptions are vital part of the method's response. One should test the method's exception, else you are not performing complete testing.

6 Group logical test cases into test suites

Finally, once you have written as many test cases as possible, try to group them into logical collections. So that when you change a piece of code in a module, you would run only the logical test suite.
public double power(double num, int exp)
{
    ...
}

public double sqr(double num)
{
    return pow(num,2);
}

public double cube(double num)
{
    return pow(num,3);
}

public double exp(double raise)
{
    ...
}
Here are you test methods:
class MathTest
{

    ...

    public void testPow()
    {
        ...
    }

    public void testSqr()
    {
        ...
    }

    public void testCube()
    {
        ...
    }

    public void testExp()
    {
        ...
    }

}
Whenever you change something to exp() or pow() you might want to execute MathTest and which inturn will execute all the test cases, which are not necessary. In this situation, you can create you own test suite, which will just execute the method you wanted to execute.
class TestPowMethods
{

    public static Test powMethodSuite()
    {
        TestSuite suite = new TestSuite();
        suite.addTest(new FunctionsTest("testPow"));
        suite.addTest(new FunctionsTest("testSqr"));
        suite.addTest(new FunctionsTest("testCube"));

        return suite;
    }

    public static void main(String[] args)
    {
        /*this will execute just pow() dependent methods */
        TestRunner.run(powMethodSuite());
    }

}

7 Check for order of execution of test cases

Always write a seperate test suite, when your module requires an order of execution of test cases, i.e run `test1..test5` before you start other tests, then you can write a seperate TestSuite in the same way as expalined in the above topic. This will give you a better hold of executing the test cases in order.

8 Always include a message to assertion

More the messages, less your debug time. Providing as many details as possbile is always good for debugging. All the assertXXXX() methods in JUnit framework has an optional parameter a `String message`. But just giving a message isnt a good thing.
assertEquals("Error code should be 100", 100, e.getErrorCode() );
When your above assertion fails, you will get a message Error code should be 100. From this you can conclude that you havent received error code 100. But you are not sure what code have you received. If, your method returns 4 to 5 error codes, you might need to know which conditional part have execute and what error code has been received. Hence write you message wisely.
assertEquals("Error code should be 100. Received = " + e.getErrorCode(), 100, e.getErrorCode() );
Now this will print the received errorCode and you have a better chance of understanding what have happened.

9 Know when to use assertEquals and assertSame

In JUnit package you might find few methods which sound similar. assertEquals and assertSame. Both have a subtle difference2. assertSame(a,b): does a == b. i.e compare both objects instances are same. assertEquals(a,b): does a.equals(b), if equals() method has been overriden. Else it invokes assertSame()