Test smells breakout at the server side java symposium

While I’ve read about this topic a lot, I wanted to come to do the topic to see how Lasse Koskela presents. [I’ve learned so much from him over the years as a moderator at coderanch ] He speaks in the same voice that he writes – casual and easy to follow. I really liked the format: present an anti-pattern and have the audience brainstorm what is wrong with it/how to improve it.  I also like his speaking style, gets away from the podium, engages with the audience, good pace, etc. And good use of humor to get participation going “I’m from Finland, we’re ok with silence”

“it’s a good thing that tests break,it means they are testing something that changes”. “as long as they don’t break for the wrong reasons”

Assertions

  • Primitive assertions – Custom assertions, assertStringContains
  • Overly broad assertion – like the golden master, but not intentional, dangerous because have to change if any small detail changes, more specific are preferred
  • Irrelevant assertions – assert not null and then assert a property on the non null object – let junit throw the null pointer

Organization

  • Split personality – Tests two things, many ways to see – var reuse, calling method under test twice, a blank line for two blocks of code
  • Split logic – don’t use information from elsewhere, it is a mystery to the reader of the test what is happening
  • Note: sometimes a refactoring makes things worse and you need to go back or explore another solution.  Better to fix the problem with the refactored code because it is likely better than the original.

Transparency – how readable is it

  • Setup sermon – Extract overly long/complex setup methods, revisit design to get file system out of the picture, avoid magic numbers “why four”
  • Use multiple setup methods to make clear what is being setup
  • Big string – building an XML file in java is bad.  Instead of. Putting in external file, could create method that takes parameters and builds the XML file. This helps because can clearly deduce the important info without reading through reams of XML.

Maintainability

  • Absolute file path – Don’t depend on the environment, platform specific makes harder. For files could use a temp directory that you wipe clean. That way there isn’t test residue
  • Also, relying on path messes up paths when checking out from a branch, use relative path instead. Or getResourceAsStream()
  • Pixel perfection – a lot of assertions for each pixel in an image, any small change breaks the test even if not meaningful (like golden master pattern). One approach is to make the burden of maintaining less. Grab good image and compare to new image. Tis makes it obvious that it a golden master and requires visual inspection. Another way is to fix the test to use the proper language in the assertions.  that tests the lines are connected without testing the pixels directly  To implement, write custom matcher to test pixels by relative position.
  • Lasse feels adding assertion messages makes the tests harder to read and doesn’t add clarity.

I like how Lasse showed the benefits of looking at your old code. It shows that you’ve learned a lot since then.

I like that some of the examples were in languages other than java. I also learned a lot!  Sorry Scott: I wanted to be at your session, but this one was awesome!

junit suite wide setup (not @BeforeClass)

Problem

How do I run setup/teardown for my whole suite, not just one class/test?

Impact of Problem

JUnit offers @Before to run a method before each test and @BeforeClass to run a method once before a class.  For a small suite, @BeforeClass may be enough.  For a large integration test suite, setting up the database is a common task that should be done once for scores of tests.  Having everything in one class is not optimal.  Nor is manually listing all the tests.

Requirements

  1. Continue being able to use ClasspathSuite to gather tests in a subdirectory at runtime.
  2. Run a method before the first of these tests is run
  3. Run a method after the last of these tests is run
  4. Take advantage of JUnit’s runner for running all the tests

Solution

package com.javaranch.test.functional;

import static org.junit.extensions.cpsuite.SuiteType.*;

import org.junit.extensions.cpsuite.*;
import org.junit.extensions.cpsuite.ClasspathSuite.*;
import org.junit.runner.*;

// use cpsuite to dynamically list out tests
@RunWith(ClasspathSuite.class)
@ClassnameFilters( { "com.javaranch.*test.*Test", "net.jforum.*test.*Test" })
// include in case have a parameterized test case
@SuiteTypes( { RUN_WITH_CLASSES, TEST_CLASSES, JUNIT38_TEST_CLASSES })
public class All_JForum_Functional_Tests {
public static void main(String[] args) throws Exception {
setUp();
JUnitCore.main("com.javaranch.test.functional.All_JForum_Functional_Tests");
tearDown();
}

private static void setUp() {
// create the database
}

private static void tearDown() {
// destroy the database
}
}

If you’ve been reading my blog, you’ll know I am doing this to integration test the back end of JavaRanch’s JForum install.  I started by cloning a database and have now moved on to the integration test framework.  Next I will describe the actual database setup.

Refactoring JUnit 3.8 to 4.0 when hierarchy extends TestCase

Problem:

I want to start writing tests in JUnit 4.0, but I have a lot of tests in JUnit 3.8.  I can’t just start writing tests in 4.0, because I rely on common setup/assertions in my custom superclass which extends TestCase. (Which means JUnit will only look for 3.8 style tests)

Solution:

Create one or more new classes to contain static method equivalents that you need.  The new code can use static imports to get these methods.  The original abstract class can delegate to them.

Limitations:

Some frameworks require you extend a certain class.  Until the framework provides a non-JUnit 3.8 dependency, there’s not much you can do here.  You can still use this approach for tests that don’t require that special framework.

UML for refactoring:

Sample code:

package com.javaranch.asserts;

import static com.javaranch.asserts.JavaRanchTestUtil.*;
import junit.framework.*;

public abstract class AbstractJavaRanchTestCase extends TestCase {
   @Override
   public void setUp() throws Exception {
      super.setUp();
      setUp_propertyConfiguration();
   }
}

Conclusion:

I’ve done this refactoring enough times that is second nature by now.  This blog post documents the refactoring since I’m not finding it on the web.

What other problems/sticky situations have you encountered when mixing JUnit 3.8 and 4.0 code?