setup for HOL-1695 – Starting Out with JUnit 5 at JavaOne

I’ll be leading a hands on lab “Starting Out with JUnit 5” at JavaOne. This is a BYOL (bring your own laptop) lab.

In order to follow the lab, you’ll need either a recent Eclipse or IntelliJ. Or you can use the command line and your favorite editor if you usually code that way. If you aren’t very familiar with IntelliJ, I recommend using Eclipse. Both lab instructors use Eclipse primarily so we will be able to help you more that with IntelliJ.

If you run into any trouble setting up for the lab, you can post a comment on this blog post or start a thread in the Testing or IDEs forum at CodeRanch.

(NetBeans does not yet support JUnit 5 which is why it isn’t an option)

Option 1: Eclipse

  1. Download and install Eclipse Oxygen
  2. From Eclipse marketplace, install the JUnit 5 beta support.

Option 2: IntelliJ

  1. Download and install IntelliJ  2017.2.1 or later. (2016.2 or later will probably work, but I haven’t tested). JUnit 5 is included.

Option 3: Maven and your choice of Java editor

  1. Install Maven if you don’t already have it.
  2. Make sure you are using Java 8. (This lab was not tested with Java 9).
  3. Your favorite Java editor is presumably already installed :).

Optional – Git

  • One of the steps will be to pull code from git. If you have a git command line (or git bash), this is a one line operation. If not, you’ll be able to download a zip file from github so this isn’t a hard requirement.

upgrading mockito – forwards and backwards

I upgraded Mockito from 1.X to 2.X. (Because I wanted to try out the JUnit 5 MockitoExtension.

JUnit 4

I like @RunWith(MockitoJUnitRunner.class). The two main benefits:

  • Inject mocks as instance variables instead of static call to MockitoAnnotations.initMocks(this);
  • Tells you about unnecessary mock setup calls you have.

JUnit 5

Instead of the Mockito Runner, you use @ExtendWith(MockitoExtension.class). This was written by the JUnit 5 team. The two main benefits:

  • Inject mocks as instance variables
  • Inject mocks as method parameters

What’s missing

In JUnit 5, Mockito no longer tells you about extra setup code that you should delete. And neither version automatically calls verify() like JMock does. I miss that feature and wish Mockito had it. It is nice to be able to have the mock library tell you the actual code didn’t call an expected value without having to remember to code a call to verify the mock. (If Mockito does have it and I didn’t notice, please leave a comment!)

converting 2K+ tests to junit 5 in one day

I set myself a goal to see if I could convert all the CodeRanch JForum tests to JUnit 5 in one day. And I mean with Java 5 package names, not just running them in JUnit 4 mode. (I wanted to see what edge cases I encountered). Let’s see how that went.

Forking

Ant doesn’t currently support JUnit 5. So I decided to create a fork of the code base for playing with JUnit 5. Which is fine. I wanted to automate as much of the process as possible so it’ll be easy to redo when the time comes. It’s also good because then I don’t have to worry about whether everyone’s IDE supports JUnit 5 yet. IntelliJ does if you are on the latest version and Eclipse does if you fiddle with it. Probably better to wait until after the official release of JUnit 5 and Eclipse before using JUnit 5 in the trunk anyway.

Scope reduction

I decided to only update the unit tests. We also have some functional tests with an odd hacked together way of loading the database.  (I wrote a lot of it so not criticizing others.) I’m sure this can be improved in JUnit 5. Will wait until release for that as well.

JUnit 3.8

We have 15 classes that still reference JUnit 3.8. They used a mix of a superclass that did inherit from JUnit’s test case and some that used JUnit 4 style annotations too. I decided to go through these by hand in the trunk (and my branch) and get rid of the references to JUnit 3.8. This was a pain because each one was a special snowflake. One even said it was deprecated with replacement code listed. But it took under an hour; a few minutes each. So not terrible.

As I was going, I missed noticing that the class missed annotation. I was excited to see that SonarLint flags this!

Migrating most of the code

I ran a program to migrate most of the code to the new syntax. Here’s the project on github.

This left me with 83 compiler errors to look at. Of those…

Manual clean up

  • Parameterized tests. The manual gives good examples of the options. Conversion was easy; I used a method source. I did run into two problems.
    • SonarLint false positive – If you only have parameterized tests, SonarLint still flags the class. This was fixed in Sonar per SONARJAVA-2390. However, SonarLint hasn’t been released since so need to ignore the error in my mind.
    • Eclipse false positive – If you only have parameterized tests, Eclipse doesn’t recognize it as being a JUnit test. I didn’t search to see if this was reported since it is so easy to work around. To “solve” both these problems, I added a non-parameteirzed test to the class.
  • Timeout. I had one test that used the @Test(timeout) parameter. It was clear from the manual how to convert this. I like that specifying the time uses Java 8’s Duration class. This lets you specify the timeout in a readable way rather than in milliseconds.
  • Expected exception. I had 16 tests that used the @Test(expected) parameter. I thought about automating this but decided against it. This is a good opportunity to add an assertion about the message to most of them. Which we should have done originally. But it was *so* easy to just write the type and be done with it before. Also, it isn’t 16 distinct classes. A number of them have multiple validation type methods.
  • About 25 of our tests use JMock. @RunWith(JMock.class) doesn’t go with JUnit 5. Since JMock was abandoned (last release in 2012), I decided to switch to the “long way” and call context.assertIsSatisfied(); for now. About half of them extend the same superclass at least. I also made a note that we should migrate away from JMock.

Then I attempted to run all the converted tests

I ran into one problem

  • One of our tests was relying on the order in which the @Before methods were run in the superclass. This behavior was never guaranteed and changed from JUnit 4 to JUnit 5. I fixed the code to not rely on the order. (And or course, the place this happened was a superclass test so it resulted in a large number of failures until I figured out what was going on!)

Observations

I met my challenge. 2K+ tests migrated in one day. And I observed that one superclass caused the vast majority of the special cases!