Ignore Junit tests conditionally

In TRNLTK project I had several data files which I received with an academic license. Thus I cannot publish them to Github. In that case, some of the tests in the CI server, drone.io, fails because those files cannot be found. First I thought of adding @Ignore to the tests which use those files, but in my local environment I should better run those tests.

So, I thought of a better solution:

• Create a system property to be set by Maven, based on the Maven profile in use
• Check that property at the tests and if not set, skip the test

By default, the property will be set to false:

<properties>
  <hasBigParseSets>false</hasBigParseSets>
</properties>

It will be set, when the Maven profile bigParseSets is activated:

<profiles>
   <profile>
      <id>bigParseSets</id>
      <properties>
         <hasBigParseSets>true</hasBigParseSets>
      </properties>
   </profile>
</profiles>

Important thing here is, to use JUnit’s Assume functionality. With that, if an assumption is not correct, then the test will be reported as “skipped / ignored”. If you use if(..) return; then your test will be shown as successful.

@Test
public void shouldTryEveryPossiblePhoneticAttributeSet() throws IOException {
    Assume.assumeTrue(TestEnvironment.hasBigParseSets());
    ...
}

Last piece:

public class TestEnvironment {
   private static final String HAS_BIG_PARSESETS = "hasBigParseSets";
   public static boolean hasBigParseSets(){
      return "true".equalsIgnoreCase(System.getProperty(HAS_BIG_PARSESETS));
   }
}

If you put your assumption in the setup method (method annotated with @Before), then the whole test class will be ignored in case of failed assumption.

Here is how to activate the profile:

mvn test -P bigParseSets
...
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.494 sec

Without the profile:

mvn test
...
Tests run: 4, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.227 sec

When you execute the tests outside Maven, e.g. within your IDE’s JUnit runner, then you need to pass the system property as -DhasBigParseSets=true.

Intellij IDEA is really smart at this. If you activate the profile, then the system profile is automatically passed to Intellij’s JUnit runner:

Just don’t forget to press popping “Import” link when you switch profiles:

(Or enable auto import to not do this every time)

I checked the alternatives a little bit. There is a good library named junit-ext which allows you to do the same thing in a better way with @RunIf(Condition.class) annotations, but it seems dead.

Other alternatives include:

• using a JUnit @Category : Cannot specify something like : “Run all tests except this category”. Even if you can, cannot integrate it well with Maven (+SureFire)
• using JUnit @Rules : Need to implement your own annotation and extend standard JUnit runner, resolver, etc