By Andrzej Doyle


2009-11-06 17:54:35 8 Comments

OK, so the @Ignore annotation is good for marking that a test case shouldn't be run.

However, sometimes I want to ignore a test based on runtime information. An example might be if I have a concurrency test that needs to be run on a machine with a certain number of cores. If this test were run on a uniprocessor machine, I don't think it would be correct to just pass the test (since it hasn't been run), and it certainly wouldn't be right to fail the test and break the build.

So I want to be able to ignore tests at runtime, as this seems like the right outcome (since the test framework will allow the build to pass but record that the tests weren't run). I'm fairly sure that the annotation won't give me this flexibility, and suspect that I'll need to manually create the test suite for the class in question. However, the documentation doesn't mention anything about this and looking through the API it's also not clear how this would be done programmatically (i.e. how do I programatically create an instance of Test or similar that is equivalent to that created by the @Ignore annotation?).

If anyone has done something similar in the past, or has a bright idea of how else I could go about this, I'd be happy to hear about it.

4 comments

@Yishai 2009-11-06 18:05:32

The JUnit way is to do this at run-time is org.junit.Assume.

 @Before
 public void beforeMethod() {
     org.junit.Assume.assumeTrue(someCondition());
     // rest of setup.
 }

You can do it in a @Before method or in the test itself, but not in an @After method. If you do it in the test itself, your @Before method will get run. You can also do it within @BeforeClass to prevent class initialization.

An assumption failure causes the test to be ignored.

Edit: To compare with the @RunIf annotation from junit-ext, their sample code would look like this:

@Test
public void calculateTotalSalary() {
    assumeThat(Database.connect(), is(notNull()));
    //test code below.
}

Not to mention that it is much easier to capture and use the connection from the Database.connect() method this way.

@Yishai 2009-11-06 18:21:46

@notnoop, that isn't my observation at all. They are ignored. The IDEA test runner reports them that way, and a look at the JUnit source code shows that it reports the test as ignored.

@Yishai 2009-11-06 20:45:45

To quote: "In the future, this may change, and a failed assumption may lead to the test being ignored." It in fact changed, as of 4.5 I believe. The current javadoc says: "The default JUnit runner treats tests with failing assumptions as ignored. Custom runners may behave differently." github.com/KentBeck/junit/blob/…

@Andrzej Doyle 2009-11-09 08:54:43

Thanks, that's exactly what I'm looking for. The semantics of Assume sound like exactly what I'm trying to do here, so even if a runner doesn't treat it as @Ignore I trust it will be handled appropriately.

@fijiaaron 2010-12-09 16:24:37

Eclipse 3.6 with Junit 4.8.1 reports false Assumptions as a passing test. Same with ant 1.8.1.

@Yishai 2010-12-09 17:06:23

@fijaaron, Regarding ANT, this is a known limitation of its XML format, and since so many tools rely on this format, changing it is a big deal, so it won't happen so fast. Regarding eclipse, since IDEA does it, I have no idea why eclipse hasn't caught up. Regarding the ANT functionality, see here: issues.apache.org/bugzilla/show_bug.cgi?id=43969

@Mark 2011-03-15 00:15:46

Note that if you are using maven's surefire plugin with JUnit in this fashion, you need to use surefire 2.7.2 or greater to get tests halted because of Assume to show up as Skipped rather than success.

@Martin 2012-02-12 12:07:56

That Eclipse reports failed assumptions as passing is a bug: bugs.eclipse.org/bugs/show_bug.cgi?id=359944

@Jeff Storey 2012-05-11 17:58:57

If you have an After method though, it appears the After method still runs, but it doesnt look like Assume can be used in an After method. Is there anyway to prevent both the before and after methods from running?

@Yishai 2012-05-13 15:27:43

@JeffStorey, the after method only runs if the assume is in the method itself, not in before. That makes sense - if there was a before, there would be an after. You can gain more fine grained control using a TestRule. kentbeck.github.com/junit/javadoc/latest/org/junit/rules/…

@Jeff Storey 2012-05-15 01:53:50

@Yishai, I set up a simple test where I put Assume.assumeTrue(false) in the before method, but the after method still runs. It sounds like that is the opposite of what you expected...

@Yishai 2012-05-15 13:16:06

@JeffStorey, I set up a test as well, and it worked as I described, or at least I thought it did. I tried again, and you are correct. I guess TestRule is the way to go, then.

@Jeff Storey 2012-05-15 13:38:12

@Yishai, thanks. I'm not sure if TestRule will work in this case though. Trying to find a way to basically ignore the whole test class if the assumption is false without annotating every method. I could always write a custom runner if needed to process some annotation.

@Yishai 2012-05-15 15:35:08

@JeffStorey, then you are looking for a couple of things. One is the @BeforeClass annotation, where you can have your assumption fail there, which will skip the whole class. Another is @ClassRule (for the fine grained control, but over the whole class, one time).

@Jeff Storey 2012-05-15 17:11:14

Thanks, I never really thought about using Assume in a beforeClass method. I'll give that a shot. And I'll look into ClassRule as well. Thanks.

@Will 2012-12-16 18:00:07

That is AWESOME! I'm going to use this method to separate my smoke tests with both eclipse and maven. Sweet.

@Don Rhummy 2018-05-02 17:47:37

This is not the correct answer. This will not cause the test to be ignored or "passed". It still runs the test method's code. The correct way is to use @KyleShrader's suggestion

@WesternGun 2018-05-21 11:15:00

In my case I go for the Assume way. Sometimes the condition is different depending on the test, I may run test A with input of type A and run testB with input of type B; then @Before is not the way to go because it executes before every test, thus the condition must be the same.

@OrangeDog 2019-06-18 17:16:14

N.B. the @After methods will still run, even if the failed assumption was in a @Before method.

@Kyle Shrader 2017-02-28 23:54:16

In JUnit 4, another option for you may be to create an annotation to denote that the test needs to meet your custom criteria, then extend the default runner with your own and using reflection, base your decision on the custom criteria. It may look something like this:

public class CustomRunner extends BlockJUnit4ClassRunner {
    public CTRunner(Class<?> klass) throws initializationError {
        super(klass);
    }

    @Override
    protected boolean isIgnored(FrameworkMethod child) {
        if(shouldIgnore()) {
            return true;
        }
        return super.isIgnored(child);
    }

    private boolean shouldIgnore(class) {
        /* some custom criteria */
    }
}

@TIn TIn 2016-08-09 19:32:44

A quick note: Assume.assumeTrue(condition) ignores rest of the steps but passes the test. To fail the test, use org.junit.Assert.fail() inside the conditional statement. Works same like Assume.assumeTrue() but fails the test.

@Andrzej Doyle 2016-08-10 11:58:09

As noted in the answers above, a failed assumption does not cause the test to pass, it returns a separate status. Some runners might erroneously report this as if it were a pass, but that's a weakness/bug in the test runner (and the default JUnit runner displays the test as ignored). And as for your final sentence, failing the test is specifically not what I want(ed) to do.

@TIn TIn 2016-08-10 17:51:27

Oh OK. The tests passed on failed assumption in my case but I wanted them to be reported as failed (I was checking for an exception from Test Watcher). Forcing a failure helped me.

@notnoop 2009-11-06 17:59:32

You should checkout Junit-ext project. They have RunIf annotation that performs conditional tests, like:

@Test
@RunIf(DatabaseIsConnected.class)
public void calculateTotalSalary() {
    //your code there
}

class DatabaseIsConnected implements Checker {
   public boolean satisify() {
        return Database.connect() != null;
   }
}

[Code sample taken from their tutorial]

@Andrzej Doyle 2009-11-09 08:59:16

Thanks for this answer - an interesting alternative syntax for the functionality, though I'll be going with Assume directly so as not to introduce another dependency.

@Richard 2010-07-02 10:20:05

I personally prefer this solution. If you have many tests that should be run based on the same conditions, this would be far more ideal than having to use Assume in every test. Also, if this can be used on a class level rather than the method level, it will be even more ideal.

@Michael-O 2013-07-17 19:36:27

junit-ext is not available on Maven Central :-(

@shambhu 2014-05-21 10:31:51

I would prefer it, 'cuz this helps to run the test conditionally at run time. It suits where a number of unit test are going to run and the requirement is to run the unit tests on particular checker. I really amazed to see that junit-ext is not available on maven repository. How would we get avail this in maven project.

@Rüdiger Herrmann 2015-03-19 11:24:19

An annotation like @RunIf separates the condition when a test should run from the actual test code, which I think is good. What I don't like is that it requires a particular test runner. Therefore I wrote a JUnit rule to conditinoally ignore tests.

@Marc 2015-08-27 09:24:57

After installing the junit-ext jar (found here code.google.com/p/junit-ext/downloads/… ) in our local repository and implementing this @RunIf annotation... nothing! It is totally ignored, and I think the reason might be that junit-ext seems to depend on junit 4.5. We need 4.9+ due to spring-test. So... never mind that.

@cosbor11 2016-03-19 01:03:50

what is the namespace for Checker ???

Related Questions

Sponsored Content

50 Answered Questions

31 Answered Questions

[SOLVED] Unit Testing C Code

11 Answered Questions

[SOLVED] How do I make Git ignore file mode (chmod) changes?

  • 2009-10-16 21:43:38
  • Marcus Westin
  • 818194 View
  • 2123 Score
  • 11 Answer
  • Tags:   git ignore chmod

33 Answered Questions

[SOLVED] How do you assert that a certain exception is thrown in JUnit 4 tests?

35 Answered Questions

8 Answered Questions

[SOLVED] How do I configure git to ignore some files locally?

  • 2009-11-18 01:36:47
  • Bjorn Tipling
  • 327053 View
  • 1177 Score
  • 8 Answer
  • Tags:   git ignore gitignore

9 Answered Questions

[SOLVED] Git command to show which specific files are ignored by .gitignore

  • 2009-01-21 20:09:36
  • Andrew Burns
  • 255350 View
  • 565 Score
  • 9 Answer
  • Tags:   git ignore

12 Answered Questions

[SOLVED] JUnit test for System.out.println()

  • 2009-07-13 13:18:13
  • Mike Minicki
  • 204909 View
  • 331 Score
  • 12 Answer
  • Tags:   java console junit

49 Answered Questions

[SOLVED] Why does visual studio 2012 not find my tests?

44 Answered Questions

[SOLVED] Is Unit Testing worth the effort?

  • 2008-09-15 21:44:00
  • Ross Fuhrman
  • 269471 View
  • 573 Score
  • 44 Answer
  • Tags:   unit-testing

Sponsored Content