By 卢声远 Shengyuan Lu


2010-09-12 03:35:26 8 Comments

I want to execute test methods which are annotated by @Test in specific order.

For example:

public class MyTest {
    @Test public void test1(){}
    @Test public void test2(){}
}

I want to ensure to run test1() before test2() each time I run MyTest, but I couldn't find annotation like @Test(order=xx).

I think it's quite important feature for JUnit, if author of JUnit doesn't want the order feature, why?

17 comments

@Walid 2019-03-17 16:52:17

With JUnit 5.4, you can specify the order :

@Test
@Order(2)
public void sendEmailTestGmail() throws MessagingException {

you just need to annotate your class

@TestMethodOrder(OrderAnnotation.class)

https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order

i'm using this in my project and it works very well !

@Rasoul Miri 2019-05-19 11:40:53

@Zon 2018-08-22 09:15:34

JUnit currently allows test methods run ordering using class annotations:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@FixMethodOrder(MethodSorters.JVM)
@FixMethodOrder(MethodSorters.DEFAULT)

By default test methods are run in alphabetical order. So, to set specific methods order you can name them like:

a_TestWorkUnit_WithCertainState_ShouldDoSomething b_TestWorkUnit_WithCertainState_ShouldDoSomething c_TestWorkUnit_WithCertainState_ShouldDoSomething

You can find examples here.

@Arezoo Bagherzadi 2018-08-04 11:41:37

you can use one of these piece of codes:

@FixMethodOrder(MethodSorters.JVM)OR `@FixMethodOrder(MethodSorters.DEFAULT)` OR `@FixMethodOrder(MethodSorters.NAME_ASCENDING)` before your test class like this:


@FixMethodOrder(MethodSorters.NAME_ASCENDING)


public class BookTest { ...}

@McCoy 2017-11-01 15:44:06

I ended up here thinking that my tests weren't run in order, but the truth is that the mess was in my async jobs. When working with concurrency you need to perform concurrency checks between your tests as well. In my case, jobs and tests share a semaphore, so next tests hang until the running job releases the lock.

I know this is not fully related to this question, but maybe could help targeting the correct issue

@joro 2014-06-12 19:00:36

Migration to TestNG seems the best way, but I see no clear solution here for jUnit. Here is most readable solution / formatting I found for jUnit:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SampleTest {
    @Test
    void stage1_prepareAndTest(){};

    @Test
    void stage2_checkSomething(){};

    @Test
    void stage2_checkSomethingElse(){};

    @Test
    void stage3_thisDependsOnStage2(){};

    @Test
    void callTimeDoesntMatter(){}
}

This ensures stage2 methods are called after stage1 ones and before stage3 ones.

@EliuX 2018-05-08 04:07:11

This approach is nice, but it would be valid to mention that if you have more than 10 tests it wont work fine unless you add a 0 prefix, e.g. void stage01_prepareAndTest(){ }

@joro 2019-03-24 09:50:38

If you have more than 10 stages (not tests) - Yes. I prefer limiting number of stages and having more tests in each stage, when this is possible.

@Aman Goel 2016-10-24 06:06:24

Its one of the main issue which I faced when I worked on Junit and I came up with following solution which works fine for me:

import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.FrameworkMethod;
import org.junit.runners.model.InitializationError;

public class OrderedRunner extends BlockJUnit4ClassRunner {

    public OrderedRunner(Class<?> clazz) throws InitializationError {
        super(clazz);
    }

    @Override
    protected List<FrameworkMethod> computeTestMethods() {
        List<FrameworkMethod> list = super.computeTestMethods();
        List<FrameworkMethod> copy = new ArrayList<FrameworkMethod>(list);
        Collections.sort(copy, new Comparator<FrameworkMethod>() {

            @Override
            public int compare(FrameworkMethod f1, FrameworkMethod f2) {
                Order o1 = f1.getAnnotation(Order.class);
                Order o2 = f2.getAnnotation(Order.class);

                if (o1 == null || o2 == null) {
                    return -1;
                }

                return o1.order() - o2.order();
            }
        });
        return copy;
    }
}

also create a interface like below:

 @Retention(RetentionPolicy.RUNTIME)


@Target({ ElementType.METHOD})

public @interface Order {
public int order();
}

Now suppose you have class A where you have written several test cases like below:

(@runWith=OrderRunner.class)
Class A{
@Test
@Order(order = 1)

void method(){

//do something

}

}

So execution will start from method named "method()". Thanks!

@Kyriakos Georgiopoulos 2018-09-05 10:48:24

Using another JUnit Runner with PowerMock Since version 1.6.0 PowerMock has support for delegating the test execution to another JUnit runner without using a JUnit Rule. This leaves the actual test-execution to another runner of your choice. @RunWith(PowerMockRunner.class) @PowerMockRunnerDelegate(OrderedRunner.class)

@Anonymous Developer 2012-11-05 20:17:44

Look at a JUnit report. JUnit is already organized by package. Each package has (or can have) TestSuite classes, each of which in turn run multiple TestCases. Each TestCase can have multiple test methods of the form public void test*(), each of which will actually become an instance of the TestCase class to which they belong. Each test method (TestCase instance) has a name and a pass/fail criteria.

What my management requires is the concept of individual TestStep items, each of which reports their own pass/fail criteria. Failure of any test step must not prevent the execution of subsequent test steps.

In the past, test developers in my position organized TestCase classes into packages that correspond to the part(s) of the product under test, created a TestCase class for each test, and made each test method a separate "step" in the test, complete with its own pass/fail criteria in the JUnit output. Each TestCase is a standalone "test", but the individual methods, or test "steps" within the TestCase, must occur in a specific order.

The TestCase methods were the steps of the TestCase, and test designers got a separate pass/fail criterion per test step. Now the test steps are jumbled, and the tests (of course) fail.

For example:

Class testStateChanges extends TestCase

public void testCreateObjectPlacesTheObjectInStateA()
public void testTransitionToStateBAndValidateStateB()
public void testTransitionToStateCAndValidateStateC()
public void testTryToDeleteObjectinStateCAndValidateObjectStillExists()
public void testTransitionToStateAAndValidateStateA()
public void testDeleteObjectInStateAAndObjectDoesNotExist()
public void cleanupIfAnythingWentWrong()

Each test method asserts and reports its own separate pass/fail criteria. Collapsing this into "one big test method" for the sake of ordering loses the pass/fail criteria granularity of each "step" in the JUnit summary report. ...and that upsets my managers. They are currently demanding another alternative.

Can anyone explain how a JUnit with scrambled test method ordering would support separate pass/fail criteria of each sequential test step, as exemplified above and required by my management?

Regardless of the documentation, I see this as a serious regression in the JUnit framework that is making life difficult for lots of test developers.

@shlomi33 2016-04-25 15:28:02

Here is an extension to JUnit that can produce the desired behavior: https://github.com/aafuks/aaf-junit

I know that this is against the authors of JUnit philosophy, but when using JUnit in environments that are not strict unit testing (as practiced in Java) this can be very helpful.

@Pascal Thivent 2010-09-12 04:10:42

I think it's quite important feature for JUnit, if author of JUnit doesn't want the order feature, why?

I'm not sure there is a clean way to do this with JUnit, to my knowledge JUnit assumes that all tests can be performed in an arbitrary order. From the FAQ:

How do I use a test fixture?

(...) The ordering of test-method invocations is not guaranteed, so testOneItemCollection() might be executed before testEmptyCollection(). (...)

Why is it so? Well, I believe that making tests order dependent is a practice that the authors don't want to promote. Tests should be independent, they shouldn't be coupled and violating this will make things harder to maintain, will break the ability to run tests individually (obviously), etc.

That being said, if you really want to go in this direction, consider using TestNG since it supports running tests methods in any arbitrary order natively (and things like specifying that methods depends on groups of methods). Cedric Beust explains how to do this in order of execution of tests in testng.

@emory 2010-09-12 05:26:17

I suspect that JUnit4 tests are in fact done in spec order. It is not guaranteed because that would (1) promote bad practices among testers and would (2) tie JUnit's hands somewhat.

@Jon Freedman 2010-09-12 09:23:19

Either you have two independent tests, or you only have one test and should code as such.

@Jon Bright 2012-04-28 14:11:08

@JonFreedman, as I understand the question, it's not a case of the tests being interdependent, just of having a spec of things to test and wanting the results to appear in that order.

@Brian DiCasa 2012-06-29 13:43:37

I can understand not enforcing order for unit tests, however when using JUnit to write integration tests it would be nice to be able to specify the order that tests are run. E.g. Run the login test first.

@marcospereira 2012-10-22 00:51:46

@BrianD. login is probably a "fixture" instead of a test that must run before all the others. I will probably write a BeforeClass that logs in and then write the tests to execute in any order.

@Trejkaz 2012-11-14 00:10:55

I would probably do login as a Rule so that one field in the class can simultaneously declare the test to login as well as logout. Although this raises a second related question, which is, what if another Rule wants to be logged in before it runs? Now you have some order dependency again.

@mirelon 2013-03-10 21:24:52

The implication "tests should be independent => tests should be ORDER independent" is not true. Consider automated grading of student's homeworks. I want to test their solution for smaller inputs first and for larger inputs later. When the solution is failing for smaller inputs (for time/memory limit), then why should the tests run for larger inputs?

@Fagner Brack 2013-03-21 12:45:59

>>>>> TL;DR and Ressurrecting mode on! -> JUnit is wrong! If I want to run the unit test in a class and assume the first succeeds to test another one (let's say for inclusion and edition of a value) it will allow the last to fail if the failure is actually in the first.

@studgeek 2013-03-27 21:07:19

Note, JUnit does not run them in the order they are defined in the class. I have seen them run in different order for at least JUnit 4.10.

@Kelly S. French 2013-05-03 13:59:44

@Pascal Wouldn't a TestSuite solve this problem? We used them to test databases where the individual tests were independent it also made sense to combine some tests together into a larger scenario, such as (create table/insert data/delete data/drop table). This was before JUnit4, so my apologies if TestSuite has been deprecated or replaced with some newer feature. My question would be whether you would recommend TestSuites at all.

@Wolfgang Fahl 2014-03-18 18:48:45

Junit 4.11 has the feature now - see other answer by Özhan Düz

@Tomáš Zato 2015-04-16 01:53:08

Just to mention this: answers It's not possible shouldn't even be posted as answers because, just as we can see now, they are often bound to be wrong eventually. Instead, just post a comment and let the OP wait for the answer once it exists.

@benez 2017-02-07 13:50:55

unfortunately tests are not order independent. you might think of a performance test where a classloader already has classes loaded from a previous test

@MasterJoe2 2017-06-30 18:55:08

@marcospereira - Your suggestion to use BeforeClass instead or test order for login makes sense. But, why do you prefer to (integration/e2e) tests without any order ?

@Rasoul Miri 2019-05-19 11:41:38

you can use there ways stackoverflow.com/a/56207417/4797289

@Martin Kersten 2015-04-30 14:20:01

Please check out this one: https://github.com/TransparentMarket/junit. It runs the test in the order they are specified (defined within the compiled class file). Also it features a AllTests suite to run tests defined by sub package first. Using the AllTests implementation one can extend the solution in also filtering for properties (we used to use @Fast annotations but those were not published yet).

@Mattk 2015-03-19 12:11:44

Not sure I agree, If I want to test 'File Upload' and then test 'Data Inserted by File Upload' why would I not want these to be independent from each other? Perfectly reasonable I think to be able to run them separately rather than having both in a Goliath test case.

@Aniket Kulkarni 2014-07-10 12:54:16

If you get rid of your existing instance of Junit, and download JUnit 4.11 or greater in the build path, the following code will execute the test methods in the order of their names, sorted in ascending order:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SampleTest {

    @Test
    public void testAcreate() {
        System.out.println("first");
    }
    @Test
    public void testBupdate() {
        System.out.println("second");
    }
    @Test
    public void testCdelete() {
        System.out.println("third");
    }
}

@Max P Magee 2014-10-02 16:11:31

We've actually tried a similar method test_001_login(), for example, but although it mostly works to preserve order, it is not guaranteed- we have several instances per test run where 004, 005, and 006 are run after 007. It makes you say, "WTF!," and run to StackOverflow for answers.

@DtechNet 2015-12-17 23:42:06

Awesome - available in JUnit 4.12

@Rodislav Moldovan 2016-02-27 21:15:04

in my tests: testAcase - worked, test_A_case / testA_case - did not!

@Edvinauskas 2017-05-05 08:32:11

I have tried this annotation parameter "MethodSorters.JVM", e.g. "@FixMethodOrder(MethodSorters.JVM)". From the API: JVM - Leaves the test methods in the order returned by the JVM. Works just fine for my what I'm doing (CRUD), runs the test methods in the order they are written in. +1

@AbVog 2018-09-03 21:38:53

This annotation is indeed an answer, but it has the caveat that it is not defined (in Junit 4.12) with @Inherited and therefore becomes ineffective on my AbstractTestCase parent class.

@CharlieS 2014-08-27 00:58:21

What you want is perfectly reasonable when test cases are being run as a suite.

Unfortunately no time to give a complete solution right now, but have a look at class:

org.junit.runners.Suite

Which allows you to call test cases (from any test class) in a specific order.

These might be used to create functional, integration or system tests.

This leaves your unit tests as they are without specific order (as recommended), whether you run them like that or not, and then re-use the tests as part of a bigger picture.

We re-use/inherit the same code for unit, integration and system tests, sometimes data driven, sometimes commit driven, and sometimes run as a suite.

@Charlie 2018-08-06 00:01:33

Have you not had time to complete this answer since 2014? ;)

@pstorli 2014-03-10 20:01:20

I've read a few answers and agree its not best practice, but the easiest way to order your tests - and the way that JUnit runs tests by default is by alphabetic name ascending.

So just name your tests in the alphabetic order that you want. Also note the test name must begin with the word test. Just watch out for numbers

test12 will run before test2

so:

testA_MyFirstTest testC_ThirdTest testB_ATestThatRunsSecond

@Jesse Glick 2012-09-17 17:02:16

The (as yet unreleased) change https://github.com/junit-team/junit/pull/386 introduces a @SortMethodsWith. https://github.com/junit-team/junit/pull/293 at least made the order predictable without that (in Java 7 it can be quite random).

@Jesse Glick 2013-12-18 21:26:21

Seems that #386 has been merged in 4.11.

@vorburger 2016-12-13 21:07:40

as noted further below this page, it's actually called @FixMethodOrder not @SortMethodsWith in 4.11

@kornero 2012-10-24 06:43:15

See my solution here: "Junit and java 7."

In this article I describe how to run junit tests in order - "just as in your source code". Tests will be run, in order as your test methods appears in class file.

http://intellijava.blogspot.com/2012/05/junit-and-java-7.html

But as Pascal Thivent said, this is not a good practise.

@Nicolas Barbulesco 2013-04-04 11:42:37

I had seen your blog post (in russian !), but this is way too complicated.

@kornero 2013-05-22 09:46:17

@NicolasBarbulesco I have two blogs (rus and eng). It is too complicated, because you sholdn't create tests with execution order dependency. My solution is workaround, but real solution - is to remove that dependency.

@default locale 2017-03-13 12:26:36

This post doesn't contain an actual answer. Please, consider adding at least the basic explanation, besides the link.

@Xiè Jìléi 2011-01-16 08:30:22

If the order is important, you should make the order yourself.

@Test public void test1() { ... }
@Test public void test2() { test1(); ... }

In particular, you should list some or all possible order permutations to test, if necessary.

For example,

void test1(); 
void test2(); 
void test3(); 


@Test
public void testOrder1() { test1(); test3(); }

@Test(expected = Exception.class)
public void testOrder2() { test2(); test3(); test1(); }

@Test(expected = NullPointerException.class)
public void testOrder3() { test3(); test1(); test2(); }

Or, a full test of all permutations:

@Test
public void testAllOrders() {
    for (Object[] sample: permute(1, 2, 3)) {
        for (Object index: sample) {
            switch (((Integer) index).intValue()) {
                case 1: test1(); break; 
                case 2: test2(); break; 
                case 3: test3(); break; 
            }
        }
    }
}

Here, permute() is a simple function which iterates all possible permuations into a Collection of array.

@Oleg Abrazhaev 2016-06-08 08:56:23

But what if tests in different files?

@toolforger 2018-11-20 05:45:45

In the first code block, test2 runs test1 again. Junit may still run test2 before test1. This is likely not what you intended, and not a valid answer to the question.

Related Questions

Sponsored Content

50 Answered Questions

65 Answered Questions

[SOLVED] How do I generate random integers within a specific range in Java?

  • 2008-12-12 18:20:57
  • user42155
  • 3834383 View
  • 3287 Score
  • 65 Answer
  • Tags:   java random integer

33 Answered Questions

[SOLVED] How do you assert that a certain exception is thrown in JUnit 4 tests?

9 Answered Questions

[SOLVED] How to make mock to void methods with Mockito

11 Answered Questions

[SOLVED] Run a single test method with maven

33 Answered Questions

[SOLVED] Making a private method public to unit test it...good idea?

  • 2011-08-16 09:11:42
  • jcvandan
  • 35897 View
  • 281 Score
  • 33 Answer
  • Tags:   c# java unit-testing

5 Answered Questions

1 Answered Questions

[SOLVED] Running JUnit4 Test classes in specified order

1 Answered Questions

[SOLVED] How to write dynamic Test Case

1 Answered Questions

[SOLVED] write a clean junit4 tests

Sponsored Content