Community
Participate
Working Groups
Build Identifier: 20100617-1415 If you choose a Run Configuration of JUnit, and set "Run all tests in the selected project, package or source folder" the @Tests are not computed in the order they have been implemented inside the source files. For some files the order is kept, for some it is mixed. Reproducible: Always
The TestSuite use the Class.getDeclaredMethods() to build the list of the tests to run which explicitly says: * The elements in the array returned are not sorted and are not in any * particular order. Hence there's no guarantee that the JUnit tests will be built in the same order than the order of tests in file. Typically, this will be VM dependent... If you need to be 100% sure of the tests order, then you have to use another way to create your test suite than the constructor TestSuite(final Class<? extends TestCase> theClass) E.g. In org.eclipse.jdt.core.tests.compiler, have a look on org.eclipse.jdt.core.tests.junit.extension.TestCase.buildTestSuite(Class) which is calling buildTestList to have the ability to control the test order in the returned list...
I'm not using the TestSuite feature of JUnit. There are just a set of files implementing tests in one package. I've never had any problem with the order of the tests. We have been running JUnit tests in a Continious Integration and Hudson environment and the order was always like it was implemented in the file. For me it looks as if the Eclipse feature does something else then executing test class by test class, since running the test class standalone works always fine.
(In reply to comment #2) > I'm not using the TestSuite feature of JUnit. There are just a set of files > implementing tests in one package. > > I've never had any problem with the order of the tests. We have been running > JUnit tests in a Continious Integration and Hudson environment and the order > was always like it was implemented in the file. > > For me it looks as if the Eclipse feature does something else then executing > test class by test class, since running the test class standalone works always > fine. Not sure whether there's something else done while executing several test classes or not. Hence, move to JDT/UI where the real JUnit knowledge is...
This also works for me for JUnit 4 tests. In JUnit 4, tests are added in org.junit.runners.model.TestClass#TestClass(Class<?>), which also uses Class#getDeclaredMethods(). That method often returns methods in declaration order, but that's not the case for all VMs and can change any time. Please reopen if you can provide a test case that fails. Also indicate your VM (java -version). My test case (works fine with Sun JDK 1.6.0_21-b06): package bug; import static org.junit.Assert.assertEquals; import org.junit.Test; public class MyTests { private static int counter; @Test public void test1() throws Exception { assertEquals(1, ++counter); } @Test public void test2() throws Exception { assertEquals(2, ++counter); } @Test public void aaa() throws Exception { assertEquals(3, ++counter); } @Test public void zzz() throws Exception { assertEquals(4, ++counter); } }
hmm, i'm working with JDK 1.5.0_22 I sent you a screenshot where you can see that the order is not the same
Created attachment 177496 [details] Settings These are the settings I use
Created attachment 177497 [details] Result And this is the result. As you can see the tests are not computed in the order they've been implemented. And this only happens when executing complete packages.
My test case from comment 4 also works fine with JDK 1.5.0_22. I cannot do anything without a self-contained example that has this problem. If you want to know why the order is sometimes different, you have to provide a self-contained example that I can reproduce -- or just set a breakpoint in org.junit.runners.model.TestClass and debug it in your test suite.
