Bugzilla – Full Text Bug Listing
| Summary: | Add OrderedTestSuite to org.eclipse.test.performance | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Markus Keller <markus.kell.r> | ||||||
| Component: | Releng | Assignee: | Markus Keller <markus.kell.r> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | enhancement | ||||||||
| Priority: | P3 | CC: | daniel_megert, kim.moir, konigsberg, remy.suen | ||||||
| Version: | 3.8 | ||||||||
| Target Milestone: | 4.2 M6 | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| Whiteboard: | |||||||||
| Attachments: |
|
||||||||
Created attachment 210148 [details]
Patch
Sorry, I forgot that the org.eclipse.test.performance project didn't use API Tools. Here's a complete patch that fixes the project setup and adds the OrderedTestSuite.
I also added a little tweak to the generated test errors if a test is missing or not listed: Opening the error from the JUnit view works now.
Fixed in master: 83eac8e066f499629737a7c7e93dc765db3640eb This can also be useful for other tests that suffer from this JDK 7 bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7023180 They decided to leave Class#getDeclaredMethods() unpredictable, although the declaration order is the only reasonable order. The patch in bug 337707 contains a sketch of an OrderedRunner class that could be used to achieve the same in JUnit 4 (needs documentation and more code to detect and report missing tests). Just a reminder for those using the OrderedTestSuite: don't forget to replace addTest(new TestSuite(YourTests.class)); with addTest(YourTests.suite()); *** Bug 386453 has been marked as a duplicate of this bug. *** |
Created attachment 210122 [details] OrderedTestSuite.java For performance tests, the execution order of test methods is often essential. In org.eclipse.jdt.ui.tests, we've implemented an OrderedTestSuite, which not only guarantees the execution order, but also emits an error if the test methods in a class don't correspond to the tests specified in the suite (e.g. if a new test got added, but is missing in the suite). This replaces manual patterns like this: public static Test suite() { // we must make sure that cold is executed before warm TestSuite suite= new TestSuite("RenamePackagePerfTests1"); suite.addTest(new RenamePackagePerfTests1("testCold_10_10")); suite.addTest(new RenamePackagePerfTests1("test_10_10")); suite.addTest(new RenamePackagePerfTests1("test_100_10")); suite.addTest(new RenamePackagePerfTests1("test_1000_10")); return new RefactoringPerformanceTestSetup(suite); } ... with safer and less verbose code like this: public static Test suite() { // we must make sure that cold is executed before warm return new OrderedTestSuite(RenamePackagePerfTests1.class, new String[] { "testCold_10_10", "test_10_10", "test_100_10", "test_1000_10" }); }