Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 349706

Summary: PreferencesTests rely on test ordering
Product: [Eclipse Project] PDE Reporter: Michael Rennie <Michael_Rennie>
Component: API ToolsAssignee: Curtis Windatt <curtis.windatt.public>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: curtis.windatt.public, daniel_megert
Version: 3.7   
Target Milestone: 3.7.1   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on:    
Bug Blocks: 349447    
Attachments:
Description Flags
Possible fix none

Description Michael Rennie CLA 2011-06-17 12:29:44 EDT 
From bug 349447:

junit.framework.AssertionFailedError: The value for RESTRICTION_NOINSTANTIATE
should be 'Error' expected:<2> but was:<1>
    at junit.framework.Assert.fail(Assert.java:47)
    at junit.framework.Assert.failNotEquals(Assert.java:283)
    at junit.framework.Assert.assertEquals(Assert.java:64)
    at junit.framework.Assert.assertEquals(Assert.java:195)
    at
org.eclipse.pde.api.tools.util.tests.PreferencesTests.testGetNonDefaultValue(PreferencesTests.java:78)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at junit.framework.TestCase.runTest(TestCase.java:168)
    at junit.framework.TestCase.runBare(TestCase.java:134)
    at junit.framework.TestResult$1.protect(TestResult.java:110)
    at junit.framework.TestResult.runProtected(TestResult.java:128)
    at junit.framework.TestResult.run(TestResult.java:113)
    at junit.framework.TestCase.run(TestCase.java:124)
    at junit.framework.TestSuite.runTest(TestSuite.java:243)
    at junit.framework.TestSuite.run(TestSuite.java:238)
    at junit.framework.TestSuite.runTest(TestSuite.java:243)
    at junit.framework.TestSuite.run(TestSuite.java:238)
    at
org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
    at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at
org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62)
    at
org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness$1.run(PlatformUITestHarness.java:47)
    at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
    at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135)
    at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3561)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3210)
    at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2696)
    at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2660)
    at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2494)
    at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:674)
    at
org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:667)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
    at
org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:123)
    at
org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.runApp(NonUIThreadTestApplication.java:54)
    at
org.eclipse.pde.internal.junit.runtime.UITestApplication.runApp(UITestApplication.java:41)
    at
org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.start(NonUIThreadTestApplication.java:48)
    at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
    at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1386)

Looking at the tests the problem is because the suite relies on the test to be run in top-down order. We must move the initializing code from #testSetupSettings() to #setup(), so that each test can run in any order.
Comment 1 Curtis Windatt CLA 2011-06-22 17:53:12 EDT 
This is even worse than originally described.  Some tests, like the ApiDescriptionProcessor tests depend not only on the order of methods within the tests, but also that other test suites (ProjectCreationTests) be run first and in order.
Comment 2 Curtis Windatt CLA 2011-06-23 17:04:38 EDT 
Created attachment 198501 [details]
Possible fix

This patch changes three test suites to create the necessary projects to run during the setup call (and deleting them during tear down).  

Since we are creating, adding content and refactoring a project for each test, the tests run significantly slower (over a second each vs < 300 ms).  There are not a lot of these tests, so the overall performance hit isn't major.

I'm running the plug-in tests some more to double check this fix.  Are we targeting this for the Java7 branch?
Comment 3 Curtis Windatt CLA 2011-06-24 16:10:18 EDT 
Released the patch to the BETA_JAVA7 branch.  While the test failures only occur on a 1.7 vm, this fix should be applied to HEAD if the BETA_JAVA7 branch does not get merged, as it allows the tests to run atomically.
Comment 4 Curtis Windatt CLA 2011-08-25 15:15:46 EDT 
Verified that the tests can now run independently. We had to fix Bug 355152.