Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 334241

Summary: pde.ui test failures in N20110112-2000
Product: [Eclipse Project] PDE Reporter: Ankur Sharma <ankur_sharma>
Component: UIAssignee: Curtis Windatt <curtis.windatt.public>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: curtis.windatt.public, daniel_megert, jeffmcaffer, Olivier_Thomann, sptaszkiewicz, Szymon.Brandys
Version: 3.7   
Target Milestone: 3.7 M6   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on: 202384, 335591    
Bug Blocks:    
Attachments:
Description Flags
Temporary fix for tests none

Description Ankur Sharma CLA 2011-01-13 06:14:09 EST
Build N20110112-2000 (Timestamp: 201101122000):  Automated JUnit testing is complete.  Test failures/errors occurred in the following:

org.eclipse.jdt.debug.tests_linux.gtk.x86_6.0
org.eclipse.jdt.debug.tests_macosx.cocoa.x86_5.0
org.eclipse.jdt.debug.tests_win32.win32.x86_6.0
org.eclipse.pde.api.tools.tests_win32.win32.x86_6.0
org.eclipse.pde.build.tests_linux.gtk.x86_6.0
org.eclipse.pde.build.tests_macosx.cocoa.x86_5.0
org.eclipse.pde.build.tests_win32.win32.x86_6.0
org.eclipse.pde.ui.tests_linux.gtk.x86_6.0
org.eclipse.pde.ui.tests_macosx.cocoa.x86_5.0
org.eclipse.pde.ui.tests_win32.win32.x86_6.0
org.eclipse.releng.tests_linux.gtk.x86_6.0
org.eclipse.releng.tests_macosx.cocoa.x86_5.0
org.eclipse.releng.tests_win32.win32.x86_6.0
org.eclipse.team.tests.cvs.core_linux.gtk.x86_6.0


HTTP Download:

http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000


Test result files (hint: copy/paste or drag-and-drop to the JUnit view):

http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.jdt.debug.tests_linux.gtk.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.jdt.debug.tests_macosx.cocoa.x86_5.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.jdt.debug.tests_win32.win32.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.api.tools.tests_win32.win32.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.build.tests_linux.gtk.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.build.tests_macosx.cocoa.x86_5.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.build.tests_win32.win32.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.ui.tests_linux.gtk.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.ui.tests_macosx.cocoa.x86_5.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.pde.ui.tests_win32.win32.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.releng.tests_linux.gtk.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.releng.tests_macosx.cocoa.x86_5.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.releng.tests_win32.win32.x86_6.0.xml
http://download.eclipse.org/eclipse/downloads/drops/N20110112-2000/testresults/xml/org.eclipse.team.tests.cvs.core_linux.gtk.x86_6.0.xml
Comment 1 Ankur Sharma CLA 2011-01-13 07:08:40 EST
The tests are passing in my local workspace. I don't think anything related have been changed. But the same 2 tests have failed on the three environments which means something is wrong somewhere.
Comment 2 Curtis Windatt CLA 2011-01-13 10:35:06 EST
Could be caused by bug 333981
Comment 3 Curtis Windatt CLA 2011-01-19 10:54:49 EST
I can reproduce locally using N20110117-1015.  Investigating.
Comment 4 Curtis Windatt CLA 2011-01-19 11:13:12 EST
Description	Resource	Path	Location	Type
Source folder 'src-new/' does not have the output folder in corresponding output entry 'output..'.  	build.properties	/org.eclipse.pde.tests.build.properties.1	line 3	Plug-in Problem
Source folder 'src/, src2/, src-new/' does not have the output folder in corresponding output entry 'output..'.  	build.properties	/org.eclipse.pde.tests.build.properties.3	line 1	Plug-in Problem
src2/ is missing from source..	build.properties	/org.eclipse.pde.tests.build.properties.1	line 1	Plug-in Problem
Comment 5 Curtis Windatt CLA 2011-01-19 12:25:39 EST
First test fails because the properties file expects 3 markers.  
javacSource, javacTarget, javacWarnings
However we only set the compiler compliance level, we don't explicitly change the javacWarnings settings on the project, we just use whatever is default.
Comment 6 Curtis Windatt CLA 2011-01-19 12:39:11 EST
Steps to reproduce:

1) Clean workspace, create or get a java project
2) Open Preferences > Compiler > Preferences
3) Set compliance level to 1.5
4) Uncheck use default settings
5) Generated compatibility 1.3
6) Source compatibility 1.3

In older builds, the assert and enum will be set to warning
In newer builds (N20110111-2000) the assert and enum will be set to error
Comment 7 Curtis Windatt CLA 2011-01-19 16:39:55 EST
Additional information:
Change happened sometime after Jan 7th build.
Problem does not appear to be coming from JDT or PDE.
The test project is initialized with the correct values (prefs file contains proper values)
However, in recent builds the properties pages say that there are no project specific settings.  When new preferences are added to the project scope, the prefs file is overwritten.
In older builds, the projects have been built (warnings available on build.properties) and the project settings loaded.

Possibly caused by bug 202384 which modified how the scopes work.
Comment 8 Curtis Windatt CLA 2011-01-19 17:08:51 EST
Confirmed that using an older version of org.eclipse.core.runtime fixes the issue.  Bug 202684 is the most likely cause, timing matches with the failing tests.

Can Szymon or someone else involved with the fix take a look at this?

While we can work around the issue by explicitly setting all the preferences we need, it seems a little odd that our project doesn't pick up the preferences from the .settings file in the project.  I tried refreshing, closing/reopening and rebuilding the project and the preferences are never picked up from the file.
Comment 9 Curtis Windatt CLA 2011-01-19 17:27:06 EST
(In reply to comment #8)
> While we can work around the issue by explicitly setting all the preferences we
> need, it seems a little odd that our project doesn't pick up the preferences
> from the .settings file in the project.  I tried refreshing, closing/reopening
> and rebuilding the project and the preferences are never picked up from the
> file.

We are also opening the project in the test code, and initialize does get called (and the project.isOpen() is true).  Commenting out the isOpen line makes our tests pass.  Something must be different between calling initialize when the project is created (but not open) and having it called during project open().
Comment 10 Szymon Brandys CLA 2011-01-21 11:37:42 EST
Szymon P., please work with the PDE team on the issue.
Comment 11 Curtis Windatt CLA 2011-01-21 12:32:03 EST
Created attachment 187304 [details]
Temporary fix for tests
Comment 12 Curtis Windatt CLA 2011-01-25 12:34:38 EST
(In reply to comment #10)
> Szymon P., please work with the PDE team on the issue.

Ping Szymon
Comment 13 Szymon Ptaszkiewicz CLA 2011-01-25 12:55:07 EST
(In reply to comment #12)
> (In reply to comment #10)
> > Szymon P., please work with the PDE team on the issue.
> 
> Ping Szymon

I'm here. I've tested it and instead of commenting isOpen we could add removeLoadedNodes(this) in the beginning of sync block inside intialize(). This would invalidate all nodes that were previously added and indicate that they need to be reloaded. We haven't done that before so I need to check if it does not break existing code. Maybe there is also some better solution.
Comment 14 Dani Megert CLA 2011-01-27 04:07:24 EST
(In reply to comment #11)
> Created attachment 187304 [details] [diff]
> Temporary fix for tests
This fix went into HEAD (please indicate this next time). Therefore not PDE UI test failures in I20110126-0800.
Comment 15 Dani Megert CLA 2011-01-27 08:19:50 EST
(In reply to comment #6)
> Steps to reproduce:
> 1) Clean workspace, create or get a java project
> 2) Open Preferences > Compiler > Preferences
> 3) Set compliance level to 1.5
> 4) Uncheck use default settings
> 5) Generated compatibility 1.3
> 6) Source compatibility 1.3
> 
> In older builds, the assert and enum will be set to warning
> In newer builds (N20110111-2000) the assert and enum will be set to error
That's not what I see. Using 3.7 M4 I also get 'Error' with the above steps. Maybe you missed some steps? Why do you create a project when you change the workspace settings?
Comment 16 Curtis Windatt CLA 2011-01-27 09:52:09 EST
(In reply to comment #15)
> That's not what I see. Using 3.7 M4 I also get 'Error' with the above steps.
> Maybe you missed some steps? Why do you create a project when you change the
> workspace settings?

Yes, those steps are not correct.  The problem was narrowed down to a change in core resources.  We have the fix in HEAD to prevent the test failures, but Szymon P. is looking at the core resources fix to see if there is still a bug here.
Comment 17 Dani Megert CLA 2011-01-27 10:03:25 EST
(In reply to comment #16)
> (In reply to comment #15)
> > That's not what I see. Using 3.7 M4 I also get 'Error' with the above steps.
> > Maybe you missed some steps? Why do you create a project when you change the
> > workspace settings?
> 
> Yes, those steps are not correct.  The problem was narrowed down to a change in
> core resources. 
You mean the problem in the tests or is there a "real" problem? If so: what are the steps to surface it?
Comment 18 Curtis Windatt CLA 2011-01-27 10:28:52 EST
(In reply to comment #17)
> You mean the problem in the tests or is there a "real" problem? If so: what are
> the steps to surface it?

I don't have steps to reproduce manually.  There could be a real problem, but it could also be that our test is doing something unexpected.  In the tests the following happens:

1) Extract a project from a zip to the workspace
(the project contains settings file to store some project scope preferences)
2) Create an IProject for it
3) Open the project
4) Set a few project scope preferences
5) Build the project
6) Check that markers are properly created

The core resources change affects when the preferences are initialized from the settings file.  Before the change, they are loaded when the project is created.  Now, they are supposed to load when the project is opened, however they are not (which may be the real issue).

The fix for the tests is to explicitly set all project scope preferences we are using rather than depending on what is stored in the settings file.
Comment 19 Dani Megert CLA 2011-01-27 11:21:45 EST
(In reply to comment #18)
> (In reply to comment #17)
> > You mean the problem in the tests or is there a "real" problem? If so: what are
> > the steps to surface it?
> 
> I don't have steps to reproduce manually.  There could be a real problem, but
> it could also be that our test is doing something unexpected.  In the tests the
> following happens:
> 
> 1) Extract a project from a zip to the workspace
> (the project contains settings file to store some project scope preferences)
> 2) Create an IProject for it
> 3) Open the project
> 4) Set a few project scope preferences
> 5) Build the project
> 6) Check that markers are properly created
> 
> The core resources change affects when the preferences are initialized from the
> settings file.  Before the change, they are loaded when the project is created.
>  Now, they are supposed to load when the project is opened, however they are
> not (which may be the real issue).
Strange, because that's what the fix actually did: it makes sure that the settings are loaded on 'open' which wasn't the case before. It would really be good to have a test case that shows the issue in the UI. Maybe you can craft one out of your steps/test?
Comment 20 Szymon Ptaszkiewicz CLA 2011-01-27 11:37:59 EST
If we are creating a project that already has some preferences then those preferences are loaded when refreshing inside project.open() but before initialized(). Nodes are loaded properly but then they got replaced inside initialized() as the parent node was not initialized yet. Those nodes are still in loadedNodes but they do not reflect the real content of prefs files any more. We get the state were loadedNodes claim that our preferences are correctly loaded but those nodes are empty.

For now, the solution is to either call removeNodes(this) inside initialize() (comment 13) or to call ((ProjectPreferences) parent).initialize(); inside load() so that even if we are refreshing, parent preferences will get initialized properly. I will open separate bug to track this issue.
Comment 21 Dani Megert CLA 2011-01-28 02:38:23 EST
> I will open separate bug to track this issue.
What's the bug number?
Comment 22 Dani Megert CLA 2011-01-28 02:42:15 EST
(In reply to comment #21)
> > I will open separate bug to track this issue.
> What's the bug number?
Bug 335591 that is.
Comment 23 Curtis Windatt CLA 2011-02-01 12:09:11 EST
I have reverted the temporary fix for the tests now that there is a fix in core resources.  I updated the tests to use the settings files for preferences rather than setting some of them programmatically in the tests.  Thanks for looking into this Szymon P (and Szymon B).