Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 344914

Summary: API problems in 'org.eclipse.help.base'
Product: [Eclipse Project] PDE Reporter: Dani Megert <daniel_megert>
Component: API ToolsAssignee: Olivier Thomann <Olivier_Thomann>
Status: VERIFIED FIXED QA Contact:
Severity: major    
Priority: P3 CC: cgold, ChrisAustin, curtis.windatt.public, daniel_megert, Michael_Rennie, Olivier_Thomann, remy.suen
Version: 3.7Flags: Michael_Rennie: review+
ChrisAustin: review+
curtis.windatt.public: review+
cgold: review+
Target Milestone: 3.7 RC2   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
Proposed fix
none
Proposed fix
none
Proposed fix
none
Proposed fix
none
Patch to fix missing filter with fix released for RC1 none

Description Dani Megert CLA 2011-05-06 01:56:45 EDT
I20110505-0800 and 3.7 M7. Was OK in 3.7 M6.

API Tools reports problems with 'org.eclipse.help.base':
http://download.eclipse.org/eclipse/downloads/drops/I20110505-0800/apitools/apitoolsverifications/html/org.eclipse.help.base/report.html

I don't know whether the breakage is expected. If so, those problems should be filtered.
Comment 1 Chris Goldthorpe CLA 2011-05-06 11:33:31 EDT
These stem from the reexport of Lucene packages in org.eclipse.help.base. Upgrading from Lucene 1.9.x to 2.9.1 caused API breakage. See Bug 248986 for a full discussion of this and the resolution that we would not increment the major version of org.eclipse.help.base when upgrading. 

What I don't understand is why the API errors are starting to show up now since the change was made in 3.7M4. API filters were added at the time when we increased the version number of Lucene.
Comment 2 Dani Megert CLA 2011-05-06 11:54:15 EDT
Olivier, any idea?
Comment 3 Chris Goldthorpe CLA 2011-05-06 14:34:21 EDT
I know why this is happening. The API errors do not show if Eclipse 3.6.0 is used as an API baseline but do show with 3.6.2. The filters added to suppress the errors when Eclipse 3.6.0 is the API baseline specify that they apply to version 3.5.0 of org.eclipse.help.base, the version number is now at 3.5.3.

The list of errors exactly corresponds to the list of filters. Since the decision was made some time ago to filter these errors rather than bump the major version of org.eclipse.help.base we just need to fix the filters.
Comment 4 Olivier Thomann CLA 2011-05-06 14:49:14 EDT
Yes, problems don't show up if the baseline is 3.6.0. This only occurs with 3.6.1 or 3.6.2. Since the build is using 3.6.2 as the baseline for API tools, errors are reported.
I'll provide a fix for this.
Comment 5 Olivier Thomann CLA 2011-05-06 14:54:29 EDT
Created attachment 194973 [details]
Proposed fix

I had to use "guess" button to apply the patch... weird.
Comment 6 Olivier Thomann CLA 2011-05-06 15:05:32 EDT
I prepare the patch based on 3.6.1 :-(. I'll do it again based on 3.6.2.
Comment 7 Olivier Thomann CLA 2011-05-06 15:08:16 EDT
Created attachment 194974 [details]
Proposed fix

New patch based on 3.6.2
Comment 8 Chris Goldthorpe CLA 2011-05-06 15:14:55 EDT
+1 for the second version of the patch.

Patch has been applied to HEAD, FIXED.
Comment 9 Dani Megert CLA 2011-05-09 02:23:01 EDT
The compatibility problems are gone but there's still the bundle version problem being reported:
http://download.eclipse.org/eclipse/downloads/drops/I20110507-0800/apitools/apitoolsverifications/html/org.eclipse.help.base/report.html
Comment 10 Olivier Thomann CLA 2011-05-09 14:18:20 EDT
I am investigating why the filter is not applied.
Comment 11 Olivier Thomann CLA 2011-05-09 15:47:26 EDT
Moving to PDE/API Tooling as the problem is trying to filter the major version change problem with the corresponding filter.
Comment 12 Olivier Thomann CLA 2011-05-09 15:55:35 EDT
Ant filter store cannot filter a problem reported against the MANIFEST.
I am investigating.
Comment 13 Olivier Thomann CLA 2011-05-09 20:25:37 EDT
Several issues there:
1) The ant filter store must handle cases where the problem has no type names
2) The problem for the major version contains a specific version (no more .qualifier) whereas the problem filter contains a version with .qualifier.
3) The problem filter doesn't contain the resource path to the MANIFEST, but the problem created during the ant task analysis does

(1) is simple to solve
(2) is more difficult as we don't know when an argument is a version. We would need to use the problem id to find out.
For (3), we need to find out how to get the appropriate resource path.

I am investigating. Not sure we need to fix this for 3.7.
Comment 14 Dani Megert CLA 2011-05-10 01:41:01 EDT
(In reply to comment #13)
> Several issues there:
> 1) The ant filter store must handle cases where the problem has no type names
> 2) The problem for the major version contains a specific version (no more
> .qualifier) whereas the problem filter contains a version with .qualifier.
> 3) The problem filter doesn't contain the resource path to the MANIFEST, but
> the problem created during the ant task analysis does
> 
> (1) is simple to solve
> (2) is more difficult as we don't know when an argument is a version. We would
> need to use the problem id to find out.
> For (3), we need to find out how to get the appropriate resource path.
> 
> I am investigating. Not sure we need to fix this for 3.7.

Is there an easy way to make the report look OK? I would not put the existing API Tools code at risk just for that at this point.
Comment 15 Olivier Thomann CLA 2011-05-10 08:39:32 EDT
With today's version of API Tools used in the builder, the api filters are not used to filter this issue because this is a case where there is no type name.
With a small fix, I might be able to handcraft a filter that should work.

We need several small fixes to fix completely this issue.
Comment 16 Olivier Thomann CLA 2011-05-10 14:12:48 EDT
Created attachment 195254 [details]
Proposed fix

With this fix, org.eclipse.help.base will need to update two filters (where the version qualifier is present) and everything should work fine.
I am running all the tests to potentially update test expected results.
Comment 17 Olivier Thomann CLA 2011-05-10 14:35:00 EDT
Created attachment 195257 [details]
Proposed fix

Same patch with updated regression tests.
Comment 18 Olivier Thomann CLA 2011-05-10 14:35:20 EDT
Michael, please review.
Comment 19 Michael Rennie CLA 2011-05-10 15:52:47 EDT
The fix looks fine, running the plug-in test suite I get 13 errors from the Ant task tests like the following:


C:\Users\mrennie\workspaces\apitools-junit-workspace\pde.apitools\apitooling.compare\test1\build.xml:20: Problem: failed to create task or type apitooling.compare
Cause: Could not load a dependent class org/apache/tools/ant/Task
       This class may be in a separate JAR that is not installed.
Action: Determine what extra JAR files are needed, and place them in one of:
        -C:\eclipse\eclipse\plugins\org.apache.ant_1.8.2.v20110505-1300\lib
        -C:\Users\mrennie\.ant\lib
        -a directory added on the command line with the -lib argument

Do not panic, this is a common problem.
The commonest cause is a missing JAR.

This is not a bug; it is a configuration problem

	at org.apache.tools.ant.UnknownElement.getNotFoundException(UnknownElement.java:487)
	at org.apache.tools.ant.UnknownElement.makeObject(UnknownElement.java:419)
	at org.apache.tools.ant.UnknownElement.maybeConfigure(UnknownElement.java:163)
	at org.apache.tools.ant.Task.perform(Task.java:347)
	at org.apache.tools.ant.Target.execute(Target.java:390)
	at org.apache.tools.ant.Target.performTasks(Target.java:411)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.eclipse.ant.internal.core.ant.EclipseDefaultExecutor.executeTargets(EclipseDefaultExecutor.java:32)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:663)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:535)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.ant.core.AntRunner.run(AntRunner.java:513)
	at org.eclipse.pde.api.tools.anttasks.tests.AntRunnerTestCase.runAntScript(AntRunnerTestCase.java:124)
	at org.eclipse.pde.api.tools.anttasks.tests.AntRunnerTestCase.runAntScript(AntRunnerTestCase.java:117)
	at org.eclipse.pde.api.tools.anttasks.tests.ApiToolingCompareAntTaskTests.runTaskAndVerify(ApiToolingCompareAntTaskTests.java:48)
	at org.eclipse.pde.api.tools.anttasks.tests.ApiToolingCompareAntTaskTests.test1(ApiToolingCompareAntTaskTests.java:34)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at junit.framework.TestCase.runTest(TestCase.java:168)
	at junit.framework.TestCase.runBare(TestCase.java:134)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
	at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62)
	at org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness$1.run(PlatformUITestHarness.java:47)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4138)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3755)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2696)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2660)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2494)
	at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:674)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:667)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:123)
	at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.runApp(NonUIThreadTestApplication.java:54)
	at org.eclipse.pde.internal.junit.runtime.UITestApplication.runApp(UITestApplication.java:41)
	at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.start(NonUIThreadTestApplication.java:48)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1386)
Comment 20 Chris Goldthorpe CLA 2011-05-10 16:59:59 EDT
I just tried out the patch to pde.api.tools and I did see one new API error in org.eclipse.help.base. Applying the quick fix added one new filter to .apifilters. Was there a second filter which needed modifying? If so can you attach a patch containing both filters so I can give a +1 and commit the filter changes.
Comment 21 Olivier Thomann CLA 2011-05-10 17:20:02 EDT
You should not change the filters as long as this is not released.
Comment 22 Olivier Thomann CLA 2011-05-10 17:20:29 EDT
(In reply to comment #19)
> The fix looks fine, running the plug-in test suite I get 13 errors from the Ant
> task tests like the following:
All tests ran fine for me.
Comment 23 Michael Rennie CLA 2011-05-10 17:55:35 EDT
(In reply to comment #22)

> All tests ran fine for me.

The patch works fine for me as well, the problem with the tests is obviously some kind of configuration problem in my workspace.
Comment 24 Olivier Thomann CLA 2011-05-10 18:08:16 EDT
Released for 3.7RC1.
Chris, once Kim will update the builder with the bundles produced by the next build, I'll provide another patch to update the filters.
Comment 25 Olivier Thomann CLA 2011-05-17 10:44:10 EDT
Created attachment 195867 [details]
Patch to fix missing filter with fix released for RC1
Comment 26 Chris Goldthorpe CLA 2011-05-17 13:11:21 EDT
+1 from me for "Patch to fix missing filter with fix released for RC1"  I've set the status to REOPENED and the target Milestone to RC2 and requested a review from Chris Austin.
Comment 27 Michael Rennie CLA 2011-05-17 13:16:01 EDT
Adding Curtis as an additional reviewer, we need 2 +1's from committers for RC2
Comment 28 Curtis Windatt CLA 2011-05-17 14:46:14 EDT
+1 filter change looks fine to me.
Comment 29 Chris Austin CLA 2011-05-17 16:42:24 EDT
+1 looks good to me also.
Comment 30 Chris Goldthorpe CLA 2011-05-17 17:59:26 EDT
"Patch to fix missing filter with fix released for RC1 " committed to HEAD, setting status to FIXED.
Comment 31 Olivier Thomann CLA 2011-05-19 17:01:12 EDT
Filters should be good, so this bug is really fixed. Now bug 346499 should be fixed in order to get clean reports.
Comment 32 Dani Megert CLA 2011-05-26 01:24:02 EDT
Verified in I20110525-0800.