Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 327654

Summary: FUP of bug 317264: Refactoring is not possible if the commons-lang.jar is in the path
Product: [Eclipse Project] JDT Reporter: Satyam Kandula <satyam.kandula>
Component: CoreAssignee: Satyam Kandula <satyam.kandula>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: amj87.iitr, daniel_megert, frederic_fusier, markus.kell.r, Olivier_Thomann, snjezana.peco
Version: 3.6Flags: daniel_megert: pmc_approved+
Olivier_Thomann: review+
Target Milestone: 3.6.2   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
Proposed patch
none
Patch for 3.6 Maintenance
none
Jar file for the added test
none
Patch for 3.6 Maintenance none

Description Satyam Kandula CLA 2010-10-13 08:16:04 EDT
bug 317264 has fixed the main problem, but it assumes that the jar file name is org.apache.commons.lang_2*. We run into the same problem if the jar file name in the path is commons-lang.jar. 
Also look at bug 293861 comment 62.
Comment 1 Satyam Kandula CLA 2010-10-13 10:13:20 EDT
Is it good enough to also check for commons-lang.jar? Are there any other names that are generally used? If not we should not check for the name of the jar but should do the enum check for all the jars. Any comments?
Comment 2 Frederic Fusier CLA 2010-10-13 11:59:27 EDT
We already saw with first patch of bug 317264 that it was too costly to test all jars. Hence, I would fix this new bug by testing the second package name in PatternSearchJob to set the SHOULD_FILTER_ENUM static variable...

We could also have only test (and has 0 impact on performance) if we would change the test to see if the package name contains "commons-lang", but of course, the test would be less accurate then...
Comment 3 Markus Keller CLA 2010-10-13 13:29:04 EDT
(In reply to comment #2)
> We already saw with first patch of bug 317264 that it was too costly to test
> all jars.

Please correct me if I'm wrong, but I thought the first patch from bug 317264 checked the package name for each match (after the match was found).

Can't you do something similar as bug 293861 comment 34 and detect this problem already while you index a JAR? In this case, you can't skip the package, but you could store an additional bit in the index that tells if the JAR contains "enum" or "assert" in a package name. Then you can do the complete check only for JARs that are known to be problematic (performance is not an issue for those), and in the normal case, search speed is at 100%.

The test for a special JAR name should be removed completely.
Comment 4 Dani Megert CLA 2010-10-14 03:09:58 EDT
Targeting 3.6.2 as 317264 is also in the maintenance stream.
Comment 5 Satyam Kandula CLA 2010-10-14 10:25:51 EDT
(In reply to comment #3)
> In this case, you can't skip the package, but
> you could store an additional bit in the index that tells if the JAR contains
As far as I could see, adding additional bit could mean changing the format of the index file which is not good.
Comment 6 Satyam Kandula CLA 2010-10-14 10:31:45 EDT
(In reply to comment #2)
> We already saw with first patch of bug 317264 that it was too costly to test
> all jars. 
The first patch did full checks without any assumption. In the second patch, we only check for the 5th package element to be enum. Taking this into consideration, I don't think we will notice any performance degradation. Let me try this out -- I didn't notice the degradation on my m/c with the first patch also :(. But let me try running more tests. 

Markus, Do you know or see people using different names for this jar?
Comment 7 Markus Keller CLA 2010-10-14 11:16:02 EDT
> Markus, Do you know or see people using different names for this jar?

I don't know, but we also didn't know about commons-lang.jar until it showed up. The fact that we have to touch this for 3.6.2 again is a good indication that this problem should finally be solved at its root and that name-specific fixes are not general enough.

> only check for the 5th package element to be enum

That's also a limitation that should be removed.

AddJarFileToIndex#isValidPackageNameForClass(String) looks like a good place to add the check (and to remember whether the JAR potentially has problems.).
Comment 8 Olivier Thomann CLA 2010-11-09 11:52:40 EST
This should be fixed once for all.
Comment 9 Satyam Kandula CLA 2011-01-07 00:35:35 EST
(In reply to comment #7)
> > only check for the 5th package element to be enum
> 
> That's also a limitation that should be removed.
We haven't seen any other places where enum is used and generalizing is having performance problem. 

> AddJarFileToIndex#isValidPackageNameForClass(String) looks like a good place to
> add the check (and to remember whether the JAR potentially has problems.).

We cannot add here because we don't have the project compliance at this time.
Comment 10 Satyam Kandula CLA 2011-01-07 00:45:37 EST
Created attachment 186246 [details]
Proposed patch

In this patch, the name of the jar is no longer checked. However, the package name check is retained. I don't see any performance impact with this patch.
Comment 11 Satyam Kandula CLA 2011-01-07 10:28:01 EST
(In reply to comment #10)
Released this patch on HEAD.
Comment 12 Satyam Kandula CLA 2011-01-11 08:29:30 EST
Reopened as this needs to be fixed for 3.6.2
Comment 13 Satyam Kandula CLA 2011-01-11 09:13:21 EST
Created attachment 186492 [details]
Patch for 3.6 Maintenance

This is for 3.6.2. 
Olivier, Dani: Need +1 to release on 3.6.2
Comment 14 Satyam Kandula CLA 2011-01-11 09:30:26 EST
Created attachment 186498 [details]
Jar file for the added test

This jar file need to be copied in the folder \org.eclipse.jdt.core.tests.model\workspace\JavaSearchBugs\lib\b327654 for the test to run successfully.
Comment 15 Olivier Thomann CLA 2011-01-11 09:42:19 EST
+1.
In filterEnum:
// enum was found in org.apache.commons.lang.enum at index 5
should be replaced by:
// enum was found in org.apache.commons.lang.enum at index 4
Comment 16 Satyam Kandula CLA 2011-01-11 11:29:07 EST
Created attachment 186520 [details]
Patch for 3.6 Maintenance

Patch with Olivier's comment..
Comment 17 Dani Megert CLA 2011-01-12 05:40:17 EST
> This is for 3.6.2. 
> Olivier, Dani: Need +1 to release on 3.6.2

While this bug might be justified and also the fix might be OK, I failed to reproduce the problem that triggered this: bug 293861 comment 62 says:

>- import the project
>https://anonsvn.jboss.org/repos/jbosstools/workspace/snjeza/jbide5036
>/jbide5036b.zip
>- select the test.Test.getName method and call Refactor>Rename
>
>You will get an exception (see jbide5036-2.log)
>
>The project has the commons-lang.jar library that contains the
>"org.apache.commons.lang.enum" package that is invalid in JDK>=1.5. 

I tried those steps on 3.6.1 (even with additional step that adds commons-lang.jar) but I always run into an NPE (bug 250958).

Before I start to review the patch I need reproducible steps that I can use to verify the fix.
Comment 18 Dani Megert CLA 2011-01-12 06:22:39 EST
> >- import the project
> >https://anonsvn.jboss.org/repos/jbosstools/workspace/snjeza/jbide5036
> >/jbide5036b.zip
> >- select the test.Test.getName method and call Refactor>Rename
> >
> >You will get an exception (see jbide5036-2.log)
> >
> >The project has the commons-lang.jar library that contains the
> >"org.apache.commons.lang.enum" package that is invalid in JDK>=1.5. 
> 
> I tried those steps on 3.6.1 (even with additional step that adds
> commons-lang.jar) but I always run into an NPE (bug 250958).
> 
I downloaded the ZIPs for both test cases and then mixed a and b. Bad news is that the NPE still occurs but with 
https://anonsvn.jboss.org/repos/jbosstools/workspace/snjeza/jbide5036/jbide5036a.zip
good news is that I can reproduce this bug now using 
https://anonsvn.jboss.org/repos/jbosstools/workspace/snjeza/jbide5036/jbide5036b.zip
Comment 19 Dani Megert CLA 2011-01-12 06:53:57 EST
Verified that the patch works. The patch itself looks good and the performance results (compared I20110112-0800 with 3.7 M4) don't indicate a problem.

+1 for 3.6.2.
Comment 20 Satyam Kandula CLA 2011-01-12 07:12:50 EST
Released on 3.6 Maintenance branch
Comment 21 Ayushman Jain CLA 2011-01-20 05:31:03 EST
Verified for 3.6.2 using build M20110119-0834.
Comment 22 Satyam Kandula CLA 2011-09-15 00:13:25 EDT
Verified for 3.8M2