Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 295920 - Exception under the latest JDK7 in the "Open Type" dialog
Summary: Exception under the latest JDK7 in the "Open Type" dialog
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.5.1   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.6 M5   Edit
Assignee: JDT-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 327613 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-11-23 14:44 EST by Kirill Grouchnikov CLA
Modified: 2010-10-15 08:33 EDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kirill Grouchnikov CLA 2009-11-23 14:44:48 EST
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2b3) Gecko/20091115 Firefox/3.6b3 (.NET CLR 3.5.30729)
Build Identifier: M20090917-0800

When Eclipse runs under one of the latest JDK7 builds (i have b71), and i start typing in the "Open Type" dialog, i get the following exception:

eclipse.buildId=M20090917-0800
java.version=1.7.0-ea
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86


Error
Mon Nov 23 11:44:25 PST 2009
An internal error occurred during: "Items filtering".

java.lang.IllegalArgumentException: Comparison method violates its general contract!
at java.util.TimSort.mergeHi(TimSort.java:868)
at java.util.TimSort.mergeAt(TimSort.java:485)
at java.util.TimSort.mergeForceCollapse(TimSort.java:426)
at java.util.TimSort.sort(TimSort.java:223)
at java.util.TimSort.sort(TimSort.java:173)
at java.util.Arrays.sort(Arrays.java:1347)
at java.util.Collections.sort(Collections.java:217)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$ContentProvider.getSortedItems(FilteredItemsSelectionDialog.java:2837)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$ContentProvider.rememberResult(FilteredItemsSelectionDialog.java:2850)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$FilterJob.filterContent(FilteredItemsSelectionDialog.java:2186)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$FilterJob.internalRun(FilteredItemsSelectionDialog.java:2124)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$FilterJob.doRun(FilteredItemsSelectionDialog.java:2096)
at org.eclipse.ui.dialogs.FilteredItemsSelectionDialog$FilterJob.run(FilteredItemsSelectionDialog.java:2083)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

TimSort is the new sort added to JDK 7.

The exception only happens if i type really fast. If i pause between each successive key stroke, the exception does not happen.

Reproducible: Sometimes

Steps to Reproduce:
1. Hit "Ctrl+Shift+T" to bring up an "Open Type" dialog
2. Start typing the camel-case letters of some type in your project.
3. If you type fast enough, the exception will be thrown.
Comment 1 Markus Keller CLA 2009-11-24 12:36:45 EST
I cannot reproduce this with the given Eclipse build and VM, and the error message is less than helpful. Sorry, I can't do anything here.

If you could set a breakpoint at the throw statement and then debug it in a target workbench, that could provide more insight on why this happens. Maybe the bug is really in TimSort and the IAE is a false accusation (that would at least explain why there's no justification in the error message).
Comment 2 Kirill Grouchnikov CLA 2009-11-24 13:15:19 EST
Not only this is consistently reproducible on my machine (under the latest JDK7 build downloaded yesterday), but the filtering is behaving quite badly even without exception.

Seems that there is an asynchronous handling of key type events, and it gets out of sync with the actual contents. So even when the exception is not thrown, the results that are shown are not matching the camel-case string that i'm typing. In most cases i need to pause for a second, hit a Backspace key and then retype the last letter so that the dialog shows the right choices. This is reproducible under JDK 6.

Thanks
Kirill
Comment 3 Dani Megert CLA 2009-11-25 02:35:56 EST
Please
1. download 3.5.1 from here:
http://download.eclipse.org/eclipse/downloads/drops/R-3.5.1-200909170800/index.php
2. start with a new workspace
3. try to reproduce and if you can reproduce write each single step here

Also: are you indeed using WindowsXP or some other OS? Some special machine (setupt)?
Comment 4 Josh Bloch CLA 2009-11-25 02:51:28 EST
(In reply to comment #1)
> I cannot reproduce this with the given Eclipse build and VM, and the error
> message is less than helpful. Sorry, I can't do anything here.
> 
> If you could set a breakpoint at the throw statement and then debug it in a
> target workbench, that could provide more insight on why this happens. Maybe
> the bug is really in TimSort and the IAE is a false accusation (that would at
> least explain why there's no justification in the error message).

What do you mean when you say the error message is less than helpful?  How could I make it more helpful?  The general contract for Comparator/Comparable is quite specific (transitivity, antisymmetry, etc.)  TimSort doesn't know exactly which provisions the comparator has violated, but it knows that something's amiss. The old sort silently returned the list in some unspecified order.  I quite like the fact that TimSort can recognize a bad comparison function and alert you to it.
Comment 5 Dani Megert CLA 2009-11-25 03:13:59 EST
>This is reproducible under JDK 6.
This sounds weird to me: AFAIK TimSort has its first appearance in Java 7.

>What do you mean when you say the error message is less than helpful? 
I don't know what Markus had in mind, but maybe it could print out the element(s) that caused the failure?
Comment 6 Kirill Grouchnikov CLA 2009-11-25 11:18:03 EST
(In reply to comment #5)
> >This is reproducible under JDK 6.
> This sounds weird to me: AFAIK TimSort has its first appearance in Java 7.

I meant to say that the lookup is broken under JDK 6 as well. It does not throw an exception, but the filtering is not done correctly - perhaps because of the broken comparator that results in unpredictable sort order (as Josh pointed out).

I will try to create a reproducible sequence of steps on a fresh workspace.

Thanks
Kirill
Comment 7 Kirill Grouchnikov CLA 2009-11-25 13:26:39 EST
Cannot seem to reproduce this in a brand new workspace. Let me try running the existing workspace with -clean option.

Thanks
Kirill
Comment 8 Markus Keller CLA 2009-11-25 13:59:59 EST
(In reply to comment #4)
> What do you mean when you say the error message is less than helpful?  How
> could I make it more helpful?  The general contract for Comparator/Comparable
> is quite specific (transitivity, antisymmetry, etc.)  TimSort doesn't know
> exactly which provisions the comparator has violated, but it knows that
> something's amiss.

Sorry, that sounded harsher than it was meant. I just get a "Comparison method violates its general contract!" and that could mean anything. I was expecting at least some context (e.g. a minimal collection of elements for which the comparator fails) to help me find out where the bug is. I checked our comparator again, and I didn't find any contract breakages.

> The old sort silently returned the list in some unspecified
> order.  I quite like the fact that TimSort can recognize a bad comparison
> function and alert you to it.

It's quite a drastic change when something that worked before now throws exceptions with 1.7. Maybe it didn't sort perfectly in all cases, but at least the user could continue working. I think this IAE should be replaced by an assertion such that it is not thrown in production scenarios.
Comment 9 Markus Keller CLA 2009-11-25 14:06:03 EST
> Cannot seem to reproduce this in a brand new workspace. Let me try running the
> existing workspace with -clean option.

This bug depends on the types that are available in your workspace. -clean should not have any influence on that.

If you get this again, can you copy the pattern to the clipboard, close and reopen the dialog, and then paste the pattern again? When you type it manually, there are timeouts involved that you cannot control, and if you can reproduce by pasting the pattern, that would rule a big category of possible problems.
Comment 10 Kirill Grouchnikov CLA 2009-11-25 14:19:31 EST
> This bug depends on the types that are available in your workspace. -clean
> should not have any influence on that.

Indeed no influence.

> If you get this again, can you copy the pattern to the clipboard, close and
> reopen the dialog, and then paste the pattern again? When you type it manually,
> there are timeouts involved that you cannot control, and if you can reproduce
> by pasting the pattern, that would rule a big category of possible problems.

I'm afraid that it's too late on my side. I have deleted the entire workspace and created a new one. Now i don't see this exception. So perhaps something got garbled in some internal indexed.

It now works for me as well.

Thanks
Kirill
Comment 11 Josh Bloch CLA 2009-11-25 14:38:28 EST
(In reply to comment #9)
> > Cannot seem to reproduce this in a brand new workspace. Let me try running the
> > existing workspace with -clean option.
> 
> This bug depends on the types that are available in your workspace. -clean
> should not have any influence on that.
> 
> If you get this again, can you copy the pattern to the clipboard, close and
> reopen the dialog, and then paste the pattern again? When you type it manually,
> there are timeouts involved that you cannot control, and if you can reproduce
> by pasting the pattern, that would rule a big category of possible problems.

(In reply to comment #8)
> (In reply to comment #4)
> > What do you mean when you say the error message is less than helpful?  How
> > could I make it more helpful?  The general contract for Comparator/Comparable
> > is quite specific (transitivity, antisymmetry, etc.)  TimSort doesn't know
> > exactly which provisions the comparator has violated, but it knows that
> > something's amiss.
> 
> Sorry, that sounded harsher than it was meant. I just get a "Comparison method
> violates its general contract!" and that could mean anything. I was expecting
> at least some context (e.g. a minimal collection of elements for which the
> comparator fails) to help me find out where the bug is. I checked our
> comparator again, and I didn't find any contract breakages.
> 
> > The old sort silently returned the list in some unspecified
> > order.  I quite like the fact that TimSort can recognize a bad comparison
> > function and alert you to it.
> 
> It's quite a drastic change when something that worked before now throws
> exceptions with 1.7. Maybe it didn't sort perfectly in all cases, but at least
> the user could continue working. I think this IAE should be replaced by an
> assertion such that it is not thrown in production scenarios.

That's a bit tough; if you simply take out the code that checks for it, you'll
get ArrayIndexOutOfBounds exceptions.  We did put in an undocumented "boolean"
system property that allows you to revert (globally) to the old sort, but then
you lose the performance advantage of TimSort.  The system property is
"java.util.Arrays.useLegacyMergeSort".
Comment 12 Dani Megert CLA 2010-10-13 02:17:29 EDT
*** Bug 327613 has been marked as a duplicate of this bug. ***
Comment 13 Dmitry Karasik CLA 2010-10-13 10:03:53 EDT
I can reproduce this fairly regularly with 1.7.

Do you need me to debug this for you?
Comment 14 Dani Megert CLA 2010-10-13 10:15:55 EDT
(In reply to comment #13)
> I can reproduce this fairly regularly with 1.7.
> 
> Do you need me to debug this for you?
Best would be to provide steps to reproduce. If that's not possible, then debugging it would be appreciated.
Comment 15 Dmitry Karasik CLA 2010-10-13 10:24:58 EDT
Steps to reproduce are:

1. Run eclipse under 1.7 VM
2. Bring up the Open Type dialog and quickly type random 2 characters then erase and do it over again, until the problem occurs.

Usually it takes a good number of repetitions (about 20) of step number 2 for it to happen.
Comment 16 Dani Megert CLA 2010-10-13 10:29:11 EDT
(In reply to comment #15)
> Steps to reproduce are:
> 
> 1. Run eclipse under 1.7 VM
> 2. Bring up the Open Type dialog and quickly type random 2 characters then
> erase and do it over again, until the problem occurs.
> 
> Usually it takes a good number of repetitions (about 20) of step number 2 for
> it to happen.
And that is in a new workspace? As you can see from the comments, it was workspace specific.
Comment 17 Dmitry Karasik CLA 2010-10-13 10:39:29 EDT
No this is in a workspace with about 500 java projects loaded.
Comment 18 Dani Megert CLA 2010-10-13 10:45:02 EDT
(In reply to comment #17)
> No this is in a workspace with about 500 java projects loaded.
OK, then it would be good if you could debug it and tell us what causes the IAE.
Comment 19 Dmitry Karasik CLA 2010-10-14 17:08:17 EDT
This happens because FilteredItemsSelectionDialog.applyFilter() gets called on a different thread and changes the filter instance variable while the sort is running.

This affects the result of CamelCaseComparator.getCamelCaseCategory() and results in inconsistent output from the Comparator.
Comment 20 Markus Keller CLA 2010-10-15 08:33:58 EDT
(In reply to comment #19)
Thanks a lot for your investigations. It turned out this has already been fixed for 3.6 as a side effect of bug 299954 (which removed the bogus CamelCaseComparator and thus removed the reference from the HistoryComparator to the current filter).