Bugzilla – Full Text Bug Listing
| Summary: | ConcurrentModificationException in JavaModelManager.initializeAllContainers | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] JDT | Reporter: | Kevan Holdaway <kholdaway> | ||||
| Component: | Core | Assignee: | Andrey Loskutov <loskutov> | ||||
| Status: | VERIFIED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | P3 | CC: | jarthana, loskutov, mauromol, morrisj, Olivier_Thomann, shr31223, udo.walker, Vikas.Chandra | ||||
| Version: | 3.6.1 | ||||||
| Target Milestone: | 4.8 M7 | ||||||
| Hardware: | All | ||||||
| OS: | All | ||||||
| See Also: |
https://git.eclipse.org/r/119187 https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=ee652a24af3d74f53b78df6582d6a7a6fb8fdefc |
||||||
| Whiteboard: | |||||||
| Attachments: |
|
||||||
|
Description
Kevan Holdaway
Jay, please investigate. Is there a chance you can provide attach the log file when the error occurs, preferably with debug on for the following? org.eclipse.jdt.core/debug/cpresolution org.eclipse.jdt.core/debug/cpresolution/advanced org.eclipse.jdt.core/debug/cpresolution/failure Thanks! Created attachment 180884 [details]
Error Logs of problem
(In reply to comment #2) > Is there a chance you can provide attach the log file when the error occurs, > preferably with debug on for the following? > > org.eclipse.jdt.core/debug/cpresolution > org.eclipse.jdt.core/debug/cpresolution/advanced > org.eclipse.jdt.core/debug/cpresolution/failure > > Thanks! You can see for the two exceptions in the logs that 2 threads are inside of the method org.eclipse.jdt.internal.core.JavaModelManager$11.run(Unknown Source) and both of them are modifying the same hashmap. That should be enough to determine there is no thread safety protection on that method. Increasing severity to match the severity of the bug in our product. Basically, this seems to have a wide impact on JDT operations when this occurs. It requires a restart of the product to fix it. (In reply to comment #4) > You can see for the two exceptions in the logs that 2 threads are inside of the > method org.eclipse.jdt.internal.core.JavaModelManager$11.run(Unknown Source) > and both of them are modifying the same hashmap. That should be enough to > determine there is no thread safety protection on that method. Assuming that the internal maps contained by JavaModelManager#containersBeingInitialized is the ones that are affected here - containersBeingInitialized itself is in ThreadLocal- under normal circumstances these maps will be modified while being iterated in the code after line no 2694 (after comment // Set all containers). So, I think there is another thread involved here, which has modified the map in question. So, I was hoping to find some clues from the log file that would lead us to the exact scenario that led to this. Will see what we can do here without affecting the performance much. (In reply to comment #6) > So, I think there is another thread involved here, which has modified the map > in question. So, I was hoping to find some clues from the log file that would > lead us to the exact scenario that led to this. Will see what we can do here > without affecting the performance much. Oops! Sorry, having mentioned the ThreadLocal, the chances of an involvement from another thread is almost ruled out. Will take a close look at the stack trace and see what the possibilities of the map being modified in the same thread. (In reply to comment #7) > (In reply to comment #6) > > So, I think there is another thread involved here, which has modified the map > > in question. So, I was hoping to find some clues from the log file that would > > lead us to the exact scenario that led to this. Will see what we can do here > > without affecting the performance much. > > Oops! Sorry, having mentioned the ThreadLocal, the chances of an involvement > from another thread is almost ruled out. Will take a close look at the stack > trace and see what the possibilities of the map being modified in the same > thread. I don't think that's true. Notice is the Iterator.next() that is causing you the issue. You only use ThreadLocal to acquire the map, but not to iterate it. ThreadLocal is not a lock and doesn't create a synchronization block. Its possible that 2 threads get the map using ThreadLocal then proceed and both start iterating. (In reply to comment #8) > I don't think that's true. Notice is the Iterator.next() that is causing you > the issue. You only use ThreadLocal to acquire the map, but not to iterate it. > ThreadLocal is not a lock and doesn't create a synchronization block. Its > possible that 2 threads get the map using ThreadLocal then proceed and both > start iterating. This is what I meant: Since we are using ThreadLocal to store/retrieve the outer map itself, won't each thread get it's own copy of the map? I don't see this map or the internal map (containers for each project) being passed around, which should take care of the concurrency issue. Let me know if I have missed something. Kevan, I could really use the log file with the DEBUG options enabled. Theoretically there are not many scenarios when the map would be modified from the same thread. The log file, with container initialization and things like that could throw some light on this. (In reply to comment #10) > Kevan, I could really use the log file with the DEBUG options enabled. > Theoretically there are not many scenarios when the map would be modified from > the same thread. The log file, with container initialization and things like > that could throw some light on this. I will try to gather some more information (using the debug options). (In reply to comment #2) > Is there a chance you can provide attach the log file when the error occurs, > preferably with debug on for the following? > > org.eclipse.jdt.core/debug/cpresolution > org.eclipse.jdt.core/debug/cpresolution/advanced > org.eclipse.jdt.core/debug/cpresolution/failure > > Thanks! Can you translate these instruction into what I would need to put in the eclipse.ini file? At first I assumed it was just -Dorg.eclipse.jdt.core/debug/cpresolution=true but ... that didn't work. How can I enable this without a source build? Assume I can only launch eclipse and tweak it using args. You need to save the following contents in a file called ".option" that you put in the same folder of your eclipse.exe. org.eclipse.jdt.core/debug=true org.eclipse.jdt.core/debug/cpresolution/failure=true org.eclipse.jdt.core/debug/cpresolution=true org.eclipse.jdt.core/debug/cpresolution/advanced=true (In reply to comment #13) > You need to save the following contents in a file called ".option" that you put > in the same folder of your eclipse.exe. > > org.eclipse.jdt.core/debug=true > org.eclipse.jdt.core/debug/cpresolution/failure=true > org.eclipse.jdt.core/debug/cpresolution=true > org.eclipse.jdt.core/debug/cpresolution/advanced=true I'm seeing a similar problem. I already created the .option (or it is .options?). What will I see with this options turned on? More information in the Eclipse log? or additional logs? (In reply to comment #14) > I'm seeing a similar problem. I already created the .option (or it is > .options?). What will I see with this options turned on? More information in > the Eclipse log? or additional logs? It is .options and the eclipse log will tell us what's going on with the container initialization, whether there are any failures etc. *** Bug 530765 has been marked as a duplicate of this bug. *** *** Bug 367440 has been marked as a duplicate of this bug. *** I see from Aeri that this is still a problem, https://dev.eclipse.org/recommenders/committers/aeri/v2/#!/problems/55c30ad5e4b0d6907d414f15 Source in initializeAllContainers() at commit 1114cb59520dae024355aa887ab8fe408024a596 (4.7.2) looks like: Map perProjectContainers = (Map) JavaModelManager.this.containersBeingInitialized.get(); if (perProjectContainers != null) { Iterator entriesIterator = perProjectContainers.entrySet().iterator(); while (entriesIterator.hasNext()) { Map.Entry entry = (Map.Entry) entriesIterator.next(); // error here, line 3033 IJavaProject project = (IJavaProject) entry.getKey(); HashMap perPathContainers = (HashMap) entry.getValue(); Iterator containersIterator = perPathContainers.entrySet().iterator(); while (containersIterator.hasNext()) { Map.Entry containerEntry = (Map.Entry) containersIterator.next(); // error here, line 3038 IPath containerPath = containerEntry.getKey(); IClasspathContainer container = containerEntry.getValue(); SetContainerOperation operation = new SetContainerOperation(containerPath, new IJavaProject[] {project}, new IClasspathContainer[] {container}); operation.runOperation(monitor); // root cause, line 3042 The errors are reported for both places at lines 3033 and line 3038 (entriesIterator.next() and containersIterator.next()). containersBeingInitialized is a ThreadLocal, so how we can have a ConcurrentModificationException here? The fun is that we don't have *different* threads issue, we have the issue with the modifying maps *while iterating over them*. The line 3042 (operation.runOperation()) is the root cause. It invalidates the iterators by changing the underlined maps *in same* thread. New Gerrit change created: https://git.eclipse.org/r/119187 (In reply to Eclipse Genie from comment #19) > New Gerrit change created: https://git.eclipse.org/r/119187 Jay, I think I have a fix. Gerrit change https://git.eclipse.org/r/119187 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=ee652a24af3d74f53b78df6582d6a7a6fb8fdefc Verified by code inspection. |