Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 326555 - Dead lock when debug session starts
Summary: Dead lock when debug session starts
Status: RESOLVED FIXED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.3 M3   Edit
Assignee: David McKnight CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 326712 327070
  Show dependency tree
 
Reported: 2010-09-29 11:38 EDT by Samuel Wu CLA
Modified: 2010-11-09 10:53 EST (History)
1 user (show)

See Also:


Attachments
patch to not wait when on main thread (1.56 KB, patch)
2010-09-29 17:09 EDT, David McKnight CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Samuel Wu CLA 2010-09-29 11:38:43 EDT
Build Identifier: org.eclipse.rse.core_3.2.1.v201008191400-7a7FFMXFC7sRd_SjaidfqT

When debug session starts, source lookup and source line breakpoint loading runs into a deadlock.

Reproducible: Always

Steps to Reproduce:
1. Start a debug session with the host
2. Do the source look up through RSE remote folder 
3. Set a line break point in the source file
4. Terminate the debug session
5. Disconnect the RSE connection and clear out the cached/saved password
6. Switch to the Debug perspective and show the Breakpoints view
7. Restart the debug session.
8. When the prompt for user ID comes up, the GUI is locked. 
Here is the stack 
java.lang.Object.wait(Native Method)
		java.lang.Object.wait(Object.java:167)
		org.eclipse.rse.core.subsystems.SubSystem$ConnectorServicePool.waitUntilNotContained(SubSystem.java:2467)
		org.eclipse.rse.core.subsystems.SubSystem.connect(SubSystem.java:2508)
		org.eclipse.rse.core.subsystems.SubSystem.checkIsConnected(SubSystem.java:568)
		org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.getRemoteFileObject(FileServiceSubSystem.java:237)
		com.ibm.etools.systems.launch.sourceLookup.AbstractRemoteSourceContainer.findSourceElements(AbstractRemoteSourceContainer.java:287)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupParticipant.findSourceElements(AbstractSourceLookupParticipant.java:67)
		com.ibm.debug.pdt.core.sourcelocator.PDTSourceLookupParticipant.findSource(PDTSourceLookupParticipant.java:154)
		com.ibm.debug.pdt.core.sourcelocator.PDTSourceLookupParticipant.findSourceElements(PDTSourceLookupParticipant.java:122)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupDirector$SourceLookupQuery.run(AbstractSourceLookupDirector.java:141)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupDirector.findSourceElements(AbstractSourceLookupDirector.java:742)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.lookupSourceElementIgnoringDuplicates(PDTModelPresentation.java:546)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.lookupSourceElement(PDTModelPresentation.java:517)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.getEditorInput(PDTModelPresentation.java:262)
		org.eclipse.debug.internal.ui.LazyModelPresentation.getEditorInput(LazyModelPresentation.java:237)
		org.eclipse.debug.internal.ui.DelegatingModelPresentation.getEditorInput(DelegatingModelPresentation.java:169)
		org.eclipse.debug.internal.ui.actions.breakpoints.OpenBreakpointMarkerAction.selectionChanged(OpenBreakpointMarkerAction.java:91)
		org.eclipse.ui.actions.SelectionProviderAction.selectionChanged(SelectionProviderAction.java:143)
		org.eclipse.jface.viewers.Viewer$2.run(Viewer.java:162)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:49)
		org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
		org.eclipse.jface.viewers.Viewer.fireSelectionChanged(Viewer.java:160)
		org.eclipse.jface.viewers.StructuredViewer.updateSelection(StructuredViewer.java:2162)
		org.eclipse.jface.viewers.StructuredViewer.setSelection(StructuredViewer.java:1699)
		org.eclipse.jface.viewers.TreeViewer.setSelection(TreeViewer.java:1139)
		org.eclipse.debug.internal.ui.viewers.model.InternalTreeModelViewer.trySelection(InternalTreeModelViewer.java:1700)
		org.eclipse.debug.internal.ui.viewers.model.TreeModelContentProvider.handleSelect(TreeModelContentProvider.java:374)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateNodes(ModelContentProvider.java:1375)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateNodes(ModelContentProvider.java:1380)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateModel(ModelContentProvider.java:1327)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.doModelChanged(ModelContentProvider.java:1298)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.modelChanged(ModelContentProvider.java:1277)
		org.eclipse.debug.internal.ui.viewers.provisional.AbstractModelProxy$1.run(AbstractModelProxy.java:87)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.debug.internal.ui.viewers.provisional.AbstractModelProxy.fireModelChanged(AbstractModelProxy.java:91)
		org.eclipse.debug.internal.ui.viewers.update.BreakpointManagerProxy$1.runInUIThread(BreakpointManagerProxy.java:173)
		org.eclipse.ui.progress.UIJob$1.run(UIJob.java:95)
		org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
		org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
		org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
		org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
		org.eclipse.jface.window.Window.runEventLoop(Window.java:825)
		org.eclipse.jface.window.Window.open(Window.java:801)
		org.eclipse.rse.ui.subsystems.StandardCredentialsProvider$PromptForCredentials.run(StandardCredentialsProvider.java:96)
		org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:179)
		org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:150)
		org.eclipse.swt.widgets.Display.syncExec(Display.java:4584)
		org.eclipse.rse.ui.subsystems.StandardCredentialsProvider.promptForCredentials(StandardCredentialsProvider.java:405)
		org.eclipse.rse.ui.subsystems.StandardCredentialsProvider.acquireCredentials(StandardCredentialsProvider.java:228)
		org.eclipse.rse.core.subsystems.AuthenticatingConnectorService.acquireCredentials(AuthenticatingConnectorService.java:189)
		org.eclipse.rse.core.subsystems.SubSystem.promptForPassword(SubSystem.java:2624)
		org.eclipse.rse.core.subsystems.SubSystem$1.run(SubSystem.java:2518)
		org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:179)
		org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:150)
		org.eclipse.swt.widgets.Display.syncExec(Display.java:4584)
		org.eclipse.rse.core.subsystems.SubSystem.connect(SubSystem.java:2514)
		org.eclipse.rse.core.subsystems.SubSystem.checkIsConnected(SubSystem.java:568)
		org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.getRemoteFileObject(FileServiceSubSystem.java:237)
		com.ibm.etools.systems.launch.sourceLookup.AbstractRemoteSourceContainer.findSourceElements(AbstractRemoteSourceContainer.java:287)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupParticipant.findSourceElements(AbstractSourceLookupParticipant.java:67)
		com.ibm.debug.pdt.core.sourcelocator.PDTSourceLookupParticipant.findSource(PDTSourceLookupParticipant.java:154)
		com.ibm.debug.pdt.core.sourcelocator.PDTSourceLookupParticipant.findSourceElements(PDTSourceLookupParticipant.java:122)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupDirector$SourceLookupQuery.run(AbstractSourceLookupDirector.java:141)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.debug.core.sourcelookup.AbstractSourceLookupDirector.findSourceElements(AbstractSourceLookupDirector.java:742)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.lookupSourceElementIgnoringDuplicates(PDTModelPresentation.java:546)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.lookupSourceElement(PDTModelPresentation.java:517)
		com.ibm.debug.pdt.internal.ui.PDTModelPresentation.getEditorInput(PDTModelPresentation.java:262)
		org.eclipse.debug.internal.ui.LazyModelPresentation.getEditorInput(LazyModelPresentation.java:237)
		org.eclipse.debug.internal.ui.DelegatingModelPresentation.getEditorInput(DelegatingModelPresentation.java:169)
		org.eclipse.debug.internal.ui.actions.breakpoints.OpenBreakpointMarkerAction.selectionChanged(OpenBreakpointMarkerAction.java:91)
		org.eclipse.ui.actions.SelectionProviderAction.selectionChanged(SelectionProviderAction.java:143)
		org.eclipse.jface.viewers.Viewer$2.run(Viewer.java:162)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:49)
		org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
		org.eclipse.jface.viewers.Viewer.fireSelectionChanged(Viewer.java:160)
		org.eclipse.jface.viewers.StructuredViewer.updateSelection(StructuredViewer.java:2162)
		org.eclipse.jface.viewers.StructuredViewer.setSelection(StructuredViewer.java:1699)
		org.eclipse.jface.viewers.TreeViewer.setSelection(TreeViewer.java:1139)
		org.eclipse.debug.internal.ui.viewers.model.InternalTreeModelViewer.trySelection(InternalTreeModelViewer.java:1700)
		org.eclipse.debug.internal.ui.viewers.model.TreeModelContentProvider.handleSelect(TreeModelContentProvider.java:374)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateNodes(ModelContentProvider.java:1375)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateNodes(ModelContentProvider.java:1380)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.updateModel(ModelContentProvider.java:1327)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.doModelChanged(ModelContentProvider.java:1298)
		org.eclipse.debug.internal.ui.viewers.model.ModelContentProvider.modelChanged(ModelContentProvider.java:1277)
		org.eclipse.debug.internal.ui.viewers.provisional.AbstractModelProxy$1.run(AbstractModelProxy.java:87)
		org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
		org.eclipse.debug.internal.ui.viewers.provisional.AbstractModelProxy.fireModelChanged(AbstractModelProxy.java:91)
		org.eclipse.debug.internal.ui.viewers.update.BreakpointManagerProxy$1.runInUIThread(BreakpointManagerProxy.java:173)
		org.eclipse.ui.progress.UIJob$1.run(UIJob.java:95)
		org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
		org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
		org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4041)
		org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3660)
		org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2629)
		org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2593)
		org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2427)
		org.eclipse.ui.internal.Workbench$7.run(Workbench.java:670)
		org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
		org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:663)
		org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
		org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
		org.eclipse.e

It seems that when password prompt is opened by the source lookup for the connection request and waiting for the user's input, Display.readAndDispatch() starts to other jobs on UI thread. The breakpoint starts to get loaded which also need access to the source code. The breakpoint waits on the UI thread and blocks the password input from the user.
Comment 1 David McKnight CLA 2010-09-29 12:57:34 EDT
Are you trying to debug a program on a different system from the system that contains the files?  Otherwise, I'm not sure why there would be an attempt to connect when you're debugging.  I tried to reproduce this by debugging a program on one system A with source lookup on a disconnected system B but I didn't hit this.
Comment 2 Samuel Wu CLA 2010-09-29 13:07:08 EDT
1. The debuggable application and the source are on two different systems. 
2. Disconnect the system and make the breakpoint view visible before the debug seesion is started. That's how we reproduced it.
Comment 3 Samuel Wu CLA 2010-09-29 14:28:05 EDT
When we replacee the following line
while (contains(cs)){ // wait until the connector service is no longer in the list
with
while (contains(cs) && Display.findDisplay(Thread.currentThread()) == null){ // 
wait until the connector service is no longer in the list
in SubSystem, the problem seems to go away
Comment 4 David McKnight CLA 2010-09-29 15:23:04 EDT
(In reply to comment #2)
> 1. The debuggable application and the source are on two different systems. 
> 2. Disconnect the system and make the breakpoint view visible before the debug
> seesion is started. That's how we reproduced it.

I tried that but I still can't reproduce the bug you're hitting.  Is there something special about the breakpoints in your breakpoint view?
Comment 5 David McKnight CLA 2010-09-29 15:24:51 EDT
(In reply to comment #3)
> When we replacee the following line
> while (contains(cs)){ // wait until the connector service is no longer in the
> list
> with
> while (contains(cs) && Display.findDisplay(Thread.currentThread()) == null){ // 
> wait until the connector service is no longer in the list
> in SubSystem, the problem seems to go away

So, in other words, it works for you if the code is changed to not wait if you're on the main thread.  I still would like to be able to reproduce this.
Comment 6 Samuel Wu CLA 2010-09-29 16:29:02 EDT
Our tester just found another scenario for this same problem. It happened our project tried to resolve a remote filter by RSE. As the password prompt was waiting for the user input, another filter resolving request was sent to RSE. The sourece lookup and breakpoint scenario is just one of the situation.

There are at least two more people in my team, not including me, who ran into the same problem. I reproduced it in my runtime workbench. Please let me know if you want to take a look.
Comment 7 David McKnight CLA 2010-09-29 16:37:59 EDT
(In reply to comment #6)
> Our tester just found another scenario for this same problem. It happened our
> project tried to resolve a remote filter by RSE. As the password prompt was
> waiting for the user input, another filter resolving request was sent to RSE.
> The sourece lookup and breakpoint scenario is just one of the situation.
> 
> There are at least two more people in my team, not including me, who ran into
> the same problem. I reproduced it in my runtime workbench. Please let me know
> if you want to take a look.

I was able to reproduce it, when I had put the Remote Systems view in the Debug perspective, launched once, disconnected the connection for the source and then launched it again.  I also confirmed that the change that checks for the main thread in the wait loop resolves the hang.  Does it resolve the hang in the other scenarios your tester hit?
Comment 8 Samuel Wu CLA 2010-09-29 16:56:07 EDT
yes, both my scenarios are fixed by the check
Comment 9 David McKnight CLA 2010-09-29 17:09:02 EDT
Created attachment 179895 [details]
patch to not wait when on main thread

Samuel, are you okay with this patch?
Comment 10 Samuel Wu CLA 2010-09-30 09:14:34 EDT
Thank you for your prompt response, Dave. Both my problems are fixed by the patch. I'll open a patch request for porting it to 3.6.1
Comment 11 David McKnight CLA 2010-09-30 16:28:29 EDT
I've committed the change to cvs and opened bug 326712 for the backport.
Comment 12 Martin Oberhuber CLA 2010-10-06 04:13:24 EDT
This was originally released to TM 3.3 -- I'm going to file a bug for the R3_2_maintenance backport.