Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 358898

Summary: [Synchronize] Having done synchronize egit can lock up UI thread during each post-change
Product: [Technology] EGit Reporter: James Blackburn <jamesblackburn+eclipse>
Component: UIAssignee: Dariusz Luksza <dariusz.luksza>
Status: CLOSED FIXED QA Contact:
Severity: major    
Priority: P3 CC: amj87.iitr, daniel_megert, dariusz.luksza, deepakazad, ed, loskutov, markus.kell.r, matthias.sohn, pwebster, remy.suen, stephan.herrmann, stepper
Version: 1.1Keywords: performance
Target Milestone: ---   
Hardware: PC   
OS: All   
Whiteboard:

Description James Blackburn CLA 2011-09-26 10:04:35 EDT
An hour ago I did a synchronize.  I'm no longer on the Synchronize perspective, (but I guess the synch output is still there...).  Doing resource change operations, like a clean build causes eclipse to lock up for minutes.

Egit seems to be doing a lot of work as a result of resource post change delta:


"main" prio=10 tid=0x0000000040115000 nid=0x7721 runnable [0x0000000040227000]
   java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at org.eclipse.jgit.treewalk.FileTreeIterator$FileEntry.openInputStream(FileTreeIterator.java:198)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.idBufferBlob(WorkingTreeIterator.java:257)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.idBuffer(WorkingTreeIterator.java:235)
        at org.eclipse.jgit.treewalk.AbstractTreeIterator.getEntryObjectId(AbstractTreeIterator.java:409)
        at org.eclipse.jgit.treewalk.TreeWalk.getObjectId(TreeWalk.java:675)
        at org.eclipse.egit.core.synchronize.ThreeWayDiffEntry.scan(ThreeWayDiffEntry.java:93)
        at org.eclipse.egit.core.synchronize.GitSyncCache.loadDataFromGit(GitSyncCache.java:87)
        at org.eclipse.egit.core.synchronize.GitSyncCache.getAllData(GitSyncCache.java:50)
        at org.eclipse.egit.core.synchronize.GitResourceVariantTreeSubscriber.refresh(GitResourceVariantTreeSubscriber.java:140)
        at org.eclipse.team.core.subscribers.Subscriber.refresh(Subscriber.java:466)
        at org.eclipse.egit.core.synchronize.GitSubscriberMergeContext.update(GitSubscriberMergeContext.java:135)
        at org.eclipse.egit.core.synchronize.GitSubscriberMergeContext.access$0(GitSubscriberMergeContext.java:116)
        at org.eclipse.egit.core.synchronize.GitSubscriberMergeContext$2.resourceChanged(GitSubscriberMergeContext.java:70)
        at org.eclipse.core.internal.events.NotificationManager$1.run(NotificationManager.java:291)
        at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
        at org.eclipse.core.internal.events.NotificationManager.notify(NotificationManager.java:285)
        at org.eclipse.core.internal.events.NotificationManager.broadcastChanges(NotificationManager.java:149)
        at org.eclipse.core.internal.resources.Workspace.broadcastPostChange(Workspace.java:395)
        at org.eclipse.core.internal.resources.Workspace.endOperation(Workspace.java:1530)
        at org.eclipse.core.internal.resources.Project.touch(Project.java:1409)
        at org.eclipse.jdt.internal.core.JavaModelManager$EclipsePreferencesListener.preferenceChange(JavaModelManager.java:1494)
        at org.eclipse.core.internal.preferences.EclipsePreferences$2.run(EclipsePreferences.java:754)
        at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
        at org.eclipse.core.internal.preferences.EclipsePreferences.firePreferenceEvent(EclipsePreferences.java:757)
        at org.eclipse.core.internal.preferences.EclipsePreferences.put(EclipsePreferences.java:770)
        at org.eclipse.jdt.internal.core.JavaModelManager.storePreference(JavaModelManager.java:4804)
        at org.eclipse.jdt.internal.core.JavaModelManager.setOptions(JavaModelManager.java:4865)
        at org.eclipse.jdt.core.JavaCore.setOptions(JavaCore.java:5101)
        at org.eclipse.jdt.internal.ui.text.java.JavaCompletionProcessor.restrictProposalsToVisibility(JavaCompletionProcessor.java:60)
        at org.eclipse.jdt.internal.ui.text.ContentAssistPreference.configureJavaProcessor(ContentAssistPreference.java:104)
        at org.eclipse.jdt.internal.ui.text.ContentAssistPreference.configure(ContentAssistPreference.java:159)
        at org.eclipse.jdt.ui.text.JavaSourceViewerConfiguration.getContentAssistant(JavaSourceViewerConfiguration.java:412)
        at org.eclipse.jface.text.source.SourceViewer.configure(SourceViewer.java:458)
        at org.eclipse.jdt.internal.ui.javaeditor.JavaSourceViewer.configure(JavaSourceViewer.java:230)
        at org.eclipse.ui.texteditor.AbstractTextEditor.createPartControl(AbstractTextEditor.java:3407)
        at org.eclipse.ui.texteditor.StatusTextEditor.createPartControl(StatusTextEditor.java:54)
        at org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.createPartControl(AbstractDecoratedTextEditor.java:440)
        at org.eclipse.jdt.internal.ui.javaeditor.JavaEditor.createPartControl(JavaEditor.java:3098)
        at org.eclipse.jdt.internal.ui.javaeditor.CompilationUnitEditor.createPartControl(CompilationUnitEditor.java:1505)
        at org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:670)
        at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:465)
        at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
        at org.eclipse.ui.internal.EditorReference.getEditor(EditorReference.java:289)
        at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.java:2945)
        at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:2850)
        at org.eclipse.ui.internal.WorkbenchPage.access$11(WorkbenchPage.java:2842)
        at org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2793)
        at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
        at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2789)
        at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2773)
        at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2764)
        at org.eclipse.ui.ide.IDE.openEditor(IDE.java:651)
        at org.eclipse.ui.ide.IDE.openEditor(IDE.java:610)
        at org.eclipse.ui.ide.IDE.openEditor(IDE.java:1109)
        at org.eclipse.ui.internal.views.markers.ExtendedMarkersView.openMarkerInEditor(ExtendedMarkersView.java:1719)
        at org.eclipse.ui.internal.views.markers.ExtendedMarkersView.openSelectedMarkers(ExtendedMarkersView.java:1188)
        at org.eclipse.ui.internal.views.markers.ExtendedMarkersView$5.open(ExtendedMarkersView.java:578)
        at org.eclipse.ui.OpenAndLinkWithEditorHelper$InternalListener.open(OpenAndLinkWithEditorHelper.java:48)
        at org.eclipse.jface.viewers.StructuredViewer$2.run(StructuredViewer.java:866)
        at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
        at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:49)
        at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
        at org.eclipse.jface.viewers.StructuredViewer.fireOpen(StructuredViewer.java:864)
        at org.eclipse.jface.viewers.StructuredViewer.handleOpen(StructuredViewer.java:1152)
        at org.eclipse.jface.viewers.StructuredViewer$6.handleOpen(StructuredViewer.java:1256)
        at org.eclipse.jface.util.OpenStrategy.fireOpenEvent(OpenStrategy.java:275)
        at org.eclipse.jface.util.OpenStrategy.access$2(OpenStrategy.java:269)
        at org.eclipse.jface.util.OpenStrategy$1.handleEvent(OpenStrategy.java:309)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1258)
        at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3588)
Comment 1 Andrew Gvozdev CLA 2011-09-30 08:43:56 EDT
I am a fan of synchronize view and use it often. But some fairly recent improvements in synchronize view made it unbearable to use it. Egit folks claim it made the view "snappy" but once you use it the whole eclipse slows down and slows even more as time go even if you don't synchronize anymore. Setting breakpoints in java code takes like half a minute. I am getting used to avoiding synchronize view and quitting and restarting eclipse right after I use it.
Comment 2 James Blackburn CLA 2011-09-30 08:56:22 EDT
(In reply to comment #1)
> I am getting used to
> avoiding synchronize view and quitting and restarting eclipse right after I use
> it.

Yep, that's what I've been doing.  Synchronize two branches to see an overview of what's changed, and use git externally + an external text editor.  The IDE can't be used for editing files having done synchronize of CDT, unless you restart.
Comment 3 Andrew Gvozdev CLA 2011-09-30 09:07:41 EDT
(In reply to comment #2)
> (In reply to comment #1)
> > I am getting used to
> > avoiding synchronize view and quitting and restarting eclipse right after I
> > use it.
> Yep, that's what I've been doing.  Synchronize two branches to see an overview
> of what's changed, and use git externally + an external text editor.  The IDE
> can't be used for editing files having done synchronize of CDT, unless you
> restart.
Well, I have a short window when I am able to edit files in the view (usually I synchronize uncommitted changes against index) but it lasts about 5-10 min or so. That is really sad experience to share.
Comment 4 Stephan Herrmann CLA 2011-10-01 11:30:47 EDT
(In reply to comment #1)
> [...] Setting breakpoints in java code takes like half a minute.

I'm not happy with EGit performance either.

But this one looks particularly weird: is EGit really updating the world
just because a breakpoint is set??
Why should EGit be interested in marker changes?

For the time being I'd really love to see an option to turn off all
that updating that locks the UI. Really.
Comment 5 Stephan Herrmann CLA 2011-10-01 11:53:02 EDT
On a similar note EGit seems to aggravate an existing problem in compare:
while compare editors are known to leak under unknown conditions,
throwing NPE long after they're closed, only with EGit in the loop did
I also see a question dialog "The file has been modified outside the
compare editor ..." without any compare editor being open any longer.

So, if EGit has special interaction with compare editors both troubles
might multiply here.
Comment 6 Stephan Herrmann CLA 2011-10-01 12:01:06 EDT
(In reply to comment #5)
> .. "The file has been modified outside the
> compare editor ..." without any compare editor being open any longer.

I was quoting from memory. "Luckily" the error occurred again, I see:

The resources being compared have changed outside the compare editor. Do
you want to save your changes? Any unsaved changes will be discarded.
Yes / No

I know it's not the exact same problem, but also an issue of some
listeners working way past their term.
Comment 7 Remy Suen CLA 2011-10-01 13:22:10 EDT
(In reply to comment #4)
> But this one looks particularly weird: is EGit really updating the world
> just because a breakpoint is set??
> Why should EGit be interested in marker changes?

Sounds weird but this does seem to be the case. It seems the resource change listener is not ignoring marker events.

Thread [Worker-1] (Suspended)	
	FileInputStream.<init>(File) line: 96	
	FileTreeIterator$FileEntry.openInputStream() line: 198	
	FileTreeIterator(WorkingTreeIterator).idBufferBlob(WorkingTreeIterator$Entry) line: 257	
	FileTreeIterator(WorkingTreeIterator).idBuffer() line: 235	
	FileTreeIterator(AbstractTreeIterator).getEntryObjectId(MutableObjectId) line: 409	
	TreeWalk.getObjectId(MutableObjectId, int) line: 675	
	ThreeWayDiffEntry.scan(TreeWalk) line: 93	
	GitSyncCache.loadDataFromGit(GitSynchronizeData, GitSyncObjectCache) line: 87	
	GitSyncCache.getAllData(GitSynchronizeDataSet, IProgressMonitor) line: 50	
	GitResourceVariantTreeSubscriber.refresh(IResource[], int, IProgressMonitor) line: 140	
	GitResourceVariantTreeSubscriber(Subscriber).refresh(ResourceTraversal[], IProgressMonitor) line: 466	
	GitSubscriberMergeContext.update(GitResourceVariantTreeSubscriber, RepositoryMapping) line: 135	
	GitSubscriberMergeContext.access$0(GitSubscriberMergeContext, GitResourceVariantTreeSubscriber, RepositoryMapping) line: 116	
	GitSubscriberMergeContext$2.resourceChanged(IResourceChangeEvent) line: 70	
	NotificationManager$1.run() line: 291	
	SafeRunner.run(ISafeRunnable) line: 42	
	NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], IResourceChangeEvent, boolean) line: 285	
	NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 149	
	Workspace.broadcastPostChange() line: 395	
	Workspace.endOperation(ISchedulingRule, boolean, IProgressMonitor) line: 1530	
	Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 2353	
	JavaLineBreakpoint(Breakpoint).run(ISchedulingRule, IWorkspaceRunnable) line: 335	
	JavaLineBreakpoint.<init>(IResource, String, int, int, int, int, boolean, Map, String) line: 153	
	JavaLineBreakpoint.<init>(IResource, String, int, int, int, int, boolean, Map) line: 132	
	JDIDebugModel.createLineBreakpoint(IResource, String, int, int, int, int, boolean, Map) line: 274	
	ToggleBreakpointAdapter$2.run(IProgressMonitor) line: 267	
	Worker.run() line: 54
Comment 8 Stephan Herrmann CLA 2011-10-01 14:00:56 EDT
(In reply to comment #7)
> (In reply to comment #4)
> > But this one looks particularly weird: is EGit really updating the world
> > just because a breakpoint is set??
> > Why should EGit be interested in marker changes?
> 
> Sounds weird but this does seem to be the case. It seems the resource change
> listener is not ignoring marker events.

Good grief!!

Could that also be one reason why each save in a JavaEditor takes so long
with EGit in the loop? Are they listening also to those marker changes?

That's none of their business :)
Comment 9 Remy Suen CLA 2011-10-23 20:11:38 EDT
*** Bug 361763 has been marked as a duplicate of this bug. ***
Comment 10 Dariusz Luksza CLA 2011-10-24 17:17:31 EDT
I'm not sure does change[1] will improve something in this case but IMHO it is good start point to investigate on this topic

[1] http://egit.eclipse.org/r/4403
Comment 11 Remy Suen CLA 2011-10-26 20:20:38 EDT
Dariusz, what is the purpose of the resource change listener anyway? It seems to me that it should not be necessary for us to read the contents of the .git folder every time somebody saves a file. The .git folder should only be read if the branch's tip pointer changes, correct?
Comment 12 Dariusz Luksza CLA 2011-10-27 02:34:03 EDT
(In reply to comment #11)
> Dariusz, what is the purpose of the resource change listener anyway? It seems
> to me that it should not be necessary for us to read the contents of the .git
> folder every time somebody saves a file. The .git folder should only be read if
> the branch's tip pointer changes, correct?

Here we are not interested in changes in repository but in changes in workspace and git index. Resource change event indicates that we should refresh results in synchronize view (and recalculate what changes in git index). Now this seams to be little bit obsolete solution, since for five weeks we have EGit specific IndexDiffChangeListener with could be used here.

Also we could replace FileTreeIterator while building GitSyncCache by reusing data in IndexDiffCache with is used for calculating decorators.

I'll look on this solution later this week.
Comment 13 Remy Suen CLA 2011-10-27 06:25:24 EDT
(In reply to comment #12)
> Here we are not interested in changes in repository but in changes in workspace
> and git index. Resource change event indicates that we should refresh results
> in synchronize view (and recalculate what changes in git index).

Okay, that makes sense to me. However, the resource change listener ultimately invokes GitSyncCache's getAllData(*) method which calls loadDataFromGit(*) and a full diff between commits are performed. If one file changes then the diff should only check that one file and not diff the entire tree.
Comment 14 Remy Suen CLA 2011-10-27 07:01:58 EDT
(In reply to comment #13)
> However, the resource change listener ultimately
> invokes GitSyncCache's getAllData(*) method which calls loadDataFromGit(*) and
> a full diff between commits are performed. If one file changes then the diff
> should only check that one file and not diff the entire tree.

I've attached a change set to Gerrit for this, Dariusz. It probably needs some work but I hope it illustrates what I mean.
http://egit.eclipse.org/r/#change,4417
Comment 15 Dariusz Luksza CLA 2011-11-18 18:21:09 EST
Merged as commit 3edaac451ef0a2977f3f250f827c4ed19400f7b6
Comment 16 Remy Suen CLA 2011-11-24 14:43:57 EST
*** Bug 364685 has been marked as a duplicate of this bug. ***