Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 282436

Summary: [performance][context] remove From Context hangs eclipse
Product: z_Archived Reporter: Alan Berezin <aberezin>
Component: MylynAssignee: Shawn Minto <shawn.minto>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P2 CC: aberezin, davidmichaelkarr, jared, mik.kersten, shawn.minto, steffen.pingel
Version: 3.2   
Target Milestone: 3.4.2   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on:    
Bug Blocks: 275512    
Attachments:
Description Flags
Stack trace
none
patch
none
mylyn/context/zip
none
patch
none
mylyn/context/zip none

Description Alan Berezin CLA 2009-07-04 15:02:02 EDT
Build ID: 20090621-0832

Steps To Reproduce:
1. Setup 2 very large (20K files) projects, call then p1 and p2.
2. Create a task. Add a few resources from p1 and a perhaps 100 from p2 to the context.
3. view resources from Navigator with Focus On Active Task. You should see a small tree of p1 items and a larger tree of p2 items.
4. Right click on p1 and Remove From Context
5. Entire UI hangs for at least 30 min.  I have to kill it.

More information:
After killing, restarting, and repeating 3 times, I can reproduce this with my existing workspace ( I didnt create the projects each time since they are already there).  There is nothing is the logs indicating a problem.

I can successfully removeFromContext leaf nodes, one at a time, in the navigator tree (it takes 10-120 seconds).  But even when all node are removed except for the project itself, removeing the project hangs the UI.

It doesnt seem to matter what filter I have for the workingSet (I have them arranged by project and one filter for ALL).

This is a fresh Galileo j2ee install with Mylyn 3.2.0-v20090617-0100-e3x and the jira plugin (installed using the mylyn addl connectors option) and the tigris svn plugin for mylyn.
Comment 1 Alan Berezin CLA 2009-07-04 15:03:55 EDT
I forgot to note that one whole cpu core is maxed while the UI is hung.  There is no indicator of a backround task.
Comment 2 Alan Berezin CLA 2009-07-04 16:47:16 EDT
I dont know if this is related or should be reported separately but there is an intolerable UI performance issue with this workspace.  If I have a task editor open (though not necessarily visible) for the current task, I see in the Context tab a section titled '51098 Invisble Elements'.
It appears that whenever performing almost any edit in a java editor, an eclipse background task is spawned, appears in the Progress window as 'Updating Invisible Elements In Context Page', and lives for perhaps 200ms-2s. This causes an intolerable UI hang for a few hundred ms to a few seconds on almost any java edit making the whole UI unusable.  It seems that if I simply close the task editor, the performance problem is gone.

When I reopen the task editor, the problem doesnt immediately reappear but it may reappear if, in the task editor, I open the Context tab and then proceed to edit some java code. The simple workaround is to keep closed the task editor for the infocus task. But it took quite a bit of frustrating UI slowness for me to dig in and find this workaround. 

Comment 3 Mik Kersten CLA 2009-07-23 12:39:38 EDT
Shawn: Could you investigate?
Comment 4 Shawn Minto CLA 2009-07-23 15:34:50 EDT
Alan, Would you be able to upgrade to Mylyn 3.2.1 and see if this problem still exists?  There was a bug in 3.2.0 that could cause multiple expansions when removing from context which could affect the performance.
Comment 5 Alan Berezin CLA 2009-07-23 16:01:18 EDT
I will give it a try tonight and let you know.
Comment 6 Alan Berezin CLA 2009-07-23 18:16:49 EDT
I upgraded to Mylyn 3.2.1 and repeated the steps and eclipse hangs for approx 4 minutes (app doesnt even redraw the screen).  The app seems to peg one whole core for that time.  However, the app does recover.  I dont know if this is better than before because my context is not exactly the same as the original test.
I noted that there was only 1 'invisible' element in this case.
Comment 7 Shawn Minto CLA 2009-07-23 18:54:09 EDT
Alan,

About how many elements were you removing from the context?  Also, did you have the context page of the task editor open when performing this?  If so, could you try with the task editor closed and see if you get the same results?
Comment 8 Alan Berezin CLA 2009-07-23 19:17:47 EDT
There were 2 elements being removed when I choose Remove From Context on the P1 project.  The other project had approx 50 elements.
I repeated the experiment with the task editor closed at the operation is fast, approx 2 seconds.  I repeated it again with the editor open but not in focus and again it hangs.  Let me know if there is any other testing you need.
Comment 9 Steffen Pingel CLA 2010-07-09 00:49:30 EDT
*** Bug 309555 has been marked as a duplicate of this bug. ***
Comment 10 Marc-André Laperle CLA 2010-07-12 23:45:01 EDT
Created attachment 174112 [details]
Stack trace

Using Mylyn 3.4.0

I also have problems with the performance of Remove From Context (Project Explorer). My workspace is pretty large (100K+ files). At first I had one project with a context containing around 10 files. I created a second project (CDT), very big. Now if I activate my task, open my second project, the whole project tree is in light gray, so I have to remove from context which is very slow. Even if I do it folder by folder, it's still very slow. I'm attaching a stack but I don't think it's very useful.
Comment 11 Jared CLA 2010-08-19 11:43:06 EDT
I can confirm this bug as well.

We have an extremely large, very, very old repository containing 87k files. We also use an unstable mainline approach to build management, so we have our mainline trunk and at this point I have 2 other branches checked out. So I have roughly 87x*3= +/- 250k files.

Whenever I do a ^H search or an SVN update with an active task, the context absorbs every file that scores a hit in the search or that's updated via SVN. That's bad enough, but it's a different bug. The big deal is the fact that when I go to drop the extraneous files out of my context it can take up to 15 minutes for the process to finish.

As the others noted, dropping individual assets is instant. Even dropping large folders (containing hundreds or thousands of items) out of the context is pretty quick. But if I try to remove one of the root folders or the project itself from the context I'm looking at 15 minutes or so. Also worth noting is the fact that the time doesn't seem to be any longer if I ctrl-click more than one build in the context window. So if I have files from 3 projects (where project == build, I check each build out into it's own project) in a context and I only want one of them there, if I ctrl-click the other two and do Remove From Context it's about 15 minutes to finish. If I do them one at a time, it's 15 minutes each.

Also worth noting is the fact that we use Subversion, JIRA, Fisheye and Crucible and I have the Atlassian connectors for Mylyn installed.
Comment 12 Shawn Minto CLA 2010-09-02 17:17:29 EDT
Created attachment 178105 [details]
patch

Here is a patch that I will commit to HEAD and backport that should improve the performance of this operation by batching the elements that are to be removed from the context and only notifying the change once.  Previously, every element that was selected was processed separately.
Comment 13 Shawn Minto CLA 2010-09-02 17:17:31 EDT
Created attachment 178106 [details]
mylyn/context/zip
Comment 14 Shawn Minto CLA 2010-09-08 18:31:13 EDT
Created attachment 178461 [details]
patch

It turns out that there was another problem here.  When removing items from context, it wasn't properly checked if the element was a part of the context or not, so it would try tro remove all elements in the tree that it could find.  I want to do a bit more testing, but the test case that I added originally had 17 changes when the project was removed from the context and it was the only item in the context.
Comment 15 Shawn Minto CLA 2010-09-08 18:31:17 EDT
Created attachment 178462 [details]
mylyn/context/zip
Comment 16 Shawn Minto CLA 2010-09-08 19:11:10 EDT
patch applied to HEAD. Still need to backport.
Comment 17 Shawn Minto CLA 2010-09-08 19:13:44 EDT
Applied to the branch.
Comment 18 Steffen Pingel CLA 2010-09-09 19:10:40 EDT
Looks like we are done here. A build with this fix is now available from the maintenance update site.