Community
Participate
Working Groups
It takes around 80 seconds to synchronize two branches of the jgit repository. A quick analysis resulted in the following: During synchronize the eclipse method SynchronizationContentProvider.getElements(Object parent) is called many times with GitChangeSetModelProvider as parameter. Each time the method getChildren is called on a new instance of GitModelRepository. Therefore the cache of the children (the incoming and ougoing commits) does not work and they are newly calculated each time. I do not believe that that is the only source of low performance. But I really wonder why SynchronizationContentProvider.getElements(Object parent) is called that often by the framework.
The CDT repository also suffers really badly from this (I'd go so far as to say it's unusable). It might make an interesting test... I'm trying to look at changes Andrew's made on his branch: git clone https://github.com/angvoz/SD80.git From a shell it takes about a second: time git diff origin/origin > diff real 0m1.626s Eclipse synchronize with origin/origin (from the Repsoitories view) is still going after 10+ mins.
Should add this is with current nightly: Eclipse EGit (Incubation) 0.11.0.201101110912 org.eclipse.egit.feature.group Eclipse JGit (Incubation) 0.11.0.201101101040 org.eclipse.jgit.feature.group
Let it run all the way through -- it took 33 mins in all. Andrew: is this faster for you, or do you just diff outside of eclipse?
Never had patience to finish. I use hint from http://wiki.eclipse.org/CDT/Developer/FAQ#Git and apply the diff to CVS repository.
Thanks. I think this is a blocker as without synchronise Egit can't be used by CDT devs as nearly all workflows pass through the synchronise view.
These commits should help... 36f26c646cb4659785b7bcd79807cf6c29b37dba ce7037c0aa77f2cad1bae78f3f1dad7969c0212c
I assume these are in the nightly build? It works really fast now which is really great. Except that synchronize HEAD+uncommitted changes against HEAD lists all files in repository now even those which have not been changed. Eclipse EGit (Incubation) 0.11.0.201101200639 org.eclipse.egit.feature.group Eclipse JGit (Incubation) 0.11.0.201101191647 org.eclipse.jgit.feature.group
(In reply to comment #7) > I assume these are in the nightly build? It works really fast now which is > really great. Except that synchronize HEAD+uncommitted changes against HEAD > lists all files in repository now even those which have not been changed. In deed, it seams that first patch (ce7037c0aa77f2cad1bae78f3f1dad7969c0212c) breaks workspace model. I'll try to fix it ASAP ...
Synchronization between branches works fine. However, synchronization between working copy and local HEAD gives bad results. I changed about 10 files (not commited, not staged), but egit says 15000+ outgoing changes. If I compare speed with git gui the latter looks significantly faster. (Tried with latest - today's build of egit)
I've pushed two changes into gerrit: http://egit.eclipse.org/r/2296 http://egit.eclipse.org/r/2297 #2296 revers previously made changes. Change #2297 add new performance improvements into sync-view so that there is no difference in synchronization time after removing previous made improvements.
Next portion of synchronization improvements commits: 43d18e25a279f12ad103f47b57b4dd9d2d8fa91d f7f8ef8b0f4d43f471bc50eb57115a0b41f3b18d f47a4d0a8793bfd77c36cb172ca247929c58c3ba
Nice! These changes reduced synchronization time for my case from >30min to <5min. And, importantly, I can start working on the differences right away even while it is still scanning the rest of files. Now only if "Open" in context menu would work (bug 328902). :)
5 mins is still a very long time...
(In reply to comment #13) > 5 mins is still a very long time... In deed 5 mins it is huge amount of time for synchronization therefore we'll continue hacking on sync performance.
Thanks for working at this Dariusz!
Next performance improvements: 15e0da03955f554d642d2afee779a09ed3d376bb
Created attachment 187929 [details] screenshot When I tried local synchronize I have got this funny message (see screenshot) After clicking Show All I've got StackOverflowError.
Created attachment 187930 [details] error log
(In reply to comment #17) > Created attachment 187929 [details] > screenshot > > When I tried local synchronize I have got this funny message (see screenshot) > After clicking Show All I've got StackOverflowError. I spot this issue yesterday. Fix for it is waiting in gerrit to be approved: http://egit.eclipse.org/r/#change,2374
(In reply to comment #19) > (In reply to comment #17) > > Created attachment 187929 [details] [details] > > screenshot > > > > When I tried local synchronize I have got this funny message (see screenshot) > > After clicking Show All I've got StackOverflowError. > > I spot this issue yesterday. Fix for it is waiting in gerrit to be approved: > http://egit.eclipse.org/r/#change,2374 this was merged as 5024c9193da8b3987d7651f6d97d961fea87458b also reducing priority as performance was massively improved (still room for more improvements though)
(In reply to comment #20) > also reducing priority as performance was massively improved (still room for > more improvements though) Down to 30 sec in my scenario - from >30 min initially! Great job.
Oh gosh, synchronize view is back to being very slow. I am considering uninstalling the last build. Eclipse EGit (Incubation) 0.12.0.201103151851 org.eclipse.egit.feature.group Eclipse JGit (Incubation) 0.12.0.201103151516 org.eclipse.jgit.feature.group
(In reply to comment #22) > Oh gosh, synchronize view is back to being very slow. I am considering > uninstalling the last build. > > Eclipse EGit (Incubation) 0.12.0.201103151851 > org.eclipse.egit.feature.group > Eclipse JGit (Incubation) 0.12.0.201103151516 > org.eclipse.jgit.feature.group It seams that commit 6c5e2ea0 broke synchronization performance. I've an idea how this fix can be improved so that it doesn't reduce the synchronization performance. I'll implement it and push it ASAP.
This[1] patch should fix performance issues without braking fix introduced by commit 6c5e2ea0 [1] http://egit.eclipse.org/r/#change,2741
I've created a simple test case for synchronization performance (http://egit.eclipse.org/r/#change,2768), it simply adds 1000 commits into repository, each commit contains 15 modified blobs. And then it runs synchronization. I've run this test case 10 times on EGit with reverted commit 6c5e2ea0, the average synchronization time was 12908.6 milliseconds (max: 20320, min: 5342). Then I've done same thing with merged commit 6c5e2ea0 and got such results average: 12033.2, max: 18425, min: 2595. Finally I've done same measurement with merged change #2741 and got average: 10803.8, max: 16043, min: 2405 This experiment shows that commit 6c5e2ea0 didn't introduce regression in performance it even improve it. Change #2741 seams to add more performance improvements.
When I synchronize the first "Fetching" phase is very fast but then "Updating" is visibly slow. Enough time to read name of each file appearing, some stay for a couple of seconds. Having more than 6000 files in my slice of the repository it takes again ~30 min, compare with 30 sec I enjoyed before, see comment 21. I compared with another older installation and build 0.12.0.201103040937 is fast. The next one I installed - 0.12.0.201103151851 - is very slow. In the older one "Updating" is very fast, way faster than "Fetching".
I've done "real word testing" using linux kernel sources here are results: without 6c5e2ea0: 40-60s with 6c5e2ea0: 13min with change #2741 : 50-70s I think that we can mere change #2741 and work on performance improvements later
(In reply to comment #27) > I've done "real word testing" using linux kernel sources here are results: > without 6c5e2ea0: 40-60s > with 6c5e2ea0: 13min > with change #2741 : 50-70s > I think that we can mere change #2741 and work on performance improvements later Is it possible to commit the patch soon? I updated egit to the latest from Hudson today and the problem is still there.
(In reply to comment #28) > Is it possible to commit the patch soon? I updated egit to the latest from > Hudson today and the problem is still there. Change #2741 was merged as commit cd15f9dafaab88f737b450381538dbab3cf1636a and it should be available in upcoming nightly build.
Thanks Dariusz, it is fast again. One thing, the synchronize view got moody at me on the first try. I removed a space in the middle of line of code and tried it. It said "Synchronizing Git", then changed the wording a bit, then showed change set icon, then the icon disappeared and I got "Syncronizing Git" again, then the changed wording, then the icon, then it repeated the cycle again and again and again until I changed my file then it stabilized and all was ok. I couldn't reproduce this exact trick again but the second time it said "No changes found" which is also incorrect. Compare with File in Git Index shows me my change fine.
(In reply to comment #30) > It said "Synchronizing Git", then changed the wording a bit, then showed > change set icon, then the icon disappeared and I got "Syncronizing Git" again, To be clear, all this refers to Synchronize panel on the left where one would normally see the change set.
Synchronize view slowed down again. Althought not that bad as waiting for half an hour but it takes now 2 min to synchronize vs. 30 sec as of comment 21. Eclipse EGit (Incubation) 0.12.0.201104080413 Eclipse JGit (Incubation) 0.12.0.201104061816 Dariusz, is it possible to set a benchmark unit test to prevent regressions? Synchronize view seem to be prone to getting slow.
Synchronize view is still completely unusable for me in 1.1. I'm trying to Synchronize against a branch to pull in changes for upstream, and every time I ctrl-S in the compare editor the whole IDE locks up for 30s+ at a time. It doesn't matter if I restart the IDE or not. Synchronize just doesn't scale to CDT.
Created attachment 204057 [details] backtrace showing issue Steps to reproduce: - Synchronize one CDT branch with another - Change a file (like a version number in a MANIFEST.MF) - Save. Eclipse locks up for up to a minute. The cause seems to be that the: synchronize.GitSubscriberMergeContext$2.resourceChanged(GitSubscriberMergeContext.java:70) is very expensive. It gets called back on the resource change, and does whole loads of work. Note that the resource change handler holds the WS lock (Bug 249951). Meanwhile ctrl-S is synchronous in the IDE, so the whole IDE is blocked while the egit does its stuff. I noticed these pauses when the synchronize perspective wasn't even active(Bug 358898). I eventually got an OOM as well.
This is a blocker for me. It takes 9 minutes to synchronize my repository! It is a reasonably large project, but nothing that Subclipse couldn't handle (before we switched to Git). What's more, once the synchronization has completed, if I try to modify a file and save it, Eclipse simply freezes up. It's just not usable. It appears from this thread that there was a point at which is was fast (and usable)? Is that the case? And if so, could someone kindly point me to a good version of EGit which I can install?
(In reply to comment #35) > And if so, could someone kindly point me to a good > version of EGit which I can install? Well, which build are you using right now? 1.1? Did you try using the nightly? Also see bug 358898.
I was using 1.1. I've just installed the latest nightly instead and it's a massive improvement. That same synchronization took 35 seconds the first time and subsequent synchronizations take about 3 seconds. And the file saving issue has been fixed. Fantastic! FYI, this is the nightly version I am using now: Eclipse EGit 1.2.0.201112121913 Eclipse JGit 1.2.0.201112121111 I look forward to more improvements!
It seems I spoke too soon. The previous test was done on a subsection of our project. When I synchronize on the entire project it is back to the painful waiting game. I have not done timings, but I can say that it takes several minutes (which includes several minutes of fetching plus several minutes synchronizing). What's more, it seems to be fetching despite me having disabled the "Always launch fetch before synchronization" option. The above issues were observed both on the nightly version indicated in my previous comment and the following nightly version (which I am now on): Eclipse EGit 1.3.0.201201041914 org.eclipse.egit.feature.group Eclipse JGit 1.3.0.201201031918 org.eclipse.jgit.feature.group On a side note, I would appreciate some clarification on what the default "Synchronize Workspace" actually does. Is it the equivalent of Advanced -> Synchronize -> FETCH_HEAD?
There also seems to be a lot of time "Re-indexing repository" whenever a change is made to the project, which is equally frustrating. For example, I just chose to "Overwrite" a file from the synchronize view and it is taking what seems like forever (minutes) to re-index.
I pulled the latest nightly of EGit and I'm seeing what seems to be this bug. I have a local repository I created that is rather large (pulled in my companies codebase to do testing) and opening the synchronize view for it takes around 10-15 minutes. This is a pretty big detractor from even using EGit because I would consider the synchronize view to be part of the main workflow of handling pulling and pushing of changes within my company. Is there a timeline on when this will be fixed, a workaround, or some alternative workflow?
In reply to: comment 39 > There also seems to be a lot of time "Re-indexing repository" > whenever a change is made to the project FYI, I opened a separate bug for this (Bug 372265) because it is a problem in all views (not just synchronize).
(In reply to comment #38) > On a side note, I would appreciate some clarification on what the default > "Synchronize Workspace" actually does. Is it the equivalent of Advanced -> > Synchronize -> FETCH_HEAD? Yes, "Synchronize Workspace" is equivalent to Advanced -> Synchronize -> FETCH_HEAD (In reply to comment #40) > Ies there a timeline on when this will be fixed, a workaround, or some > alternative workflow? Used "presentation mode" has impact on synchronize view performance. "Workspace presentation model" was some time ago rewritten and now it is more then ten time faster then previous one. Some time ago I've pushed new implementation of "Git Commits Model" but until now it isn't fully merged in master branch. Never the less, "workspace presentation model" will be always faster then "Git Commits" since second commit traverse each commit in given range to collect specific changes (first one traverse only changes between two points). Please add information with presentation model do you use. This will tell me on with part of synchronization I should focus more.
[Batch change] Remove passed Target Milestones If anyone on CC list is going to fix/implement this, feel free to assign a new, post-1.3/2.0, target milestone.
Another portion of performance/memory usage optimisation was recently merged as commit df7527930d3ff1b4529e7716a27e33d2f3d6bde2
Another performance problem was fixed in bug 396209 for 2.3. Please try with EGit 2.3.1 and reopen if there are still problems.
