Community
Participate
Working Groups
Build Identifier: I20110613-1736 We have a large workspace with 1500 plugins checked out in parallel and have recently switched to 3.7. Within our daily process we synchronize, update and compile this workspace. Given Eclipse 3.6 this process took about 7 minutes. With Eclipse 3.7 this process spiked to around 28 minutes with the exakt same workspace. We tried different setup with a completely new checked out workspace with the same results. It seems like eclipse triggers a second build besides the raised build times for one build. Are there any regression test for large workspaces ? Reproducible: Always
We haven't seen an increase in our tests. Is there a way you could give us some reproducible testcase?
No but I can provide you with some profiler information if you have some hotspots that would be interesting. Personally I think the dependency management may be causing it or the new build configuration system (which we do not actively use).
If you could give profile information that could help. You could also try to run with some debug options turned on to narrow down. Here are the instructions to turn on some builder options. Please run with 3.7 and 3.6 and give the times. Create a file called .options in the eclipse folder with the following lines in it. ###### org.eclipse.jdt.core/debug=true org.eclipse.jdt.core/debug/builder=true org.eclipse.jdt.core/debug/builder/stats= true ###### Then run eclipse as %eclipsec.exe -debug and give the console output.
(In reply to comment #3) Can you give some profile data to help us find the problem?
I profiled a reduced set of our workspace as full compile without any significant differences. However the problem we experience is happening when doing svn updates and subsequent compiles so I guess the incremental compiler may be the hot spot. Unfortunately for comparable profile results I have to setup identical workspaces at the same revision to do an update. As we are currenlty in an extensive QA phase I hadn't had the time to do so but I definetly will do so as soon I have a chance. If you're interested in the full compile profile result I can attach them. Another thing that has been reported by my colleages is that more subsequent updates cause the compile time to rise. One of my colleages erased the metadata directory and things went back to normal for the first compile. I haven't had the time to validate those "rumours" but for sure I can say that the compile times are higher in 3.7.
(In reply to comment #5) > I profiled a reduced set of our workspace as full compile without any > significant differences. However the problem we experience is happening when > doing svn updates and subsequent compiles so I guess the incremental compiler > may be the hot spot. Unfortunately for comparable profile results I have to > setup identical workspaces at the same revision to do an update. As we are > currenlty in an extensive QA phase I hadn't had the time to do so but I > definetly will do so as soon I have a chance. If you're interested in the full > compile profile result I can attach them. Sure, give me the profile result you have. I will see if that could give us any info. > > Another thing that has been reported by my colleages is that more subsequent > updates cause the compile time to rise. One of my colleages erased the metadata > directory and things went back to normal for the first compile. > I haven't had the time to validate those "rumours" but for sure I can say that > the compile times are higher in 3.7. Are you telling that a full build is taking lot of time or is it the incremental build that is taking lot of time. It will help if you could get some profiles for this too. Can you tell me your machine configurations?
Created attachment 203144 [details] ProfileData for 3.6 and 3.7 - no significant differences
Created attachment 203146 [details] 3.7 Profile Data without comparable reference to 3.6 The second attachment profile was taken a while ago with 3.7. I wanted to know where the incremental compiler burns its cpu cycles so no comparable reference to 3.6 exists.
I'm saying the svn-update plus incremental build is taking a lot of time and by looking at the UI it seems like the svn update/synchronization is not the one that is causing these times. The full build is equally fast to 3.6. Machine Configurations are (with slight variations depending on the developer): OS : Microsoft Windows 7 Professional Version 6.1.7600 Build 7600 Manufacturer Dell Inc. Model Precision WorkStation T5500 System Type x64-based PC Processor Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, 2395 MHz, 4 Cores(e), 8 logical(r) CPUs(en) (RAM) 12,0 GB RAID0
I've been chasing this down for months now failing to provide reproducible results because it seems to happen only on mass data. However I have come to a suspect. The compile times will rise significantly if the svn update is rather large. From profiling results I suspect that not the compile is the issue here but the DataTree that forwards incremental deltas. I profiled adding a dependency to a ~50 projects workspace in 3.6 and 3.7 which caused a 22s and 24s compile respectively. Probably the introduction of build configurations play a role here. The compile process slows down on the percent level the closer it gets to 100%. Any directions even patch suggestions are welcome.
We also have a large workspace with around 750 plugins checked out in parallel, and 500+ in the target platform, but I haven't noticed that 3.7(or SR1,SR2) is any slower than 3.6 for us. It is horrendously slow though, and seeing Eclipse attempt to build everything, and then start again immediately is somewhat frustrating. A full workspace build takes many minutes and it seems easily triggered by adding a new plugin dependency to any plugin. By any plugin, I mean any plugin: We attempted to debug what was going on by creating a workspace with 3 plugins and experimenting with dependency. If we added plugin 2 as a dependency to plugin 1, all 3 plugins were rebuilt. We repeated the test with 3 normal java plugins but adding project dependencies. Adding project 2 as a dependency of project 1 caused project 1 to rebuild but not project 3. This led us to believe the magic that computes plugin dependencies and the workspace build order (which seems to be different in plugin development than standard java development) is getting something wrong. While probably not related, this is extremely frustrating as it appears to trigger a complete workspace rebuild when anything is changed in a plugin manifest! When I next get an opportunity I will profile one of our long workspace builds with Yourkit and post any findings.
I can confirm the plugin dependency behavior as well. This may be the reason why svn updates take much more time than in previous releases because of changed dependencies.
Created attachment 228373 [details] Profile Data for archive verification A small part of the problem is that the JavaModelManager introduced a library check through validateLibraryContents back in 3.7. When adding or removing a dependency this takes some additional time as can be seen from the attached screenshot. I'd prefer to have a preference that disables the library check because in our case a great deal of libraries have to be checked. Another option would be to cache the valid archives as only the invalid archives are cached. What do you think ?
The library check issue is https://bugs.eclipse.org/bugs/show_bug.cgi?id=229042
One thing we expericend was a duplicate build that occured. Bug 354993 seems to be responsible for that.
Jesper, we could use some help here - if this area would interest you. Please take a look at the attached profiles. You can also generate your own profiles for certain workloads - My understanding is that yourkit is available for fee free use for eclipse development.
(In reply to comment #10) > I've been chasing this down for months now failing to provide reproducible > results because it seems to happen only on mass data. However I have come to > a suspect. The compile times will rise significantly if the svn update is > rather large. From profiling results I suspect that not the compile is the > issue here but the DataTree that forwards incremental deltas. Jens, just to isolate that part, how are you getting the Subversion updates: Subclipse, Subversive, or externally+Refresh?
Finally nailed the problem. It's rooted in having os dependent fragments in the target platform. When PDEState#resolveState(incremental / true) is called the OSGi resolver for bundles determines what needs to be updated. The ResolverImpl#addHostsFromFragmentConstraints adds all unresolved hosts from fragments without considering os properties. From there all dependent bundles will be resolved. For example *a lot* of plugins depend on org.eclipse.swt.win32.win32.x86 and thus org.eclipse.swt is being added - and its dependents. The org.eclipse.core.filesystem.[os].[arch] is a problem as well. Removing them from the platform (except the platform we compile againts) yields the expected bundles for update and the compile time drops again. However fragments should NOT be considered during bundle resolution if their platform properties cannot be fulfilled. Has this become and equinox rt issue then ?
(In reply to comment #18) > Has this become and equinox rt issue then ? Jesper, thanks for studying the analysis from comment#18 and suitably forwarding the bug. Even if this moves out of JDT/Core, we should study the performance characteristics of JDT/Core on some suitable workloads, but this can be taken up a side activity.
So how will this issue be processed ? This issue may be not equinox related as the resolver probably does its job as expected. I'm not that much into that one to tell whether is should be better parametrized from PDE or if the resolver contains the problem by always trying to reresolve unresolvable bundles. As the 4.3 release train is going to QA right now I'd be happy to have this bug fixed.
Jay, we need to study this to see if this should be routed elsewhere.
(In reply to comment #21) > Jay, we need to study this to see if this should be routed elsewhere. Since this is closely related to how Equinox resolves dependencies and/or how PDE handles those, so I'd recommend rerouting to somebody who knows more about that, at it would likely take me a a while to study that further. I'm not necessarily saying this isn't a JDT bug, but there's clearly an interaction.
Is there a particular person or does this bug gets its product and component reset by me?
I am moving the bug to PDE - feel free to move back or to other components if found appropriate. Please refer to comment #22
(In reply to comment #18) > Finally nailed the problem. > It's rooted in having os dependent fragments in the target platform. > When PDEState#resolveState(incremental / true) is called the OSGi resolver > for bundles determines what needs to be updated. The > ResolverImpl#addHostsFromFragmentConstraints adds all unresolved hosts from > fragments without considering os properties. From there all dependent > bundles will be resolved. For example *a lot* of plugins depend on > org.eclipse.swt.win32.win32.x86 and thus org.eclipse.swt is being added - > and its dependents. The org.eclipse.core.filesystem.[os].[arch] is a problem > as well. Removing them from the platform (except the platform we compile > againts) yields the expected bundles for update and the compile time drops > again. > > However fragments should NOT be considered during bundle resolution if their > platform properties cannot be fulfilled. > > Has this become and equinox rt issue then ? 1) Have a target platform with fragments for multiple platforms (i.e. delta pack) 2) PDE adds all of the fragments to an OSGi State (PDEState) 3) PDE attempts to resolve the state 4) The Equinox resolver is slow to resolve the fragments Is the resolver slow to resolve the fragments because it is repeating work? Or are you saying it attempts to resolve all of the unresolved fragments if an unrelated bundle is changed? cc;ing Tom for RT input.
I do not think this is a new issue since the logic has been this way since the fix for bug 173411 way back in the 3.3 release (as in Wassim was still involved!). But from comment 18 I don't understand two things: 1) I don't think hosts should be getting pulled in unless the fragment is unresolved AND the fragment has additional package or bundle requirements. 2) I thought PDE setup the state's platform properties in such a way that all platform specific bundles/fragments would be resolvable no matter what platform the tooling was running on.
Moving to PDE UI > 1) I don't think hosts should be getting pulled in unless the fragment is > unresolved AND the fragment has additional package or bundle requirements. Testing with a simple target platform (just core filesystem and some of the various fragments from the delta pack), confirms this. That method isn't doing any additional work that would explain the performance issues. Perhaps there is a problem where we are always resolving the fragments during an incremental resolve? (In reply to comment #26) > 2) I thought PDE setup the state's platform properties in such a way that > all platform specific bundles/fragments would be resolvable no matter what > platform the tooling was running on. We set the state environment properties based on what you have set in your target platform. This does not have to match what you are running on. This does not mean all the fragments are resolvable. After the state is done resolving it has resolver errors for the platform incorrect fragments.
(In reply to comment #27) > Perhaps there is a problem where we are always resolving the fragments > during an incremental resolve? Not positive yet, but it does look like this is happening. On a simple target I modify the requirements on a bundle that doesn't depend on the fragments. However, during the state incremental resolve, the refresh list is expanded from the modified bundle to include all of the unresolved fragments. This is because we are 1) in dev mode 2) have unresolved bundles (fragments) and 3) the unresolved bundles have at least one dependency (on their host). From org.eclipse.osgi.internal.module.ResolverImpl.addDevConstraints(BundleDescription[]) if (!developmentMode) return reRefresh; // we don't care about this unless we are in development mode // when in develoment mode we need to reRefresh hosts of unresolved fragments that add new constraints // and reRefresh and unresolved bundles that have dependents
There isn't time in 4.3 to follow up on this. In 4.4 we can take another look and see if Equinox is doing some extra work when you have unresolved fragments.
Is this being scheduled for 4.6 ?
Moving to 4.6 for investigation and better visibility of this bug.
Moving out of 4.6 since there are too many PDE defects for 4.6
We still see this issue. Is there any hope in sight?
Julian, can you have a look, if this problem still exists with your performance patches?
Created attachment 276799 [details] snippet to evaluate dev constraints The dev constraint issue from comment #28 is still present in 4.10, but we've since then fixed our target platform. IMO the primary issue is that there was no indication whatsoever what's causing the long builds or that the target platform was incomplete. The products ran fine, even with those unresolved bundles (I think the unresolved stuff was either an optional dependency or not used at all, but I can't remember exactly). We've built a view which invokes ResolverImpl#addDevConstraints reflectively to check the active target platform (snippet attached). Maybe something like that could be integrated in the target editor, but #addDevConstraints only operates on the active target platform, so it would not be live.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
Please reopen if you plan to work on this.
