Community
Participate
Working Groups
The MD5 computation seems to take a big chunk of time during startup when there are many bundles in the dropins folder. We should consider omitting computation of the MD5 property in this case since it isn't really needed for artifacts generated based on the dropins folder.
I will investigate, and if it provides a big improvement we could consider it for 3.6.1.
Created attachment 170801 [details] Profiler output from running reconciler app on large product after installing a feature
Created attachment 171104 [details] Fix v01 This patch introduces a publisher option to avoid MD5 computation. This option is then used in the RepositoryListener (reconciler). All other callers of the publisher will get MD5 hashes by default. This patch is very conservative in avoiding breaking any provisional API. If I was doing this in 3.7 I would clean this up and remove the old methods. In particular, there are several places where IPublisherInfo is not passed through where it is needed. If we ever promote this to real API we'll want the method signatures to pass IPublisherInfo around as much as possible to allow for these kinds of options in the future.
Pascal, what do you think of this. I can't think of any scenarios where computing MD5 hashes in the reconciler is useful. From the profiler output we can see this has a big impact on performance which cancels most of our reconciler performance improvements since 3.4.2 (which didn't compute MD5 hash).
This makes a huge difference to startup/reconcile time! Here is the scenario I tested: 1) Unzip Eclipse SDK 3.6 RC4 (I20100603-1500) 2) Startup once, shutdown. 3) Unzip the Helios modeling EPP package into dropins (1000 plugins, 232 features) 4) Startup. Here is the result both with and without the patch: 3.6 RC4 Before patch: Starting application: 130625 Application Started: 155484 3.6 RC4 after patch: Starting application: 29329 Application Started: 36282 Total startup time is 4x faster!
Created attachment 171131 [details] Fix v02 This is the fix I was running with. The only difference from the previous one is that it also omits MD5 computation when publishing features.
Created attachment 171135 [details] YourKit hotspot list before fix
Created attachment 171136 [details] YourKit hotspot list after fix I didn't believe those numbers, so I ran again before with and without the patch with a profile attached. I got similar results, and the two attachments show the hotspots (methods taking the most time) while running the scenario. 85% of the startup was taken by MD5 hash computation before the patch, which obviously disappears after the patch.
(In reply to comment #5) This is great news!
Removing the MD5 computation is fine by me. But I have an even better scenario: download RC4, use the p2 UI to install everything, restart. No time taken!
Fix released for 3.6.1 and 3.7.
