Community
Participate
Working Groups
Currently the Equinox launcher calls Bundle.start() sequentially. This missing the opportunity to run this activation in parallel, similar to Mickaels work for the JDT compiler. See Bug 531554. See https://www.eclipse.org/lists/equinox-dev/msg08901.html especially: ----- Activation of a bundle happens when the launcher calls Bundle.start(). You can certainly write a launcher that calls start() on many bundles concurrently from multiple threads. ----- I suggest we investigate if we could allow parallel activation. This could potentially result in a significant startup improvement. For example, here are the startup times of my bundles, running parts of this in parallel would be helpful: Lars Vogel <lars.vogel@vogella.com> 11:20 AM (4 hours ago) to General Hi, several committers and contributors asked about the tracing option in Eclipse to measure startup time on EclipseCon. As promised, I documented this and you find it here: http://www.vogella.com/tutorials/EclipsePerformance/article.html#example-tracing-the-startup-time-of-plug-ins Stuff we did to improve startup time in the past: - Move Activator code to OSGi DS services - Move blocking initialization code to Jobs - Remove useless code if it exists Here is the ranked output for my SDK + EGit + SonarLint + Asciidoctor toolilng +e4 spies + Codemining based on the shell script from the tutorial. The number at the beginning is the required time to start in milliseconds, e.g., jdt.ui takes a bit more than half a second. 793 org.eclipse.egit.ui_5.0.0.201806131550-r 590 org.eclipse.jdt.ui_3.16.0.v20181019-1929 393 org.sonarlint.eclipse.ui_3.6.0.201806071228 314 org.sonarlint.eclipse.core_3.6.0.201806071228 278 org.eclipse.jdt.core_3.16.0.v20181023-1015 147 org.eclipse.ui.trace_1.1.200.v20180829-0709 144 org.eclipse.core.runtime_3.15.100.v20180907-0807 133 org.eclipse.update.configurator_3.4.100.v20180821-1913 121 org.eclipse.egit.core_5.0.0.201806131550-r 97 org.eclipse.core.resources_3.13.200.v20181016-1140 95 org.eclipse.ui.workbench_3.112.100.v20181023-1210 79 org.eclipse.equinox.simpleconfigurator_1.3.100.v20180827-1122 70 org.eclipse.osgi_3.13.200.v20180906-2135 68 org.eclipse.pde.ui_3.10.500.v20181023-0807 56 org.eclipse.equinox.console_1.3.200.v20181008-1820 52 org.eclipse.pde.core_3.12.200.v20181012-1153 49 org.eclipse.equinox.p2.ui.sdk.scheduler_1.4.100.v20180827-1122 43 org.eclipse.emf.common_2.15.0.v20180723-1316 43 org.eclipse.core.jobs_3.10.200.v20180912-1356 40 org.eclipse.jsch.core_1.3.200.v20180827-1041 36 org.eclipse.core.net_1.3.300.v20180827-1041 33 org.eclipse.core.contenttype_3.7.100.v20180817-1401 32 org.apache.felix.scr_2.0.14.v20180822-1822 30 org.eclipse.jdt.launching_3.11.100.v20181019-1948 27 org.eclipse.equinox.registry_3.8.200.v20181008-1820 26 org.eclipse.equinox.app_1.4.0.v20181009-1752 25 org.eclipse.equinox.preferences_3.7.200.v20180827-1235 24 org.eclipse.emf.ecore_2.15.0.v20180722-1159 22 org.apache.felix.gogo.runtime_1.1.0.v20180713-1646 20 org.eclipse.core.expressions_3.6.200.v20180817-1401 16 org.eclipse.debug.core_3.13.100.v20180910-0519 13 org.eclipse.team.ui_3.8.200.v20180827-1338 13 org.eclipse.equinox.p2.metadata.repository_1.3.100.v20180822-1302 13 org.eclipse.equinox.common_3.10.200.v20181021-1645 11 org.eclipse.team.core_3.8.300.v20180827-1041 11 org.eclipse.equinox.p2.reconciler.dropins_1.3.0.v20180921-1132 10 org.apache.felix.gogo.shell_1.1.0.v20180713-1646 9 org.eclipse.equinox.event_1.5.0.v20181008-1938 9 org.eclipse.compare_3.7.400.v20181019-1444 8 org.eclipse.e4.ui.workbench_1.8.0.v20181022-1348 8 org.apache.felix.gogo.command_1.0.2.v20170914-1324 7 org.sonarlint.eclipse.jdt_3.6.0.201806071228 5 org.eclipse.ltk.core.refactoring_3.9.100.v20180828-0626 5 org.eclipse.equinox.p2.repository_2.4.200.v20181005-1131 4 org.showshortcuts_1.0.1 4 org.eclipse.ui.editors_3.11.300.v20180914-0606 4 org.eclipse.equinox.p2.ui.sdk_1.1.100.v20180827-1122 3 org.eclipse.ui.workbench.texteditor_3.11.200.v20181017-0917 3 org.eclipse.equinox.p2.core_2.5.100.v20180822-1532 2 org.eclipse.ui.ide_3.14.200.v20181017-1322 2 org.eclipse.ui.cheatsheets_3.5.300.v20180823-1505 2 org.eclipse.ui_3.111.0.v20181017-1216 2 org.eclipse.tm.terminal.view.ui_4.5.0.201808290101 2 org.eclipse.ltk.ui.refactoring_3.9.200.v20180828-0626 2 org.eclipse.help_3.8.200.v20180821-0700 2 org.eclipse.equinox.p2.metadata_2.4.100.v20180822-1302 2 org.eclipse.e4.ui.workbench.swt_0.14.400.v20181022-1348 1 org.sonarlint.eclipse.m2e_3.6.0.201806071228 1 org.sonarlint.eclipse.cdt_3.6.0.201806071228 1 org.eclipse.wst.sse.ui_1.5.0.v201807252207 1 org.eclipse.ui.views.log_1.2.300.v20180827-1331 1 org.eclipse.ui.net_1.3.300.v20180827-1041 1 org.eclipse.ui.console_3.8.300.v20181019-1609 1 org.eclipse.pde.launching_3.7.300.v20180926-1404 1 org.eclipse.jdt.core.manipulation_1.11.0.v20181019-1929 1 org.eclipse.equinox.security_1.2.600.v20181022-1735 1 org.eclipse.equinox.p2.ui_2.5.300.v20180921-1019 1 org.eclipse.equinox.p2.operations_2.5.200.v20180928-1359 1 org.eclipse.equinox.p2.garbagecollector_1.1.100.v20180822-1302 1 org.eclipse.equinox.p2.engine_2.6.100.v20180822-1302 1 org.eclipse.equinox.p2.directorywatcher_1.2.100.v20180822-1302 1 org.eclipse.emf.ecore.xmi_2.15.0.v20180706-1146 1 org.eclipse.e4.tools.preference.spy_0.1.0.v20171212-1226 1 org.eclipse.core.filesystem_1.7.200.v20180828-0158 0 org.eclipse.ui.monitoring_1.1.300.v20180828-1350 0 org.eclipse.tips.ide_0.1.200.v20180821-0700 0 org.eclipse.jdt.codemining_1.0.0.201808270308 0 org.eclipse.egit.codemining_1.0.0.201808270308 0 org.eclipse.e4.ui.css.swt_0.13.300.v20180920-1522 0 org.eclipse.e4.tools.services_4.8.200.v20181022-1512 0 org.eclipse.e4.tools.bundle.spy_0.18.0.v20170601-1400 0 org.eclipse.core.filebuffers_3.6.300.v20180822-1339 0 org.eclipse.compare.core_3.6.300.v20180827-1041
This is not a launcher issue. I think it could be a framework issue (if you are suggesting it be done as part of normal start-level increments for bundles). I'm not convinced you will see much improvements and I think you may find it creates some potential for deadlocks on startup because of the way bundles in Eclipse are lazily activated in a chain sequentially. Today it is single threaded which reduces the potential for out of order locks. I think you will expose some dirty laundry in many bundles where they have no issues with single threaded but will deadlock when done in parallel.
The Java-Doc for the BundleActivator notes that "The Framework must not concurrently call a BundleActivator object. " https://osgi.org/javadoc/osgi.core/7.0.0/org/osgi/framework/BundleActivator.html Beside this, in my opinion it is not the best strategy to solve problems just by making activation parallel. Instead, I would recommend investing the time, to avoid long running operations in BundleActivators. Even if it seems not realistic, it is the right way. Working around it will not reduce the technical debt, but increase it. In addition to that, even if it would be allowed, to start bundles in parallel, it would also raise the grade of complexity instead of recuding it.
(In reply to Mark Hoffmann from comment #2) > The Java-Doc for the BundleActivator notes that "The Framework must not > concurrently call a BundleActivator object. " > > https://osgi.org/javadoc/osgi.core/7.0.0/org/osgi/framework/BundleActivator.html That should be OK, as we would call different activators in parallel not accessing the same. > Beside this, in my opinion it is not the best strategy to solve problems > just by making activation parallel. Instead, I would recommend investing the > time, to avoid long running operations in BundleActivators. These days we have multiple cores so using them is a good pattern. As the bug title says, we should investigate this, see if we break something and how much faster the startup gets. Afterwards, we can decide if we want to offer this as an option during startup or not.
(In reply to Lars Vogel from comment #3) > > These days we have multiple cores so using them is a good pattern. I didn't mean using parallelism in general is not a good pattern. Using long running operations in the BundleActivator is obviously not a good pattern: https://help.eclipse.org/photon/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fruntime_model_bundles.htm This document recommends e.g. using operations in the start method, that can be performed quickly. > As the bug title says, we should investigate this, see if we break something > and how much faster the startup gets. Afterwards, we can decide if we want > to offer this as an option during startup or not. I think this is not the best way to solve a problem, that already has an obvious solution. It may be a quick solution to multi-thread the start of activators, but the costs are, like Thomas mentioned: - increased probability for locks - increased complexity in the launcher, to handle potential locks It feels like a brute force way, to increase the startup time, without solving the real issues with long running activators. Putting stuff into to services, that then can be injected into components, can be considered as an alternative, I think.
Created attachment 276392 [details] Recommendation improvents for activator I attached a quick rework of the JDT UI activator using OSGi Promises as well an additional component. This was done without a testing, but should do the same. The launch time improvements were essential in my tests. Maybe this helps for your investigation
(In reply to Mark Hoffmann from comment #5) > Created attachment 276392 [details] > Recommendation improvents for activator > > I attached a quick rework of the JDT UI activator using OSGi Promises as > well an additional component. > > This was done without a testing, but should do the same. The launch time > improvements were essential in my tests. > > Maybe this helps for your investigation Mark, can you please create Gerrit patch?
New Gerrit change created: https://git.eclipse.org/r/131854
New Gerrit change created: https://git.eclipse.org/r/131872
I have been taking a look at changes to the framework start-level implementation to start bundles with the same start-level in parallel. So far I have not found much measurable startup improvements for Eclipse Platform startup. But I have found significant improvements for scenarios that involve large number of bundles with many DS components. In some scenarios I see a close to a 40% improvement with my 4 core i7 mac book using 8 threads to start the bundles. I plan to add an option to enable this so we can experiment with using parallel bundle start. By default the option will be disabled.
(In reply to Thomas Watson from comment #9) > I plan to add an option to enable this so we can experiment with using > parallel bundle start. By default the option will be disabled. Thanks Tom, great news.
New Gerrit change created: https://git.eclipse.org/r/140322
Gerrit change https://git.eclipse.org/r/140322 was merged to [master]. Commit: http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=2e8fc7a7f648e3ad325e55863aca098c8e108db6
I'm closing this as fixed as I have merged the initial work to allow parallel bundle activation. I may do some follow up work to adjust the number of threads used by default when the value of "0" is used for the config option. Right now a "0" value will cause the thread count to be equal to java.lang.Runtime.availableProcessors(). In some cases I have found that increasing that to 2 x java.lang.Runtime.availableProcessors() gives a bit more performance boost. That could be important when running in a docker environment when you typically are not given all the processors. I think it is common only to get 2. Here is the commit message with the options that can be enabled to try it out: By default the framework start-level implementation activates all bundles that have the same start-level in sequential order using the bundle ID (install order) to determine the order the bundles are started from a single thread. A new configuration option is added equinox.start.level.thread.count that is set to 1 by default. This will cause the framework to continue with the behavior of activating all bundles with a single thread. If equinox.start.level.thread.count is set to 0 then the framework will use an executor with the number of threads == to java.lang.Runtime.availableProcessors() to activate the bundles in parallel. This means that the order the bundles with the same start-level are activated becomes random. The equinox.start.level.thread.count config setting can also be set to a value > 1 to give an explicit value for the number of threads to use (instead of using java.lang.Runtime.availableProcessors() to determine the number)
Can you please add a note in the N&N?
Thanks, Thomas for the development. I did run several starts of a runtime Eclipse but the deviation in startup time seems very high. I measure with the same setting between 6 and ~9 seconds. Can you recommend a reasonable way to measure startup performance with this setting for the IDE?
New Gerrit change created: https://git.eclipse.org/r/140824
(In reply to Lars Vogel from comment #15) > Thanks, Thomas for the development. I did run several starts of a runtime > Eclipse but the deviation in startup time seems very high. I measure with > the same setting between 6 and ~9 seconds. > > Can you recommend a reasonable way to measure startup performance with this > setting for the IDE? I don't see the fluctuations like you do. I usually just edit the config.ini to set: equinox.start.level.thread.count=-1 Then I start with the native launcher using the "-debug -data testworkspace" options This prints out Starting application: 910 Application started in : 6170ms You should know that only that first number from "Starting application" can really be influenced by the parallel bundle activation from the start-level launch. By this time all the bundles have been activated by the framework. The rest of the activation that occurs in a typical Eclipse launch happens in response to a class load with lazy activation. This is one reason I think you will find very little gain in parallel bundle activation at this level, because most of Eclipse bundles are lazy activated with a trigger class load. That activation then happens on the thread that is loading the class.
Furthermore, I really only think this parallel bundle activation from the framework launch helps in cases where many bundles are installed that are eagerly activated.
Gerrit change https://git.eclipse.org/r/140824 was merged to [master]. Commit: http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=92e8d29b20e01924ed449c7e5c0f1bb7b96a151b
New Gerrit change created: https://git.eclipse.org/r/140831
Gerrit change https://git.eclipse.org/r/140831 was merged to [master]. Commit: http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=ff70c66be91389ffb10151276725b21683f5fe64
New Gerrit change created: https://git.eclipse.org/r/140837
Gerrit change https://git.eclipse.org/r/140837 was merged to [master]. Commit: http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=1671181d2493ef3161a7d2c6838f10e78905b53e
Thomas, can you post your eclipse.ini file? I might doing something silly in my setup, as I cannot see any startup differences.
(In reply to Lars Vogel from comment #24) > Thomas, can you post your eclipse.ini file? I might doing something silly in > my setup, as I cannot see any startup differences. See my comment 17. Because of the use of lazy activation with class loading triggers, I see very little benefit of parallel activation for Eclipse startup. It helps for other usages of OSGi where there are lots of eager start bundles with many service components. This is the more traditional scenario for OSGi. Again, I simply add the following to the end of my config.ini equinox.start.level.thread.count=-1 It does not surprise me that you see no difference because that is what I see also.
Thomas, please add to N&N for 4.12
New Gerrit change created: https://git.eclipse.org/r/142593
(In reply to Eclipse Genie from comment #27) > New Gerrit change created: https://git.eclipse.org/r/142593 Lars, I have a gerrit review here to add it. Not sure why I cannot submit the change from gerrit.
Gerrit change https://git.eclipse.org/r/142593 was merged to [master]. Commit: http://git.eclipse.org/c/www.eclipse.org/eclipse/news.git/commit/?id=919c18f830e4639631443056666b0f2d13724671
(In reply to Thomas Watson from comment #28) > (In reply to Eclipse Genie from comment #27) > > New Gerrit change created: https://git.eclipse.org/r/142593 > > Not sure why I cannot submit the change from gerrit. The verified +1 was missing. I've merged it.
I see random full builds when switching to a new build. It looks like a timing issue. Could this be caused by this bug fix?
(In reply to Dani Megert from comment #31) > I see random full builds when switching to a new build. It looks like a > timing issue. Could this be caused by this bug fix? I don't see how because this option is disabled by default. How long have you seen this?
(In reply to Thomas Watson from comment #32) > (In reply to Dani Megert from comment #31) > > I see random full builds when switching to a new build. It looks like a > > timing issue. Could this be caused by this bug fix? > > I don't see how because this option is disabled by default. How long have > you seen this? Thanks Tom. I've figured out the issue. I imported an e4 test bundle that was causing this (triggering bug 547633).
