Community
Participate
Working Groups
Build Identifier: After importing three large TS projects, about 90 threads were started which continued to stick around even after the import and build operations completed. The following is a snapshot BEFORE and AFTER the projects were imported: <b>BEFORE</b>: Tigerstripe 2 [Eclipse Application] org.eclipse.equinox.launcher.Main at localhost:61674 Thread [main] (Running) Daemon Thread [Thread-0] (Running) Daemon Thread [State Data Manager] (Running) Daemon Thread [Start Level Event Dispatcher] (Running) Daemon Thread [Framework Event Dispatcher] (Running) Daemon Thread [Bundle File Closer] (Running) Thread [Worker-0] (Running) Daemon Thread [Java indexing] (Running) Thread [Worker-1] (Running) Thread [Worker-2] (Running) Thread [Worker-4] (Running) Thread [xDE - Logging Thread] (Running) Thread [XDE JarCache - xde-jarcache.xml] (Running) Thread [PackageRepository Validation Thread] (Running) /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java (Jul 13, 2010 5:22:26 PM) <b>AFTER</b>: Tigerstripe 2 [Eclipse Application] org.eclipse.equinox.launcher.Main at localhost:61674 Thread [main] (Running) Daemon Thread [Thread-0] (Running) Daemon Thread [State Data Manager] (Running) Daemon Thread [Start Level Event Dispatcher] (Running) Daemon Thread [Framework Event Dispatcher] (Running) Daemon Thread [Bundle File Closer] (Running) Thread [Worker-0] (Running) Daemon Thread [Java indexing] (Running) Thread [Worker-1] (Running) Thread [Worker-2] (Running) Thread [Worker-4] (Running) Thread [xDE - Logging Thread] (Running) Thread [XDE JarCache - xde-jarcache.xml] (Running) Thread [PackageRepository Validation Thread] (Running) Thread [Worker-5] (Running) Thread [Worker-6] (Running) Thread [Worker-7] (Running) Thread [Worker-8] (Running) Thread [Worker-9] (Running) Thread [Worker-10] (Running) Daemon Thread [Thread-10] (Running) Daemon Thread [Timer-2] (Running) Thread [Worker-11] (Running) Thread [Worker-12] (Running) Thread [Worker-13] (Running) Thread [Worker-14] (Running) Thread [Worker-15] (Running) Thread [Worker-16] (Running) Thread [Worker-17] (Running) Thread [Worker-18] (Running) Thread [Worker-23] (Running) Thread [Worker-25] (Running) Thread [Worker-27] (Running) Thread [Worker-24] (Running) Thread [Worker-21] (Running) Thread [Worker-20] (Running) Thread [Worker-19] (Running) Thread [Worker-22] (Running) Thread [Worker-26] (Running) Thread [Worker-28] (Running) Thread [Worker-29] (Running) Thread [Worker-30] (Running) Thread [Worker-31] (Running) Thread [Worker-32] (Running) Thread [Worker-33] (Running) Thread [Worker-34] (Running) Thread [Worker-35] (Running) Thread [Worker-36] (Running) Thread [Worker-37] (Running) Thread [Worker-38] (Running) Thread [Worker-39] (Running) Thread [Worker-40] (Running) Thread [Worker-41] (Running) Thread [Worker-42] (Running) Thread [Worker-43] (Running) Thread [Worker-44] (Running) Thread [Worker-45] (Running) Thread [Worker-46] (Running) Thread [Worker-49] (Running) Thread [Worker-50] (Running) Thread [Worker-48] (Running) Thread [Worker-47] (Running) Thread [Worker-51] (Running) Thread [Worker-52] (Running) Thread [Worker-53] (Running) Thread [Worker-54] (Running) Thread [Worker-57] (Running) Thread [Worker-55] (Running) Thread [Worker-56] (Running) Thread [Worker-61] (Running) Thread [Worker-59] (Running) Thread [Worker-60] (Running) Thread [Worker-58] (Running) Thread [Worker-62] (Running) Thread [Worker-63] (Running) Thread [Worker-64] (Running) Thread [Worker-65] (Running) Thread [Worker-66] (Running) Thread [Worker-67] (Running) Thread [Worker-68] (Running) Thread [Worker-69] (Running) Thread [Worker-70] (Running) Thread [Worker-71] (Running) Thread [Worker-72] (Running) Thread [Worker-74] (Running) Thread [Worker-73] (Running) Thread [Worker-75] (Running) Thread [Worker-77] (Running) Thread [Worker-76] (Running) Thread [Worker-78] (Running) Thread [Worker-79] (Running) Thread [Worker-80] (Running) Thread [Worker-81] (Running) Thread [Worker-82] (Running) Thread [Worker-83] (Running) Thread [Worker-85] (Running) Thread [Worker-84] (Running) Thread [Worker-86] (Running) Thread [Worker-87] (Running) Thread [Worker-88] (Running) Thread [Worker-90] (Running) Thread [Worker-89] (Running) Thread [Worker-91] (Running) Thread [Worker-92] (Running) Thread [Worker-93] (Running) /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java (Jul 13, 2010 5:22:26 PM) Reproducible: Always Steps to Reproduce: 1. Start TS in debug mode 2. Notice the number of threads in Debug view 3. Import three or more large TS projects 4. Observe Debug view after projects have been imported
Yes I am also seeing large numbers of worker threads. I hadn't raised an issue as I wanted to see how large it got and if it caused any lockups or crashes. Navid, it might be worth running jstack and attaching the result.
Created attachment 174250 [details] Stack Trace (In reply to comment #1) > Yes I am also seeing large numbers of worker threads. > > I hadn't raised an issue as I wanted to see how large it got and if it caused > any lockups or crashes. > > Navid, it might be worth running jstack and attaching the result. What would be even more helpful would be to figure out who's starting these threads :-) I'll probably spend some time debugging this later. I'm attaching the convoluted stack trace of some of these threads displayed in Debug view.
This is probably going to to be really valuable.
I've nailed down the culprits to the following: TigerstripeWorkspaceNotifier.signalModelChange() TigerstripeWorkspaceNotifier.signalArtifactReourceAdded() TigerstripeWorkspaceNotifier.signalArtifactResourceChange() ProjectDiagramSynchronizer.initialize() The above methods spin off a MASSIVE number of Eclipse jobs. This doesn't give Eclipse a chance to clean up the worker threads (Worker threads are used to execute these jobs). There is a 60 second expiration for each Worker thread if no new jobs is added to the queue. However, TS spins off too many Eclipse jobs and this continuously feeds off the Workers and keeps them alive. TigerstripeWorkspaceNotifier is the workspace resource change listener that notifies Artifact managers and other listeners. There is no reason for spinning off this many threads when three large TS projects are imported. When I disabled the above listeners, import operation on the three projects took 10-15 seconds. With the listeners enabled, the exact same import operation took more than 2 minutes! There is definitely some performance improvements that needs to be made here. However, it should be done carefully so there is no loss of functionality.
(In reply to comment #4) > I've nailed down the culprits to the following: > TigerstripeWorkspaceNotifier.signalModelChange() > TigerstripeWorkspaceNotifier.signalArtifactReourceAdded() > TigerstripeWorkspaceNotifier.signalArtifactResourceChange() > ProjectDiagramSynchronizer.initialize() > > The above methods spin off a MASSIVE number of Eclipse jobs. This doesn't give > Eclipse a chance to clean up the worker threads (Worker threads are used to > execute these jobs). There is a 60 second expiration for each Worker thread if > no new jobs is added to the queue. However, TS spins off too many Eclipse jobs > and this continuously feeds off the Workers and keeps them alive. > > TigerstripeWorkspaceNotifier is the workspace resource change listener that > notifies Artifact managers and other listeners. There is no reason for > spinning off this many threads when three large TS projects are imported. > > When I disabled the above listeners, import operation on the three projects > took 10-15 seconds. With the listeners enabled, the exact same import > operation took more than 2 minutes! > > There is definitely some performance improvements that needs to be made here. > However, it should be done carefully so there is no loss of functionality. Based on bug scrub: - frequency of closed diagram update shouldn't be too low. Maybe every 1s? - Job "leak" needs to be fixed.
Created attachment 176289 [details] Patch Here's my shot at this. This patch significantly reduces the number of threads created for import (and other) operations. There is definitely a lot more room for improvement. However, the changes I'm thinking of are architectural changes to the resource listeners in Tigerstripe. They are a bit more involved. I'll file a separate enhancement to redesign this.
Thanks Navid, I suggest we test out your patch and then see if we need to take any further action on threads. The more involved changes may make sense as part of a wider core rewrite (if needed).
Submitted patch. Modified files: ProjectDiagramsSynchronizer TigerstripeWorkspaceNotifier WorkspaceListener WorkspaceHelper
