Community
Participate
Working Groups
Steps to reproduce: fresh workspace, disable validation, create Tomcat 5.5 runtime and server config, disable autopublish (otherwise bug 121603), import "war" attached to bug 119146, fix "war" project build path problem ;-) and add the project to the tomcat server. Create "test.txt" file in war/WebContent directory. Problem: UI completely frezzes for about a minute on my T42 with CPU 100%. Same freeze happens if when I edit/save test.txt file. WTP 1.0RC5, WinXP SP2, IBM JDK 1.4.2. I did not have much time to look at this problem tonight, but J2EEFlexProjDeployable#members invoked during ServerPublishInfo#hasDelta seems to be causing the delay. RC4 did not have this problem and I can't think of a workaround even from my server plugin, hence "Blocker"
I am experiencing this problem with RC5 also. With the target server as Tomcat (4.1 and 5.0) my PC hangs with javaw.exe taking 100% of CPU. This occurs after tomcat server has been started and any code is changed. The build process hangs on the Invoking Validation task. My environment is WTP 1.0RC5, Win XP SP2 and sun JDK 1.4.2_10. I would regard this as a show stopper as development is impossible with tomcat as the target server. Note I did not see the problem with OC4J as my target server.
Moving bug since this appears to be a performance regression in J2EEFlexProjDeployable.members()
Added to Hot List at the request of Tim deBoer and set priority to P2. Targetting to 1.0.1.
The problem is caused by O^2 algorithm in ...common.componentcore...VirtualContainer#members, so I guess there are more scenarios that are broken for large workspaces.
Looked some more. I am now 99% sure that the problem was triggered by the fix for bug 120562 which added IVirtualFolder#members() call to J2EEFlexProjDeployable#members(). IVirtualFolder#members has few (i.e. more than two) places with extremely inefficient nested iterations over list of files in a folder. Will try to speed it up.
Created attachment 32156 [details] suggested improvement for J2EEFlexProjDeployable Very expensive J2EEProjectUtilities.getSourceContainers(IProject) was called for every file in J2EEFlexProjDeployable during execution of J2EEFlexProjDeployable#members(). Fix is to cache java source folder paths for duration of J2EEFlexProjDeployable#members().
Created attachment 32157 [details] suggested improvement for ComponentDeployable Replaced nested loops with HashSet when adding ModuleResources to ModuleFolder
Created attachment 32158 [details] suggested improvements for VirtualContainer and VirtualFile Reworked VirtualContainer#members(int) to avoid use of nested loops for filtering of non-unique children. Note that new implementation may behave somewhat different for "fuzzily" matching components (i.e. component with /a/b/c/d runtimePath in /a virtual folder) as I did not completely understand intent of original code. Changed VirtualFile to cache underlying IFile.
Comment on attachment 32156 [details] suggested improvement for J2EEFlexProjDeployable Removing this patch - this caching is not needed with my new patch.
Created attachment 32160 [details] Performance improvements This patch in addition to patch https://bugs.eclipse.org/bugs/attachment.cgi?id=32157&action=edit Will greatly improve this scenario - these patches will be dropped to R101 stream soon after more testing
Created attachment 32162 [details] Revised patch for ComponentDeployable Original patch did not always set ModuleFolder members, which resulted in this exception later on !ENTRY org.eclipse.wst.server.core 4 0 2005-12-22 16:27:35.030 !MESSAGE Could not publish to the server. !STACK 0 java.lang.NullPointerException at java.util.Arrays$ArrayList.<init>(Arrays.java:2356) at java.util.Arrays.asList(Arrays.java:2342) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getExistingModuleResource(ComponentDeployable.java:187) at org.eclipse.wst.web.internal.deployables.ComponentDeployable.getMembers(ComponentDeployable.java:129) at org.eclipse.jst.j2ee.internal.deployables.J2EEFlexProjDeployable.members(J2EEFlexProjDeployable.java:135) at org.eclipse.wst.server.core.internal.ServerPublishInfo.fill(ServerPublishInfo.java:232) at org.eclipse.wst.server.core.internal.Server$5.visit(Server.java:805) at org.eclipse.wst.server.core.internal.Server.visitModule(Server.java:2173) at org.eclipse.wst.server.core.internal.Server.visitModule(Server.java:2184) at org.eclipse.wst.server.core.internal.Server.visit(Server.java:2157) at org.eclipse.wst.server.core.internal.Server.doPublish(Server.java:802) at org.eclipse.wst.server.core.internal.Server.publish(Server.java:788) at org.eclipse.wst.server.core.internal.PublishServerJob.run(PublishServerJob.java:145) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)
I ran few tests to measure time spent filling ServerPublishInfo in Server#doPublish (I have special tomcat plugin that deploys webapps by reference, nothing is ever copied anywhere so filling server publish info is the only operation that takes noticable time. I verified that these results generally apply to regular tomcat plugin as well but obviously I was getting much bigger numbers). (In reply to comment #9) > (From update of attachment 32156 [details] [edit]) > Removing this patch - this caching is not needed with my new patch. > This patch is important for my real workspace. Fill ServerPublishInfo in Server#doPublish takes ~15 seconds with this patch and ~45 seconds without it. (In reply to comment #10) > Created an attachment (id=32160) [edit] > Performance improvements With this patch, performance degrades grately with each publish. On my real workspace, it takes ~15 seconds to fill ServerPublishInfo for the first time, ~45 for the second time and I did not have enough patience to wait for the third execution to finish. This patch negates all performance gains for regular tomcat plugin -- second publish never completes. Having all this said, ~15 seconds to fill ServerPublishInfo is way too much. I will try to create sample project(s) that better reflect performance problems that I see in my workspace. I will also try to see how to fix these problems.
Created attachment 32256 [details] sample projects This set of projects has structure similar to my real workspace and shows same performance problems. Things to note: 1) util0/src/.../pkg0 has 5000 classes. 2) .../src and .../gen have same packages (this triggers performance degradation I mention in comment #12) 3) many dependent "utiliy jar" projects
Created attachment 32257 [details] Revised patch for ResourceTreeNode This is revised version of attachement #32160 (Performance improvements) which plugs ComponentResource leaks for projects that have multiple component resources with same runtimePath but different srcPath (multiple java source folders, for example). There are however other problems with original patch that I could not solve so far: Cached "transient" ComponentResources get persisted in .settings/.component. To reproduce: publish "war" to tomcat, change java build path of util12 project and remove "gen" java source folder. This has to be fixed 1.0.1 in my opinion. I saw some weird behaviour that could be explained by stale ComponentResource. How does the cache get refreshed when new folders get added/removed? In any case, this has simple workaround -- restart WTP. VirtualFile#getUnderlyingResource can return wrong IFile if two component resources have same runtimePath but different srcPath. One of my 0.7 migrated util jar projects has the following layout util/ src/ (java source folder, runtimePath: /) foo.properties test/ (NOT a java source folder, runtimePath: /) For this project, J2EEFlexProjDeployable#members() sometimes return /util/test/foo.properties. Note however, that this project layout is migration error and I do not know if there are legitimate layouts that will have the same problem.
Created attachment 32258 [details] Suggested improvements for ServerPublishInfo#save() ServerPublishInfo#save() was called several times during Server#doPublish (once for each published module). This was taking noticable time for publishing 12k files in 23 modules.
Is there any workaround for 1.0 users, other than downgrading to RC3? For now, WTP is rather unusable because of this bug. There should be a way to work around this bug before 1.01, which is more than a month away.
Hi Andreas, You have two choices here. One is to downgrade to RC3 as you suggest, the other is to apply Igor and Chuck's patches. I'd suggest RC3 is the least risky option, as we do know that it passes the existing unit test suite. The other option is to apply the patches. Igor sent me the compiled JARs, which I've tried on RC3 briefly, but did get resource deployment failures. I haven't tried those JARs on a 1.0 build yet. If you want the JARs, do email me, and I'll forward them on.
*** Bug 121603 has been marked as a duplicate of this bug. ***
Ok, I have investigated the patchs provided here and made sure the obsolete ones were obsoleted in the defect. From J2EE tooling, there are three patchs now attached to this defect. After applying these patchs and running through functional tests, the members method on the deploy still functions properly. Also, after running through profiling tools, the performance is exponentially better. Thanks to Igor for the help in driving this defect towards resolution. The J2EE parts of this defect are released and will be committed for the next committers 1.0.1 WTP driver tonight on 1/04/06. Tim, I will reassign to you for investigation into the ServerPublishInfo piece. Once Tim's work is complete, for clarity sake, I propose any additional nuances or problems be opened as a new defect.
Patch to ServerPublishInfo is technically unrelated to this performance regression and just a smart optimization to improve on the previous performance. Igor's patch is good and I've checked it into HEAD. Will tag for 1.0.1 shortly.
Released for next 1.0.1 build.
Thanks and Thanks, I have been very patient with the this slow performance of Eclipse where I have to wait 'Publishing to Tomcat' takes much longer than 0.7.1. My machine is AMD 64 3500+ with 1GB RAM, fedora core 4 x86_64. Every time I open Eclipse, it takes more than 1 minute to finish publishing. I almost change to Netbeans since I can't stand with this slow performance. But I still like Eclipse.
Hi Jerry, just to make sure - have you confirmed this issue is fixed with a recent M driver? http://download.eclipse.org/webtools/committers/drops/M-M200601102004-200601102004/ If you are still seeing issues let us know.
*** Bug 121484 has been marked as a duplicate of this bug. ***
Does this fix also speed up ear exports such as those targeting Weblogic 8.1?
verified
Closing.
New Gerrit change created: https://git.eclipse.org/r/107648
