Bugzilla – Full Text Bug Listing
| Summary: | Publishing to Tomcat takes much longer on Helios than on Galileo | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [WebTools] WTP ServerTools | Reporter: | Jason Morawski <rpgdude1> | ||||||||||||||
| Component: | jst.server | Assignee: | Larry Isaacs <larryisaacs> | ||||||||||||||
| Status: | CLOSED DUPLICATE | QA Contact: | Angel Vera <arvera> | ||||||||||||||
| Severity: | normal | ||||||||||||||||
| Priority: | P3 | CC: | alexandru.ionita, antoniosanct, arvera, bowman.michael, christopher.klewes, fleury.jerome, franzroth, hakon.lugert, jal, jasonpet, julialopez, lostale, marco.trevisan, mauromol, perseusli, remy.suen, rhanneken, stryker, tapani.jalonen, thatnitind, thomas | ||||||||||||||
| Version: | unspecified | ||||||||||||||||
| Target Milestone: | --- | ||||||||||||||||
| Hardware: | PC | ||||||||||||||||
| OS: | All | ||||||||||||||||
| Whiteboard: | |||||||||||||||||
| Attachments: |
|
||||||||||||||||
|
Description
Jason Morawski
I have the same problem, but only in Windows 32bit. Add and remove projects at Servers tag runs about thirty seconds to open the window, and thirty seconds more to accept and start publishing the selected projects. I tried to reproduce this in Ubuntu 64 bits but the response time is minor than Windows 32bit, although it seems that it is slower than before (with Galileo). I have the same problem on Mac OS X 64 bits. Publishing to Tomcat is very very long, i'm loosing so much time so I'm going back to Eclipse Galileo. I just downloaded Helios, and tried this scenario but I didn't see any performance degradation, in a windows 32 bit. Jason, were you using 32 or 64 bit? Nothing of any consequence has been changed within the Tomcat plug-ins with respect to publishing (excluding the "Serve Modules Without Publishing" option), so the source of the behavior is likely coming from something the Tomcat plug-ins are using. In any of these cases, is this behavior happening in a new workspace created with WTP 3.2 and the workspace contains only one or two project and only one or two servers? For any who this is not true, could you create a new workspace, import a "problem" project, copying it into the workspace, create one Tomcat server and see if the behavior continues. (In reply to comment #3) > I just downloaded Helios, and tried this scenario but I didn't see any > performance degradation, in a windows 32 bit. Jason, were you using 32 or 64 > bit? I'm running Debian squeeze amd64. Perhaps we might also need to know what JDK you are using, I am trying to get a hold of a 64 bit platform to see if I can test there, but I can only find 64 windows. (In reply to comment #4) > Nothing of any consequence has been changed within the Tomcat plug-ins with > respect to publishing (excluding the "Serve Modules Without Publishing" > option), so the source of the behavior is likely coming from something the > Tomcat plug-ins are using. In any of these cases, is this behavior happening > in a new workspace created with WTP 3.2 and the workspace contains only one or > two project and only one or two servers? For any who this is not true, could > you create a new workspace, import a "problem" project, copying it into the > workspace, create one Tomcat server and see if the behavior continues. I tried using a fresh workspace and I still experienced the same behavior. (In reply to comment #6) > Perhaps we might also need to know what JDK you are using, I am trying to get a > hold of a 64 bit platform to see if I can test there, but I can only find 64 > windows. I'm using OpenJDK 6b18-1.8 built for amd64 by Debian Can you setup the following .options and rerun the scenario: #Master Tracing Options #Fri Jun 25 11:54:45 EDT 2010 org.eclipse.wst.server.core/performance=true org.eclipse.wst.server.ui/performance=true org.eclipse.wst.server.core/debug=true org.eclipse.jst.server.core/publishing=true org.eclipse.wst.server.core/resources=true org.eclipse.wst.server.core/listeners=true org.eclipse.wst.server.core/extension_point=true org.eclipse.wst.server.ui/extension_point=true org.eclipse.jst.server.tomcat.core/debug=true org.eclipse.jst.server.core/debug=true org.eclipse.wst.server.ui/debug=true Created attachment 172785 [details]
Debug log of initial publish to Tomcat
First I'll mention that I'm not sure how well the progress bar progresses during publishing, so I wouldn't worry about how long it may sit on 0%. That's an area of implementation that probably could use a thorough review. I would only worry about the total time. Also, saying that it takes longer to publish the first time (i.e. all resources have to be copied) in Helios than Galileo is a different issue from the project already being added to the server and published and a changed file takes longer to publish (i.e. only the changed file should publish) in Helios than Galileo. Jason, I mention this because your original report definitely refers to the later, but your use of "initial" makes me wonder if the log doesn't apply to the first. Was the log of an "original" publish of a project to the server or is the log of a first publish of a project that had been added to the server and published in a prior session? In any event, the log appears to show that js.jar is getting published 4 times and shrinksafe.jar is published 2 times, which is rather odd. This could confuse the "delta" publishing such that these jars are always written multiple times for each publish. Please attach the ".classpath" and ".settings/org.eclipse.wst.common.component" files from this project so we can see what might be amiss in those files. If you have both Helios and Galileo versions, attaching both would be great. Created attachment 172803 [details]
Debug log of republish to Tomcat
The initial publish to Tomcat appeared to have the same behavior as the auto-republish so I wasn't sure if this data was needed. Attaching updated log with republish behavior.
Created attachment 172804 [details]
Classpath file
Created attachment 172805 [details]
WST component file
(In reply to comment #11) > First I'll mention that I'm not sure how well the progress bar progresses > during publishing, so I wouldn't worry about how long it may sit on 0%. That's > an area of implementation that probably could use a thorough review. I would > only worry about the total time. > > Also, saying that it takes longer to publish the first time (i.e. all resources > have to be copied) in Helios than Galileo is a different issue from the project > already being added to the server and published and a changed file takes longer > to publish (i.e. only the changed file should publish) in Helios than Galileo. > > Jason, I mention this because your original report definitely refers to the > later, but your use of "initial" makes me wonder if the log doesn't apply to > the first. Was the log of an "original" publish of a project to the server or > is the log of a first publish of a project that had been added to the server > and published in a prior session? > > In any event, the log appears to show that js.jar is getting published 4 times > and shrinksafe.jar is published 2 times, which is rather odd. This could > confuse the "delta" publishing such that these jars are always written multiple > times for each publish. Please attach the ".classpath" and > ".settings/org.eclipse.wst.common.component" files from this project so we can > see what might be amiss in those files. If you have both Helios and Galileo > versions, attaching both would be great. In the Servers view under my module shows js.jar listed 4 times as a J2EE Application Client module and shrinksafe.jar listed twice along with some other jars. I am not even using these jars as J2EE Application Client modules so I dont understand why they even show up. Is there some way to remove them? Maybe this can solve my problem? The story of the J2EE Application Client behavior is that new in Helios, any jar that contains a Main-Class attribute in its MANIFEST.MF gets special treatment as a child module even when it is in a dynamic web project. The Tomcat support still just copies the jar during publishing, just like other jars. The "special treatment" will apply to the jar regardless of where it is found in the project. You see js.jar listed four time because you have four copies under your "war" folder (i.e. the web content folder for your project): WEB-INF/lib/js.jar WEB-INF/platform/plugins/org.mozilla.rhino_1.7.1.v20090521/lib/js.jar js/util/buildscripts/webbuild/server/lib/js.jar js/util/shrinksafe/js.jar It's not clear if you intend for all of these, plus potentially a number of other files, to be part of the web content portion of your web application. These will certainly add time to the "original" publish, but "delta" publishing should not be re-copying these jars on subsequent publishing. As a result, I'm not certain this would be the cause of the longer times. One thing I would try first is to modify the "Default output folder" for this project. It is currently set to "war/WEB-INF/classes", which puts it within what you have set as a web content folder, i.e. your "war" folder. This triggers extra handling in WTP to try to keep Java build artifacts distinguished from the other content resources. I've seen this extra handling cause problems in certain cases, and might be contributing in this case. Changing the "Default output folder" to the typical "build/classes" would eliminate this as a possible cause. If this doesn't improve the behavior, then my only remaining guess is that "delta" publishing is not doing its job and it re-copying files unnecessarily. Unfortunately, the logging is not showing whether or not this is occurring. I'll need to look into this a little deeper. (In reply to comment #16) > The story of the J2EE Application Client behavior is that new in Helios, any > jar that contains a Main-Class attribute in its MANIFEST.MF gets special > treatment as a child module even when it is in a dynamic web project. The > Tomcat support still just copies the jar during publishing, just like other > jars. The "special treatment" will apply to the jar regardless of where it is > found in the project. You see js.jar listed four time because you have four > copies under your "war" folder (i.e. the web content folder for your project): > > WEB-INF/lib/js.jar > WEB-INF/platform/plugins/org.mozilla.rhino_1.7.1.v20090521/lib/js.jar > js/util/buildscripts/webbuild/server/lib/js.jar > js/util/shrinksafe/js.jar > > It's not clear if you intend for all of these, plus potentially a number of > other files, to be part of the web content portion of your web application. > These will certainly add time to the "original" publish, but "delta" publishing > should not be re-copying these jars on subsequent publishing. As a result, I'm > not certain this would be the cause of the longer times. > > One thing I would try first is to modify the "Default output folder" for this > project. It is currently set to "war/WEB-INF/classes", which puts it within > what you have set as a web content folder, i.e. your "war" folder. This > triggers extra handling in WTP to try to keep Java build artifacts > distinguished from the other content resources. I've seen this extra handling > cause problems in certain cases, and might be contributing in this case. > Changing the "Default output folder" to the typical "build/classes" would > eliminate this as a possible cause. If this doesn't improve the behavior, then > my only remaining guess is that "delta" publishing is not doing its job and it > re-copying files unnecessarily. Unfortunately, the logging is not showing > whether or not this is occurring. I'll need to look into this a little deeper. I tried changing the "Default output folder" to "build/classes" and I still experience the same behavior. I would recommend keeping the "Default output folder" set to "build/classes" as it may help avoid other issues in the future. Upon deeper inspection of the log file, I think I see what is causing the slowdown. In Galileo, only dependent projects appeared as child modules and the destination of all child modules was hard coded to "WEB-INF/lib". If someone manually modified their "org.eclipse.wst.common.component" file to deploy the dependent project to a different location in the webapp, it would be ignored. While not technically correct, this didn't create much of a problem. In Helios, that has changed. Jars can appear as child modules and the Deployment Assembly page makes it easy to set the destination to something other than "WEB-INF/lib", which may be useful for EAR projects, but perhaps not all that useful for web projects. In any event, the hard coding of "WEB-INF/lib" was replaced with a call to IWebModule.getURI(IModule). It turns out this is a rather expensive call. Its attempt at caching isn't helpful in the context of server publishing and the call appears to rebuild the "members" tree of the entire project every time it's called to get the destination for a child module. I'll have to see if a more efficient method of obtaining that destination can be found. In the meantime, the publishing time could be reduced a good bit by organizing your project so that only web content meant to be in the webapp is included. For example, I would assume that the "war/js/util/buildscripts/webbuild/server/lib" folder doesn't contain anything that needs to be in the web content of your webapp. Likewise for "WEB-INF/platform/plugins/org.mozilla.rhino_1.7.1.v20090521/lib". Assuming the jars found under the "war" folder, but not under "war/WEB-INF/lib", are not needed, ten of the thirteen child modules would be eliminated. This would make a noticeable reduction in the publishing time as it appears that for this project, it's using roughly 3 seconds per child module. Also, getting unneeded web content out of the way, the getURI() call could take less time for the child modules that remain. Created attachment 172958 [details]
Catch exception unpublishing a project
Marking -debug .options, the program catches the exception attached. Occurs six times, one by each jar that is annotated like module.
If the publish process repeats many times, then the waiting time becomes a very tedious problem.
I'll try to reorganize the modules as explained in last comment.
Thanks a lot! Regards!
(In reply to comment #18) Larry your comments make me think of a problem that I was talking to Jason Peterson. This looks related to bug 304673 and Bug 316032. (In reply to comment #18) > I would recommend keeping the "Default output folder" set to "build/classes" as > it may help avoid other issues in the future. > > Upon deeper inspection of the log file, I think I see what is causing the > slowdown. In Galileo, only dependent projects appeared as child modules and > the destination of all child modules was hard coded to "WEB-INF/lib". If > someone manually modified their "org.eclipse.wst.common.component" file to > deploy the dependent project to a different location in the webapp, it would be > ignored. While not technically correct, this didn't create much of a problem. > In Helios, that has changed. Jars can appear as child modules and the > Deployment Assembly page makes it easy to set the destination to something > other than "WEB-INF/lib", which may be useful for EAR projects, but perhaps not > all that useful for web projects. In any event, the hard coding of > "WEB-INF/lib" was replaced with a call to IWebModule.getURI(IModule). > > It turns out this is a rather expensive call. Its attempt at caching isn't > helpful in the context of server publishing and the call appears to rebuild the > "members" tree of the entire project every time it's called to get the > destination for a child module. I'll have to see if a more efficient method of > obtaining that destination can be found. > > In the meantime, the publishing time could be reduced a good bit by organizing > your project so that only web content meant to be in the webapp is included. > For example, I would assume that the > "war/js/util/buildscripts/webbuild/server/lib" folder doesn't contain anything > that needs to be in the web content of your webapp. Likewise for > "WEB-INF/platform/plugins/org.mozilla.rhino_1.7.1.v20090521/lib". Assuming the > jars found under the "war" folder, but not under "war/WEB-INF/lib", are not > needed, ten of the thirteen child modules would be eliminated. This would make > a noticeable reduction in the publishing time as it appears that for this > project, it's using roughly 3 seconds per child module. Also, getting unneeded > web content out of the way, the getURI() call could take less time for the > child modules that remain. I really don't think this is an issue with my project structure. This looks like an issue with the publishing algorithm. I did not have this problem in Galileo. I will revert to Galileo until this problem is resolved. Same problem with Windows 7 and Eclipse Helios 32 bits The same behavior for me on a Linux 32 bit box with both OpenJDK 1.6.0_17 ans Sun JVM 1.6.0_20. The question to others seeing this issue is how many jars or projects appear as child modules under the dynamic web project on the server? (In reply to comment #25) > The question to others seeing this issue is how many jars or projects appear as > child modules under the dynamic web project on the server? 3 - freemarker-2.3.15, hsqldb-1.8.1.2, and javassist-3.9.0.GA.jar (In reply to comment #26) > (In reply to comment #25) > > The question to others seeing this issue is how many jars or projects appear as > > child modules under the dynamic web project on the server? > > 3 - freemarker-2.3.15, hsqldb-1.8.1.2, and javassist-3.9.0.GA.jar To further clarify, all of those are jars and are part of a simple struts2/hibernate web application. I'm seeing this behavior with this as the only project being synchronized with my tomcat 6 server. (In reply to comment #25) > The question to others seeing this issue is how many jars or projects appear as > child modules under the dynamic web project on the server? 11 jars. For what it's worth, it's not just publishing that's slow for me. Simply starting or stopping the web app also takes a very long time. I started having these problems only after upgrading to Helios. I'm running Windows XP Professional SP3, Eclipse Helios 32-bit, Sun JDK 1.6.0_13, and Tomcat 5.5.25. Experiencing the same lag as described in previous comments. The autopublishing process takes at least 30 seconds even after the initial publishing has been done. This happened only after upgrading from Galileo to Helios. Using: Mac Os X 10.6.4 Sun JDK 1.6.0_20 Tomcat 5.5.23 Do we know if this is in fact related to bug 316032, could we duplicated? Can someone test this in this weeks 3.2.1? and confirm that the problem is gone? A co-worker of mine has the same problem on Mac OS-X 10.6.4. He has an SSD drive and with Galileo publishing was lightning fast. Now with Helios it takes tens of seconds just to start copying files. During this phase, no I/O activity on disk is performed, while both cores of the CPUs are busy... I'm talking about both a full republish and incremental ones. As suggested by Angel, he tried to install WTP 3.2.1 (M-3.2.1-20100701091247: is this version ok to test?) from http://download.eclipse.org/webtools/downloads/drops/R3.2.1/M-3.2.1-20100701091247/ and he uncompressed the ZIP in the dropins folder (so that he now has the following structure: eclipse/dropins/wtp/plugins and eclipse/dropins/wtp/features). However, the problem seems not to be solved. Hi! I've downloaded the 3.2.1 version from http://www.eclipse.org/downloads/download.php?file=/webtools/downloads/drops/R3.2.1/M-3.2.1-20100701091247/wtp-M-3.2.1-20100701091247.zip and drop plugins and features folder in my eclipse helios folder Publishing in tomcat works fine now! :D Thank you!!!!!!!!!!!!!!! (In reply to comment #32) > Hi! > I've downloaded the 3.2.1 version from > http://www.eclipse.org/downloads/download.php?file=/webtools/downloads/drops/R3.2.1/M-3.2.1-20100701091247/wtp-M-3.2.1-20100701091247.zip > and drop plugins and features folder in my eclipse helios folder > > Publishing in tomcat works fine now! :D > > Thank you!!!!!!!!!!!!!!! This works for me also. Thank you. My co-worker just tried to unpack the 3.2.1 ZIP directly into the eclipse installation folder instead of dropins, but he still has the problem :-( Just to share some info, we had a strong suspicion this would cause speed degredation initially, but we thought enabling the caching was too risky so close to release, so we saved it for 3.2.1. Created attachment 173528 [details]
Thread dumps of republish process + Eclipse config dump.
Hi all,
I'm the co-worker Mauro Molinari was talking about.
To help you in further investigating I attached a set of thread dumps performed during the re-publishing of the webapp.
Only one source file was touched in order to trigger republishing (an "applicationContext.xml" Spring file).
I also attached my current Eclipse configuration dump.
The web project is showing 15 child-module JAR files.
The WEB-INF/lib path, however, contains many more JAR files (168).
Sorry for not digging into this sooner, but I was mostly on vacation last week. I've traced the code and can confirm that Bug 316032, i.e. the lack of caching, is the cause of the performance issue. Without the caching, the IWebModule.getURI() call would call FlatComponentDeployable.getFlatComponent() which would always return a new IFlatVirtualComponent object whose "members" field was null. The delay would come from scanning the resources in the project to re-initilize that field. This delay would occur for each child module of the dynamic web project. With the caching enabled, the "members" is not null and the delay does not occur. For any who don't see improvement with WTP 3.2.1, check the "bundles.info" file in the "eclipse\configuration\org.eclipse.equinox.simpleconfigurator" folder and make sure the "org.eclipse.jst.j2ee" plug-in is running version 1.1.401 or later. Version 1.1.400 means you still have the lack of caching problem. If there is still an issue, a new bug should be opened. *** This bug has been marked as a duplicate of bug 316032 *** Hi Larry, Thanks for your explaination. It turns out that I'm using org.eclipse.jst.j2ee 1.1.400 and I can't figure out why since I installed WTP 3.2.1 as depicted by Mauro in comment 31 and comment 34. Any suggestion on how to solve this? Thanks again, regards. Try adding "-clean" and "-debug" options to your Eclipse startup. You can add them to the eclipse.ini on separate lines just above the "-vmargs" line. The "-clean" option should force Eclipse to "see" the WTP 3.2.1 plug-ins. If the 1.1.401 version of "org.eclipse.jst.j2ee" isn't able to run, the Error Log view should show you why, thanks to the "-debug" option. If further help is needed, please ask on the WebTools Newsgroup (http://www.eclipse.org/forums/index.php?t=thread&frm_id=88). Hi Larry, I solved uninstalling all the WTP components using p2 and then dropping WTP 3.2.1 to the "dropins" folder. Now the IDE works as expected, so thank you very much! *** Bug 319461 has been marked as a duplicate of this bug. *** *** Bug 319953 has been marked as a duplicate of this bug. *** |