Community
Participate
Working Groups
Build Identifier: 20100218-1602 Hi. I have a dynamic web project and found out that sometimes its external jars are published to server well, and sometimes they are not, with no evident reasons. I have debugged a lot, running the project just after workbench load, and observed the moment when the org.eclipse.jst.j2ee.internal.deployables.J2EEFlexProjDeployable for my project is created. Important thing here is which org.eclipse.jst.j2ee.componentcore.J2EEModuleVirtualComponent object it gets, as it calculates members() using the component javaRefsOnly field. And the matter is that there are 2 J2EEModuleVirtualComponent objects created before J2EEFlexProjDeployable: 1. from JavaReconciler.initialProcess(), as in order to resolve some classpaths it initializes WebAppLibsContainer. This object does not resolve javaRefsOnly or even when (very seldom) resolves them, gets all the external jars correctly. 2. from ServersView2.createPartControl() as I have a Servers view opened. This object resolves javaRefsOnly just after creation and gets an empty array of them. (There is also one more way to get a component: from ServerEditor.createPartControl() if a server editor is opened, so it should be closed for simplicity.) So I observe 2 stable scenarios: J2EEFlexProjDeployable object gets component #1, calculates libraries correctly and publishes all the stuff; or it gets component #2, calculates members() with no libraries and does not publish them. This looks quite natural: JavaReconciler and GUI runs 2 distinct threads and whn J2EEFlexProjDeployable object is created, it gets its component from some resolved components store which got component from the thread which was earlier. Probably, there's some bug in this behavior as well, but the most evident question is why component #2 gets empty javaRefsOnly. I have debugged more and observed that when it calls ClasspathDependencyUtil.getComponentClasspathDependencies method and the latter calls javaProject.getResolvedClasspath(true) the result is JavaProject.RESOLUTION_IN_PROGRESS which is exactly an empty array (and an array of 41 element for component #1, to stress the difference). But the spelling "RESOLUTION_IN_PROGRESS" guesses that the caller should wait until resolution is ended, whereas the caller treats it as a final result (and also this is why component #1 is resolved correctly - it resolves in the same thread with JavaReconciler so it is natural that it does this after classpaths are resolved, whereas component #2 resolution is held simultaneously with classpath resolution). So I would ask to fix ClasspathDependencyUtil.getComponentClasspathDependencies to treat correctly the RESOLUTION_IN_PROGRESS result and also clarify logic. There one can see comments like "//TODO this call to javaProject needs to be removed. Need to figure out what exactly this is attempting to do." And I have checked: they persist in the latest version the same as in my 1-year old version of org.eclipse.jst.j2ee plugin. Reproducible: Sometimes Steps to Reproduce: 1. Start workbench 2. Run the project or select "publish" in context menu of server and see the result in the folder where it is published.
I have debugged a bit more and things now are quite clear. There are 2 threads. 1. GUI, where ServerViewer is initialized and calls org.eclipse.jst.j2ee.internal.deployables.J2EEDeployableFactory.createModules(...){ IVirtualComponent component = ComponentCore.createComponent(project);//point #1 if(component != null){ return createModuleDelegates(component);//point #2 } return null; } 2. JavaReconciler, calling ComponentCore.createComponent(project) (point #3) for resolving reasons. Thus, they both call ComponentCore.createComponent (points 1 and 3) to create (or get already created) a component and then thread 1 creates deployable at the point #2. Most natural is to expect that the deployable is created with component created by thread 1. But let's have a look at org.eclipse.wst.common.componentcore.internal.util.ComponentImplManager.createComponent(IProject project, boolean checkSettings) { try { IVirtualComponent component = ComponentCacheManager.instance().getComponent(project);//point #5 if(component != null) { return component; } IComponentImplFactory factory = findFactoryForProject(project, descriptors); if(null != factory){ component = factory.createComponent(project);//point #6 if(component != null) { ComponentCacheManager.instance().setComponent(project, component);//point #7 registerListener(project); } return component; } } catch (Exception e) { // Just return a default component } <... more code here...> } This method is called from ComponentCore.createComponent and it gets a component from cache if it was already created (point #5) or creates (point #6) and puts into cache (point #7) otherwise. But in fact, there are more side-effects. One of the created components javaRefsOnly is resolved incorrectly. But it is not always the one created by thread 1 as org.eclipse.jst.j2ee.internal.common.classpath.J2EEComponentClasspathContainer.update(), resolving the javaRefsOnly field also calls ComponentCore.createComponent to get the component and it gets the one saved in cache. So it does not matter, which component is put into deployable, the success secret is that it should either: 1. not be the component saved into cache; 2. be resolved after JavaReconciler resolves classpaths. And there are 3 scenarios for the deployable: 1. thread 2 worked much quicker, it puts its component into cache, it is resolved and deployable gets it correctly - success 2. thread 2 is quicker but not so quick, the component from thread 2 is into cache, but resolved by GUI thread incorrectly - fail 3. thread 1 is quicker, its component is in the cache, gets to deployable and most probably resolved incorrectly as JavaReconciler has not finished reconciling - fail 4. (usually observed in debug but possible in real life as well) both threads get into ComponentImplManager.createComponent method simultaneously (it is not sinchronized!) and cache randomly gets the component from thread 1 or 2, whereas deployable gets the one from thread 1. If thread 1 is quicker, then deployable and cache components are different - success, otherwise - fail. So it is quite realistic but complicated picture of why library publishing sometimes works and sometimes - does not work. One can see that in order to reproduce this behaviour one needs large Java project (for Java reconciler to run for a long time). A thing, adding chaos is also that ComponentImplManager.createComponent method is not synchronized, which causes lots of unexpected behavior. It looks like a bug as well, but I am not sure. ClasspathDependencyUtil.getComponentClasspathDependencies behavior is incorrect surely.
Assigning to Jason for initial investigation.
The reference caching in the J2EEModuleVirtual component should be improved to account for these different code paths and ensure it is always updated correctly and returns consistently. I'm not worried about ComponentCore not being synchronized as long as the caching is fixed. Sure, the references may be computed twice, once for the component in the cache and once for the duplicate component, but that's OK as long as the results are consistent. I'm more worried about the deadlock risk which may be introduced if ComponentCore were synchronized.
Back in March bug 307615 fixed some caching issues in J2EEModuleVirtualComponent by removing caching for classpath references in the getJavaClasspathReferences() method. I believe this change fixed the issues you mentioned. Please update to the latest WTP version and let us know if you still see any of the inconsistencies that you mentioned.
Vasily, I am going to close this as FIXED, as per Aidyl's comment #4. If you do not believe that this is fixed, please reopen it with further details/information.
Well, it looks like the proposed change will solve the problem. I am not sure if it does not create side-effects (I believe there was really a reason to introduce classpath references caching). What concerns testing, I would prefer to try the nearest stable Eclipse release instead of just downloading latest WTP.
