Community
Participate
Working Groups
Build Identifier: Hi, I've been doing some work on m2eclipse integration for WTP for quite some time now and I've thought of a few enhancements in WTP's API that'd make my life easier : I'd like to create custom components that would generate archives the maven way, and which could be consumed by WTP for export/deployment. For instance, for a web project we would have in its .component : <dependent-module deploy-path="/" handle="module:/ejb-client/someejbproject"> <dependency-type>consumes</dependency-type> </dependent-module> <dependent-module deploy-path="/" handle="module:/test/someutilityproject"> <dependency-type>consumes</dependency-type> </dependent-module> The ejb-client module would allow to deploy a ejb client jar in the web project, defined in maven (seelcting a few classes and interfaces in someejbproject) The test module would allow to deploy a jar containing only the test classes of someutilityproject. I've discussed this feature with Rob Stryker, according to him : - "J2EEFlexProjDeployable has the list of participants. some of these participants (JEEHeirarchyExportParticipant) tells what is a child module and what is not. You'd need something to recognize a reference as a child module and not just as resources to be published" - "so you'd just want to use maven for the overlays, filesets, include,s excludes, etc and then expose it all via virtualfolder and virtualfile and then designate that it's a child module and needs to be zipped" - "to get your custom reference recognized as a zipped child inside a war project, might require small changes to j2eeflexprojdeployable to allow other adopters to add participants" We might also want to be able to let a 3rd party plugin (maven) generate the archives completely, and tell WTP to pick this archive. What do you think? Reproducible: Always
I think this idea is solid. The only thing I see that is slightly wrong is that under the current model, in order to be a child module, you must be a "USED" reference, not a "CONSUMES" reference. So even if / after we patch J2EEFlexProjDeployable to accept extensions, you'd need to change that bit ;) You can see in FlatVirtualComponent, when handling consumed references, it does not check if they're child modules or not. However handling USED references clearly checks isChildModule(etc). Publishers typically only can zip up child modules since they don't really inspect non-child resources. Other than that, it'd be interesting to hear the jee team's thoughts on this bug.
> We might also want to be able to let a 3rd party plugin (maven) generate the archives completely, and tell WTP to pick this archive. To do this would be very simple and no different than making a regular USED archive reference. This would require no changes to J2EEFlexProjDeployable as far as I can tell. The real case that would require changes to flexprojdeployable is marking a USED project reference (or a used reference type you created on your own) as a child module so it can be zipped. JEE: what do you guys think of allowing adopters to add participants via some API? To allow them to do it would open the projects to inconsistant behaviour if rogue plugins are changing behaviour, but on the other hand there is currently no way to mark a new project type as a child reference without such changes.
This sounds like useful work to me (keep in mind I know nothing about the technology ... just oft requested function). To cross reference other work in this area ... some have said (on cross-project list) they want to reinvigorate the m2e project, and provide maven support (?connector?) for Java projects, but they would not have time/resource to do similar for JEE projects. Is this work related to that work in any way? Or just coincidental timing? See bug 330226 as well as the cross-project list, for some of the discussions.
Just coincidental timing, David ;) Although this is an adopter (m2e) trying to do cool things with the flat virtual component which was added in last release, which is pretty neat.
David, just FYI, m2eclipse-wtp is not *officially* supported by Sonatype, but unofficial support consists of setting up a CI build, helping us on technical questions, pushing releases ... Bottom line is m2eclipse-wtp won't be migrated to eclipse.org (at least for the time being). I have a few more requests in mind that should improve the maven/WTP integration. You'll here from me again soon.
(In reply to comment #3) > This sounds like useful work to me (keep in mind I know nothing about the > technology ... just oft requested function). > > To cross reference other work in this area ... some have said (on cross-project > list) they want to reinvigorate the m2e project, and provide maven support > (?connector?) for Java projects, but they would not have time/resource to do > similar for JEE projects. > > Is this work related to that work in any way? Or just coincidental timing? > > See bug 330226 as well as the cross-project list, for some of the discussions. David, I would go so far and say this is exactly what will help us provide better Maven support for WTP. The majority of the recent fixees we (JBoss) have pushed for in WTP with respect to module assembly have been driving from the needs we saw to get Maven (and other modular build tools) to coexist better with WTP (and vice versa). We are working with Fred Bricon (maintainer of the current m2e WTP project) to do what we can - this one is one of them.
I vote for this as well. I'll add this as a proposed item in our plan. I welcome this integration as well, and will do my best to accommodate your requests Fred.
This enhancement will have to be pushed forward to 3.3.1.
Detailing the ejb-client use case : Let's say we have an EJB and a Web project in the workspace, both mavenized. The maven EJB project declares it exposes/exports an EJB-Client jar, containing mostly interfaces, but not only (see http://maven.apache.org/plugins/maven-ejb-plugin/examples/generating-ejb-client.html). The web project depends on the EJB-Client, so it will only see a subset of the classes from the EJB project in the workspace. Now we want to deploy the Web project on a server, but instead of jarring the content of the EJB project with it, we need to jar only the declared ejb-client classes. Creating an archive containing only a subset of classes infered from the maven definition is doable in m2e-wtp, using a custom IVirtualComponent implementation. However, this component needs to be treated as a child module in order to be zipped during export/deployment. Failure to do so will result in failure to start any server not supporting exploded jars.
Created attachment 213383 [details] First patch (no behavior change) on potential changes The following is a potential patch for opening up this list to 3rd party plugins. What we see here is the addition of an extension point for registering IFLattenParticipantPRoviders, which is a simple interface designating a class that can return IFlattenParticipant objects when queried. These providers, in the extension point, provide a class, and a weight. The weight is to keep this list of providers ordered. The model, when asked to find JEESingleROotParticipant, will return the first non-null result upon iterating through the list of providers and asking for the JEESingleROotParticipant participant. IF multiple participants CAN return here, the first one (highest weighted) that responds non-null will get to provide it. Additionally, we now see that modules and their delegates, such as the J2EEFlexProjDeployable class, will send all queries for their participants through this model. The deployables themselves will now return a list of strings, participant names, which their super class will interpret into the actual participants. (Subclasses of FLatComponentDeployable can of course override this and hard-code their own list of participants if they wish.) SO far, everything above changes no behavior and simply introduces new (internal) classes and methods. Officially, the entire flatten component model is still internal, and this may need to be opened up one day. What's left to do to fully open up is to allow actual additions to these lists somehow. Right now, each deployable has its list of what its DEFAULT participant list looks like. Current ideas include persisting a full list either in the component.xml file, or a new settings file. If the setting / file is not there, however, the default list would still be used. New (internal, but accessible) API would be put out to add or remove a participant from the list. SO, for example, maven would be able to use some api to add MavensCustomParticipant to SOmeDynamicWebProject, and at what position in the list it should go. The list would then be persisted somewhere. Other usecases, however, are looking for the ability to register GLOBAL participants. The idea of global participants certainly makes life easier for some adopters, but can be a bit error prone. If a global participant is not carefully coded, ESB projects, for example, may receive functionality that the global participant only intended for JEE projects. Global participants would also not really give users any clue what's going on. Settings stored in a .settings file would at least let users pass us information that might help us debug what's going on, or at least know whether their problem is with an eclipse participant or not. Admittedly the current patch suffers from the same problems, in that a tricky adopter could contribute a participant provider with a higher weight that responds to the same ID, again leaving users (and us) a little confused. The only workaround I have for this would be contributing tracing and ensuring that class names are logged, but that's about it. Possible solutions are to allow a Global participant list, and deployables which wish to use them can, though not all are required to. (Obviously the intention here is that jeetools would accept the global participant list and add it to their custom list items). In the event that a user (or adopter) does choose to persist the list for some project, the first modification (such as to add MyCustomParticipant) would take the web-project's default participants + global participants + MyCustomParticipant and store it as a setting. At this point, only the project setting list would be the one "running the show" so to speak. I'm definitely open to ideas for this. :)
One of the things to note here is that JavaEEComponentExportOperation does not use exactly the same list of participants as J2EEFlexProjDeployable. To be fair, the list is very similar, but it is not 100% unified. J2EEFlexProjDeployable: participants.add(new SingleRootExportParticipant(new JavaEESingleRootCallback())); participants.add(new JEEHeirarchyExportParticipant()); participants.add(new AddClasspathLibReferencesParticipant()); participants.add(new AddClasspathFoldersParticipant()); participants.add(new AddMappedOutputFoldersParticipant()); participants.add(new IgnoreJavaInSourceFolderParticipant()); if (ClasspathDependencyEnablement.isAllowClasspathComponentDependency()) { participants.add(new ReplaceManifestExportParticipant(new Path(J2EEConstants.MANIFEST_URI))); } JavaEEComponentExportOperation: participants.add(createHierarchyParticipant()); // DIFFERENT, custom inner class participants.add(new AddMappedOutputFoldersParticipant(filteredExtensions)); // DIFFERENT, filters stuff participants.add(FilterResourceParticipant.createSuffixFilterParticipant(filteredExtensions)); // DIFFERENT, new addition participants.add(new AddClasspathLibReferencesParticipant()); participants.add(new AddClasspathFoldersParticipant()); if (ClasspathDependencyEnablement.isAllowClasspathComponentDependency()) { participants.add(new ReplaceManifestExportParticipant(new Path(J2EEConstants.MANIFEST_URI))); } The differences here seem to be a simpler heirarchy participant which designates any non-binary component as a child, a different constructor for AddMappedOutputFoldersParticipant, and the addition of FilterResourceParticipant. To note, the addition of FilterResourceParticipant seems to be redundant, since AddMappedOutputFoldersParticipant has the exact same participant listed as a delegate participant inside it. It would seem having a FilterResourceParticipant added in JavaEEComponentExportOperation is redundant and not necessary.
Created attachment 213598 [details] Version two with fixes for unit tests Some unit tests were failing and this looks like it fixes them.
I can't really comment on the content of the patch, I don't know much about WTP's internals, but I hope other core WTP committers can take a look. Anyway, a few minor coding remarks : - use StringBuilders (not synchronized) instead of StringBuffers (synchronized) - give initial size to new list instances when possible - since public void addFlattenParticipant(String id, int position) is public, you might want to add some guard conditions (not null/empty, position >-1) About protected IFlattenParticipant[] getFlattenParticipants(String[] ids), you should definitely log an error if the participant is not found: could mean the project was created on a machine having the right plugins (ex. m2e-wtp) and opened on an eclipse instance not having them installed. Also, if you could add some code snippet(s) showing me how, as an adopter, I could add new participants -say MavenPackageParticipant- to a module component, that would be helpful. I really hope we can add that feature for Juno, so I can start working on implementing my stuff ASAP. Thanks for your involvment in this Rob, I'm very glad this is going in the right direction.
I have started reviewing - and most of these changes initially look good. Creating ext point - so the providers are read dynamically - also removing the unnecessary Java EE dependency for new adopters of this API. I'll continue to investigate the changes to areas like the JEEExportOperation.. And of course running the full test suite. I'm also adding Carl and Roberto to cc list
Rob.. can you try running a test I think you wrote a couple years ago? I get a NPE on this.... Web Operation, Servlet, & Deploy Tests org.eclipse.jst.j2ee.flexible.project.fvtests.WebDeployTest testMembersDeployment(org.eclipse.jst.j2ee.flexible.project.fvtests.WebDeployTest) java.lang.NullPointerException at org.eclipse.core.runtime.Path.append(Path.java:261) at org.eclipse.wst.common.componentcore.internal.flat.ChildModuleReference.<init>(ChildModuleReference.java:68) at org.eclipse.wst.common.componentcore.internal.flat.FlatVirtualComponent.addUsedReferences(FlatVirtualComponent.java:241) at org.eclipse.wst.common.componentcore.internal.flat.FlatVirtualComponent.treeWalk(FlatVirtualComponent.java:168) at org.eclipse.wst.common.componentcore.internal.flat.FlatVirtualComponent.cacheResources(FlatVirtualComponent.java:121) at org.eclipse.wst.common.componentcore.internal.flat.FlatVirtualComponent.fetchResources(FlatVirtualComponent.java:101) at org.eclipse.wst.web.internal.deployables.FlatComponentDeployable.members(FlatComponentDeployable.java:227) at org.eclipse.jst.j2ee.flexible.project.fvtests.WebDeployTest.testMembersDeployment(WebDeployTest.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62) at org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness$1.run(PlatformUITestHarness.java:47) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4140) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3757) at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1015) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332) at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:909) at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:85) at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:580) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:535) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124) at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.runApp(NonUIThreadTestApplication.java:54) at org.eclipse.pde.internal.junit.runtime.UITestApplication.runApp(UITestApplication.java:41) at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.start(NonUIThreadTestApplication.java:48) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584) at org.eclipse.equinox.launcher.Main.run(Main.java:1438) at org.eclipse.equinox.launcher.Main.main(Main.java:1414)
Hi Chuck: If I remember correctly, last week, when running these tests, I was just as confused about this particular one as you are. It seems to fail when run as part of the suite, but succeed if run separately (sometimes). Since the behaviour (when I ran the tests) was hte same with or without my patch, I didn't think it was relevant for this discussion. I just ran the test again standalone in eclipse 4.2 wtp 3.4 without running the entire test suite, and WITH my patch, and it seems to pass. This particular test seems intermitent in its success or failure status. Very annoying.
* Explain why you believe this is a stop-ship defect. Or, if it is a "hotbug" (requested by an adopter) please document it as such. This is a 'hotbug' requested by the m2e-wtp team, who needs this for customizing projects' export and publish capabilities based on data in the maven models. * Is there a work-around? If so, why do you believe the work-around is insufficient? There is no clean workaround for this. There are only two real places that publishing can be customized. The first is at the module level, when listing a module's members, and the second is at the server level, who receives the list of member resources from the module and decides how to zip them / publish them. Without a fix usable on the module level, as this patch is, m2e-wtp would need to make their own utilities, and then CONVINCE all server adapters (jboss, generic server, glassfish, tomcat, etc) to depend on m2e-wtp and make use of this utility. Needless to say, most adapters would not prefer to add unnecessary dependencies on m2e-wtp. Resistance will be very large, and m2e-wtp will most likely find themselves unable to accomplish their goals. * How has the fix been tested? Is there a test case attached to the bugzilla record? Has a JUnit Test been added? I (Rob Stryker) have tested this extensively, and written a few unit tests for coverage. * Give a brief technical overview. Who has reviewed this fix? Currently, jeetools has one super-class which is popular for virtual component module projects. This class is FlatComponentDeployable. Subclasses most-often override a method, named getParticipants(). These participants are involved in turning a project into a form suitable for servertools API, and thus, for consumption by server adapters. For the past 2-3 releases, any subclasses would simply return an array of IFlattenParticipant objects which they insist be part of their processes. For Jeetools, for example, this list contains functionality such as determining what can be a child module, which files must be excluded (source code, .svn, etc), replacement of the manifest file with an updated version, and others. Outside adopters may use this superclass for other project types, such as esb, portal, management archives, or any other custom application type. At nowhere in this process, however, could a tool such as m2e add, for example, a fileset filter based on a project's pom.xml's include/exclude patterns. While trying to increase extendability and letting other extenders like m2e-wtp reach into our private zones, careful attention was spent ensuring that users will not be confused and that there is a place users can check what is going on. The list of participants will be stored in the org.ecli[pse.wst.common.component xml file. API's have been written for adding and removing a participant to a project type. A note for current extenders: Current extenders currently override the getParticipants() method and return a list of participants by some internal logic of their own. Any current extenders should not be affected by this change. Their list of participants will be respected. They will not, however, benefit from behavior additions or removals. The internal property set inside org.eclipse.wst.common.component will be ignored by these adopters unless they update their behaviour. To comply with the new workflow and benefit from these changes, adopters should stop overriding getParticipants(), and instead perform the following actions: 1) override getDefaultFlattenParticipantIDs() with a list of string participant id's 2) use the org.eclipse.wst.common.modulecore.flattenParticipantProvider extension point to register a participant provider which will resolve these participant id's into actual participants. Users should be careful here to either use approved id's that already resolve, or use custom strings that only they will resolve. Resolving id's that do not belong to you is frowned upon and goes against the spirit of this patch. * What is the risk associated with this fix? The risk is non-zero, but low. The entire test suite is unchanged by this patch, which is a promising sign. The most troubling part of this patch is what sloppy or crafty adopters may do with this new access, which may confuse users. We believe the risk of this is low, however.
seeking approval from chuck
Thanks Rob for a very good explanation of these changes. I have been running tests for the last couple days, and haven't found any regressions. As Rob stated, any existing extenders of FlatComponentDeployable(we have a couple at IBM) will not be affected - as they override their own "getParticipants" but can move to this open scheme on their schedule. I vote to accept this change.
Rob, Chuck, I just released these changes to HEAD for WTP 3.4
Thanks for the help, Roberto.
This fix was dropped 2 weeks ago into HEAD... resolving....
