Community
Participate
Working Groups
Created attachment 141791 [details] CCE in log - content type disabled In the WTP build we have many test failures because our persistence.xml and orm.xml files are not being read. The attached stack trace shows the CCE that disables our xml content types. We have determined that moving XmlRootElementVersionContentDescriber and XmlRootHandler from the org.eclipse.jpt.eclipselink.core plugin to the org.eclipse.jpt.core plugin resulted in these problems. In the eclipseLink plugin in our last release we supported both an eclipselink 1.0 xml file and an eclipselink 1.1 xml file. In the current code we are attempting to support the generic 1.0 and 2.0 files in the core plugin so we moved the above classes that handle looking for the version of the xml file in the content type description. Beyond this, we do not know what is causing this failure. The failures do not occur in our local workspaces, only on the build. Tran is going to attempt to reproduce in a local build and Brian/Neil are going to attempt to reproduce in a local workspace on Linux. For the time being we have commented out the test failures in the build, but you can still see the exception in the test log since it sometimes occurs in a background thread.
http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg07047.html From the WTP dev mailing list: Now I think we are getting to the reason for the issue. SAXParserFactory is the JAXP way of creating the SaxFactory to be used. However since you let JAXP determine and figure out the configuration and the parser to be used, you can run into configuration issues. My experience with using JAXP within OSGI based bundles is that you can get some unexpected results that crop up. There are a couple of possible solutions. 1. Try setting the setXIncludeAware() method to either true or false to see if that fixes your problem. You many need to set the other configuration options as well. http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/parsers/SAXParserFactory.html 2. Call and use the Xerces specific implementations directly, and setup your configurations. I personally try to avoid using JAXP within OSGi because of the unpredictablility when it interacts between bundles. It may work for a while, and then it just goes nuts one day. Dave
David, I am trying to reproduce this issue localy and I was wondering if you had encountered the following exception in the past. When I run a modified version of the wtp.tests/testScript/test.xml, the e.o.jpt.utility.tests error log shows: Failed to invoke suite():java.lang.NoClassDefFoundError: Filter while for the e.o.jpt.core.tests I am getting: java.lang.Exception: Could not find plugin "org.eclipse.jpt.core.tests"
No, don't recall getting that, but I could offer a few guesses: 1. org.eclipse.jpt.core.tests must be "unpacked" (that is, the feature must not have "unpack=false" for that plugin. (This might be relevant if you've built it by hand, or something). 2. The tests (and tests framework) must be invoked from Eclipse's antRunner application; not directly from ant. HTH
Created attachment 142058 [details] Could not find plugin org.eclipse.jpt.core.tests Thanks David, I am invoking the tests with antRunner. The org.eclipse.jpt.core.tests is unpacked, and I have verified the installed directory. I have investigated more on why it cannot find the core.tests plugin. It seems that the plugin-path property is not used when invoking wtp-junit-tests.xml, since the result does not change when I modify it. I have attached the full stack trace.
Update - I have finally managed to reproduce this exception using Linux and the IBM JRE 6.0 in a dev environment. The problem reproduces outside of the build (and outside of the tests) in this environment, which is good news for debugging purposes. I haven't started debugging yet, but wanted to let folks know that some progress has been made.
Thanks for letting me know. Is it easy to describe how you run them? I'm also trying to run the tests, stand alone, on windows for now, and found to get plugin-path defined, I had to specify 'testRoot' and 'dropingsFolder' I found this in the wtp.tests/build.xml file. <property name="plugin-path" value="${testRoot}${dropinsFolder}/eclipse/plugins"/> Depending on how you are actually running them, pretty much anything in that build.xml file has to be specified somehow. Are you using that build.xml file directly? Or the "standalone.xml" file? I'm trying the standalone.xml file route, and so far I have a quite a few system properties defined on the ant line. It seems to be going through all the scripts and resulting in "build successful" (i.e. no test failures) but I don't think any tests are actually running yet. Might have to move to Linux :) My script, so far, in a runtests.bat file. set ANT_HOME=D:\apache-ant-1.7.0 set JAVA_HOME=D:\JDKs\ibm-java-win-605 set PATH=%ANT_HOME%\bin;%JAVA_HOME%\bin;D:\Programs\CVSNT;%PATH% set BUILD_HOME=D:\buildsFreshZips\runTests set BASE_BUILDERS=%BUILD_HOME%\basebuilders echo "build home: " %BUILD_HOME% ant -Denv.DISPLAY="" -DkeyCfgFile=D:\buildsFreshZips\runTests\tests\releng\maps\build.cfg -Denv.JAVA_6_HOME=D:\JDKs\ibm-java-win-605 -DbuildDirectory=%BUILD_HOME%\tests -DtestRoot=%BUILD_HOME%\tests -DdropinsFolder=\eclipse\dropins -Dbase.install.dir=%BUILD_HOME%\tests -Denv.LOCAL_PREREQS_CACHE=%BUILD_HOME%\prereqsCache -Denv.RELENG=releng -Denv.RELENG_BUILDER=releng.wtpbuilder -Dbuild.home=%BUILD_HOME% -DbuildType=I -DbuildId=3.2I -Dtimestamp=20090719171559 -Denv.BASEOS=win32 -Denv.BASEWS=win32 -Denv.BASEARCH=x86 -Dbaseos=win32 -Dbasews=win32 -Dbasearch=x86 -Dbuild.stream=R3.2 -DbuildBranch=R3.2 -Djava16-home="D:\JDKs\ibm-java-win-605" -DtestTarget=dali-core-tests -f D:\buildsFreshZips\runTests\releng.wtpbuilder\distribution\wtp.tests\standaloneTest.xml
(In reply to comment #5) > Update - I have finally managed to reproduce this exception using Linux and the > IBM JRE 6.0 in a dev environment. Additional data: I have further isolated the problem to only be reproducible with the IBM JRE, which has been confirmed on IBM Java JRE 6.0 Linux SR3 and now SR5. I am unable to reproduce with Sun JRE 1.6.0_14.
I am launching the tests from releng.wtpbuilder/build.xml, therefore it is calling the build.xml file directly. And yes I have defined the 'testRoot' and 'dropingsFolder', but I found that the 'plugin-path' property defined in the wtp-junit-tests.xml is overridden by the org.eclipse.jpt.core.tests/test.xml script. And no matter what value I give in the test.xml script, I am getting the same exception.
Created attachment 142089 [details] org.apache.tools.ant.UnknownElement When I turn on failonerror in wtp-junit-tests.xml, I can see an Ant exception <java taskname="suiteUnitTest" fork="true" resultProperty="suitejunitresult" failonerror="true" timeout="${timeout}" dir="${testRoot}/eclipse" jvm="${jvm}" classname="org.eclipse.equinox.launcher.Main"> It seems like Ant is throwing an exception on an unknown element. I tried to switch the Ant version to 1.6, but I am getting the same result. Do you know how to get further details? Thanks.
(In reply to comment #9) > Do you know how to get further details? > I you use -debug for the ant invocation, you'll get tons of detail. My guess is that there's a missing property value somewhere, that causes XML to become invalid (for example, in the snippet you gave, if "timeout" was an empty string, the XML (to ANT) might end up with an attribute that looked like timeout= dir="${testRoot}/eclipse" Just as an example.
I've opened bug 284142 to continue the discussion of the local build tests not running.
If you are relying on the OSGi ServiceTracker to get you a SAX based parser, it may be getting you one that is XInclude aware. new ServiceTracker(bundleContext, SAXParserFactory.class.getName(), null); However the default XML Reader by Xerces is not XInclude aware. if (factory.isXIncludeAware()) { factory.setXIncludeAware(false); } So you may need to specifically tell the factory to not enable XInclude processing. I'm in the middle of testing this fix for the XSL resolving items that Neil pointed out. What is odd is that depending on how this is run and where can affect the type of SAX based parser that is returned. If you know you don't need XInclude processing you may beable to just turn it off.
Earlier today I discovered that there was a similar problem occurring in the wst.xml.core tests (see bug 284200). This led me to think about a difference between our jpt.core and jpt.eclipse.core, which was a dependency on wst.xml.core in our jpt.core. Upon removing the wst.xml.core dependency in jpt.core I observed that we were able to successfully build the SAXParser while using the IBM JRE. This points to problems with getting incompatible content from wst.xml.core usage. In my debugging I haven't noticed any XInclude aware Parsers being used. All have been false. I will double check this though.
(In reply to comment #12) As we discussed, I tried forcing the factory to be XIncludeAware with no luck. In the environment where parser creation fails the XIncludeAware attribute in the SAXParserFactory is 'false' on arrival. I've also tried setting XIncludeAware to 'true' on the factory, but this doesn't help either. The same exception is encountered. I'm somewhat thinking (hoping) that when the problem that exists in wst.xml.core is fixed, our problem will go away with it. As I had mentioned before, when we remove our dependency on the o.e.wst.xml.core plugin, this entire problem goes away for us. That said, I suppose there are probably other workarounds that could be deployed.
(In reply to comment #14) > (In reply to comment #12) > As we discussed, I tried forcing the factory to be XIncludeAware with no luck. > In the environment where parser creation fails the XIncludeAware attribute in > the SAXParserFactory is 'false' on arrival. I've also tried setting > XIncludeAware to 'true' on the factory, but this doesn't help either. The same > exception is encountered. > > I'm somewhat thinking (hoping) that when the problem that exists in > wst.xml.core is fixed, our problem will go away with it. As I had mentioned > before, when we remove our dependency on the o.e.wst.xml.core plugin, this > entire problem goes away for us. That said, I suppose there are probably other > workarounds that could be deployed. > Fixing the issue in XSL, won't fix the issue in Dali, but the same solution should work, once I get a build that I can verify that my proposed work arounds are happening. Doing some more research on this issue has revealed that this may be a known problem with class loaders and context. http://xerces.apache.org/xerces2-j/faq-general.html#faq-5 The problem appears to be JRE related. This particular ClassCastException appears to crop up everyonce in a while. I found similar issues in the platform bug reports for Ant: https://bugs.eclipse.org/bugs/show_bug.cgi?id=143008 The problem is that it appears the Orbit Xerces and the JRE's xerces implementation may not necessarily be the same version. If this happens you can get a mixture of the two versions and you run into this class cast exception. I believe we had some Xerces and IBM JRE issues in the past that we had to work around but not sure if this is a similar problem or not. Bottom line, fixing the XSL issue, isn't going to magically fix the Dali issue, but the same solution once discovered should work.
XSL tools issue is fixed, JPT still is throwing the same issue. See comments in bug 284200 for how it was resolved in wst.xsl.
Note, with or without Dave's good advice, you can use a system property -Djaxp.debug=1 to cause diagnostic information to be printed to system err. This would be a ton of stuff (Not sure, you might be able to run it on just during your tests). But, one piece of info it'll tell you is how (and when) it was deciding exactly what parsers to create (e.g. getting hints from property file, a META-FILE/services file, or hard coded implementation class). This would then tell you (or rule out) an "surprise" sources (e.g. issues like this can come from 'commons.logging', I've heard) and/or might suggest order differences you could control in manifest prereqs. Just thought I'd remind us all of this technique, if it might help.
Updated bug summary and severity to reflect the current status of the bug. This is not simply a test issue. Update - Removing the exporting of the xerces plugin dependency from wst.xml.core appears to completely solve the parser creation problems that we are experiencing in Dali. I'm examining the jaxp debug logs to identify any differences between the failing and non-failing case. See bug 284200 for additional details.
(In reply to comment #18) > ... Removing the exporting of the xerces plugin dependency from > wst.xml.core appears to completely solve the parser creation problems that we > are experiencing in Dali. ... You do know that's not a solution, right? Its a little like saying your boat won't sink if you take it out of the water. (ha ha ... forgive my bad jokes). You should experiment with changing the order of your classpath (is my guess) via the order of your required bundles.
(In reply to comment #19) > (In reply to comment #18) > > ... Removing the exporting of the xerces plugin dependency from > > wst.xml.core appears to completely solve the parser creation problems that we > > are experiencing in Dali. ... > > You do know that's not a solution, right? Its a little like saying your boat > won't sink if you take it out of the water. (ha ha ... forgive my bad jokes). > > You should experiment with changing the order of your classpath (is my guess) > via the order of your required bundles. > Actually, I did this experiment with XSL tools, it didn't fix the issue. Only two solutions fixed it, and one is the correct way to do it. 1. Calling the Xerces SaxFactoryImpl directly instead of relying on the standard Jaxp factory method. 2. Removing the Re-Export of the Xerces dependencies. Committers from PDE have already said we should not be re-exporting plugin dependencies. Fixing item 2 is the correct solution, as I've seen numerous other bug reports in EMF and other eclipse plugins that start having this classcast issue once WTP's plugins are installed. I understand that fixing this is technically an API breakage, but forcing work arounds is even worse for an adopter.
Created attachment 142966 [details] fix to get factory with proper classloader The issue here is that you get the SaxFactory using the (internal!) Activator plugin from that platform plugin (org.eclipse.core.contenttype). Then, when you go to get the SaxParser from that Factory it does what it always does, and uses the Factory's classloader to look up how and where to create the SaxParser. In my patch, I've duplicated the platform's method to get the Factory, but put it in your own JpaCore Plugin, and it would return the same factory class (from the JRE, in both cases, since it comes from an OSGi Service) ... but, this Factory class now has YOUR classloader. Hence, when you ask it for for a SaxParser, it will use YOUR classloader to look up which SaxParser to create, and therefore find one that's consistent with other XML Code in your plugin. So, good news ... not only fixes your classloader problem, but also removes your use of an Internal Class! I tested this on my local machine and it fixed the issue I was seeing in my standalone Dali test ... let us know if it fixes your issue and/or causes other ripple effects. [And, then maybe you can fix the abstract title of this bug :) ]
Better count comment #21 as a theory. I tried re-running my standalone test to confirm, and now it won't run at all, so doesn't give me confidence the one test was correct. It's a good theory though :)
(In reply to comment #22) > Better count comment #21 as a theory. I tried re-running my standalone test to > confirm, and now it won't run at all, so doesn't give me confidence the one > test was correct. It's a good theory though :) > I never could get my standalone test running again, but learned how to reproduce in workspace, could see my solution in #21 did not work, and toyed with some other solutions. First, Dave's suggestion to use the implementation of the factory worked for me. I still used the OSGi service ... just to try it; parserTracker = new ServiceTracker(getBundle().getBundleContext(), "org.apache.xerces.jaxp.SAXParserFactoryImpl", null); I also try setting the Thread.currentThread().setContextClassLoader, as is sometimes recommended in some web hits I found on internet ... but not sure this solution is quite right either (only advantage, I guess, is the specific impl class doesn't have to be hard coded in your code, such as if xml.core ever stopped using xerces, your plugin would continue to work). I do feel something is wrong with xerces and IBM's compatibility, so I've also also contacted my favorite expert on the subject, and will see if he has a recommendation. But in the short term, I'd recommend you use the specific implementation as Dave suggests.
Created attachment 143069 [details] patch using the context classloader solution Some xerces team members pointed me to a xerces faq that involves multiplie classloader issues, http://xerces.apache.org/xerces2-j/faq-general.html#faq-5, which points to "how to" paper at http://www.ibm.com/developerworks/websphere/library/techarticles/0310_searle/searle.html. We have used this classloader context technique in other places in our WTP code, such as see org.eclipse.jst.j2ee.internal.xml.GeneralXmlDocumentReader, so I think it is a reasonable "by design" technique, that will work on any JRE, and, even, in case xerces happens to be removed later. Technically, you might not have to remove your dependency on the internal Activator class from the Platform, but it would be best to get rid of that anyway, so I've left that change in this patch. I tested this patch with my little dev. env. test case, but not with Dali test cases, per se. BTW, given the lateness, I'd be fine promoting a build and simply listing the limitation.
(In reply to comment #24) > BTW, given the lateness, I'd be fine promoting a build and simply listing the > limitation. > Let's do that. I'll do some testing with the patch.
Comment on attachment 143069 [details] patch using the context classloader solution The patch appears to work on IBM and Sun JRE's. I'm happy with this for now since it does appear to be the "official" workaround to the problem. Patch committed with very minor tweaks. I've uncommented the failing tests suites to see if we get a passing result from the build. I'll update the bug in the morning based on the results.
This final patch (07-30) works both locally and in the build so setting this to fixed. Thanks for everyone's help on this.
*** Bug 285329 has been marked as a duplicate of this bug. ***
For future reference, using the Platform's (internal) Activator to get the SAXFactory would have (still) failed, even using the Threads context classloader (TCCL). The reason is that the Platform's bundle gets a SAXFactory appropriate to it (and it's bundleContext), and it already has one by the time Dali requests one, so that would almost never be the right thing to do. On the other hand, I've observed just using the normal xerces recommend way to get a Factory, namely SAXParserFactory.newInstance(), works fine in this Dali case, without even using the TCCL. I'll continue to work with OSGi experts to at least document some "how to" best advice, if not actually make it work transparently.
(In reply to comment #29) > For future reference, using the Platform's (internal) Activator to get the > SAXFactory would have (still) failed, even using the Threads context > classloader (TCCL). > > The reason is that the Platform's bundle gets a SAXFactory appropriate to it > (and it's bundleContext), and it already has one by the time Dali requests one, > so that would almost never be the right thing to do. > > On the other hand, I've observed just using the normal xerces recommend way to > get a Factory, namely SAXParserFactory.newInstance(), works fine in this Dali > case, without even using the TCCL. > > I'll continue to work with OSGi experts to at least document some "how to" best > advice, if not actually make it work transparently. > Need to be careful with SAXParserFactory.newInstance() as this can lead to a lot of copies of the SAXParserFactory being around. If a particular class is long running and isn't garbage collected or freed by the JVM, you could have many copies of it. Ideally, I would recommend having a pool of the particular Parser pool you want already configured. That is the purpose of the OSGI XMLParser service, to provide you with a parser that corresponds to the configuration you requested.
For cross and future reference, see bug 285505. I think the equinox framework can improve future clients transitions/refactorings -- after Helios M2 --- but there should not be a need for you to change your code ... unless you just wanted to. Just wanted to document it.
*** Bug 285751 has been marked as a duplicate of this bug. ***
