Community
Participate
Working Groups
Build ID: M20060921-0945 Steps To Reproduce: 1.The execution of an AGR test (local) with RAC running with security enabled hangs at 60%. 2. I have not been able to run other test types (manual, Junit) with local security enabled on Vista either. More information: I have tried restarting RAC, running as administrator, and restarting workbench.
This is more suited to be critical than major (unless there is a workaround to it).
Updating to critical as per Ali's request and after trying this again. It is possible that this is a configuration problem on my end but I can't seem to get around it.
Updated the description to be more clear. I am also experiencing this with remote tests where security is enabled only on the remote host.
*** Bug 168949 has been marked as a duplicate of this bug. ***
Assigning to Samson since the problem appears to be in the security layer. Copying Igor.
After our complete test pass today it was discovered that this is also happening on Windows 2000 with the 4.2.2 candidate driver. Please investigate. Steps to reproduce: 1) Ensure you have the auto gui plug-in from the Technology preview section of the download page 2) Create a new Plugin project 3) Select File> New> Test> TPTP Automated GUI Test> And place it in the src folder of your plugin from 2) 4) Record a test case that creates a new Java Project (select the recording button in the Test Cases page and perform UI actions) 5) Add the test case to the behaviour tab of the test suite 6) Save it 7) Ensure local RAC is running with security enabled 8) Create a new Test launch configuration and select the test suite to run 9) Run 10) Notice that it hangs at 57% on Windows 200/XP and 60% on Windows Vista. This works without security enabled.
Hi Liz. Please also provide the steps for: 1. JUnit Test 2. URL Recording 3. Manual Test This will help me to determine whether it is a problem in the test framework or the RAC. Thanks.
Created attachment 57237 [details] shared memory improvement Hi Liz Could you repeat your tests with the modified AC library hcclsm.dll. Please replace existing file hcclsm.dll in %AC_HOME%\bin folder by the file I attached. The modified library contains improvement for shared memory (bug #165947) and could prevent tests hanging.
It now hangs at 66% before producing a socket write error from the test execution harness. I deleted the old file and replaced it with the new one. Is there anything else I need to do?
Hi Liz. Did you run the RAC from a console window? If so, did any error message get printed out? On my thinkpad there are lots of javax.net.ssl exceptions printed out. I am searching the Sun Java support for similar problem.
Hi Samson, after running it in a console window I noticed the following errors in the window after 66% java.lang.NullPointerException at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:221) at java.lang.ClassLoader.defineClass(ClassLoader.java:158) at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.executeCommand(FileClientHandlerExtendedImpl.java:121) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.run(FileClientHandlerExtendedImpl.java:289) at java.lang.Thread.run(Thread.java:797) java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.executeCommand(FileClientHandlerExtendedImpl.java:142) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.run(FileClientHandlerExtendedImpl.java:289) at java.lang.Thread.run(Thread.java:797) Caused by: java.lang.NullPointerException at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:221) at java.lang.ClassLoader.defineClass(ClassLoader.java:158) at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:221) at java.lang.ClassLoader.defineClass(ClassLoader.java:158) at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at org.eclipse.hyades.internal.execution.core.file.dynamic.FileServerCom mandFactory.createFileServerCommand(FileServerCommandFactory.java:424) ... 7 more java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct orAccessorImpl.java:67) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC onstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:521) at org.eclipse.hyades.internal.execution.core.file.dynamic.FileServerCom mandFactory.createFileServerCommand(FileServerCommandFactory.java:430) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.executeCommand(FileClientHandlerExtendedImpl.java:142) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.run(FileClientHandlerExtendedImpl.java:289) at java.lang.Thread.run(Thread.java:797) Caused by: java.lang.NullPointerException at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:221) at java.lang.ClassLoader.defineClass(ClassLoader.java:158) at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:221) at java.lang.ClassLoader.defineClass(ClassLoader.java:158) at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at org.eclipse.hyades.internal.execution.core.file.dynamic.DetermineServ erReachCommand.<init>(DetermineServerReachCommand.java:218) ... 12 more java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.executeCommand(FileClientHandlerExtendedImpl.java:142) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.run(FileClientHandlerExtendedImpl.java:289) at java.lang.Thread.run(Thread.java:797) Caused by: org.eclipse.hyades.internal.execution.core.file.dynamic.InvalidFileSe rverCommandException at org.eclipse.hyades.internal.execution.core.file.dynamic.FileServerCom mandFactory.createFileServerCommand(FileServerCommandFactory.java:466) ... 7 more java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.executeCommand(FileClientHandlerExtendedImpl.java:142) at org.eclipse.hyades.internal.collection.framework.FileClientHandlerExt endedImpl.run(FileClientHandlerExtendedImpl.java:289) at java.lang.Thread.run(Thread.java:797) Caused by: java.lang.NullPointerException at org.eclipse.hyades.execution.core.loader.ScopedChannelClassLoader$Con sumer$ContextClassLoader.loadClass(ScopedChannelClassLoader.java:334) at java.lang.ClassLoader.loadClass(ClassLoader.java:557) at org.eclipse.hyades.internal.execution.core.file.dynamic.FileServerCom mandFactory.createFileServerCommand(FileServerCommandFactory.java:424) ... 7 more
Note that the AGR test case can be run successfully using Sun JDK 1.5.0_11 with security turned on. Need to investigate further to see if this is indeed a IBM JRE problem.
Today's investigation using a fresh 4.2.1.1 GA TPTP driver (clean workspace and configuration) shows that AGR test cases can be run successfully using either IBM or Sun JRE. I have tried it on my Windows XP laptop as well as my Windows XP desktop and both shows the same result. One thing I have noticed is that the CPU goes really high when the test artifacts are being transferred. This may due the a known problem when transferring files over a SSL connection.
I have also found it works with 4.2.1.1 on Windows XP. However, with the 4.2.2 candidate driver I am still unable to get AGR test suites running with security on Windows XP or Vista. They run fine without security. Are you able to run with 4.2.2, security enabled? Are there any changes from 4.2.1.1 that you think could be causing this?
I have also found that on EM64T running a TPTP client on a 32 bit JVM (IBM 5) and the RAC on a 64 bit (SUN 5) would cause the AGR to hang at 60%. Not sure if this is a supported scenario but just wanted to record this finding.
Given all the random behaviours we have seen, I can somewhat conclude that the problem lies in the file server code. The reason why it only happens when security is enabled is because Java NIO does not work properly with SSL - the former requires non-blocking I/O while the latter is blocking. After reading many web discussion about this problem the only way to make NIO and SSL to work is to use JDK 1.5 and rewriting the code using the new transport abstraction - the SSLEngine class. Since I have no knowledge on the file server code I think the effort of having this fixed is much larger than I originally thought.
Reset target to 4.4 i1.
Cannot contain in i1.
Set priority to P1 for 4.4 plan closure.
Samson, I was able to intermittently run these in 4.3.1 (with security enabled). However, with 4.4 its back to the consistent failures. Not sure how relevant this will be to your investigation but I thought I should mention it just in case.
Not containable in 4.4.
(In reply to comment #21) > Not containable in 4.4. How can we defer a defect that is blocking Test Tool function for a important use case (AGR test execution using a secure Agent Controller)? We need to reconsider for 4.4.
Samson, please explain why this had to be deferred. Thanks.
Hi Joanna. This has to be deferred because the problem is caused by a limitation in the secured file server limitation. The current implementation uses Java NIO on top of SSL based on the JDK 1.4 specification. This has known problems (see my comment #16). The effort or rewriting the secured file server cannot be contained since the original owner has left the project. I have estimated an 80 hours of work for this if I were to do this myself. I am expecting a compatibility issue since the recommended way of implementation (using the SSLEngine class) may not be compatible with older TPTP workbench.
Samson, can you summarize the platforms where this problem occurs ? It seems to be an exception caused by a change after 4.2.1.1 and it seems to affect more than just Vista. Most of the consuming products are running with security on so this may end up being a blocking issue for upstream consumers. We'll have to understand the implications before moving the defect to future
This problem is not AGR specific and will appears on any platform when large amount of data is being transferred through the secured connection. The AGR is hit the most since it needs to transfer lots of files and some are relatively big workbench jars.
Note that this is intermittent on Windows ( see above comment on working in 4.3.1, back to failures in 4.4 intermittently).
(In reply to comment #26) > This problem is not AGR specific and will appears on any platform when large > amount of data is being transferred through the secured connection. The AGR is > hit the most since it needs to transfer lots of files and some are relatively > big workbench jars. So this sounds like it is not a regression, and may be affecting any user of file transfer. Yet no-one has yet complained. Do I understand this correctly?
There is an IBM consumer that is running into this problem. They are planning to ship the TPTP 4.2.2 RAC, so if you can provide a fix or workaround in that stream, that would be great. Thanks.
(In reply to comment #29) > There is an IBM consumer that is running into this problem. They are planning > to ship the TPTP 4.2.2 RAC, so if you can provide a fix or workaround in that > stream, that would be great. > Thanks. It would be helpful to get an answer to my earlier question. I doubt anyone is shipping" the AGR or AGR tests. If there is a reproducable test that is related to the GA functions, it would be useful.
From what I can tell this is a regression because of reports of being able to run with security in pre-4.2.2 releases. That said, it could also be an intermittent problem. I agree that its unlikely they are shipping AGR or AGR tests. I have not heard any other complaints from users about this, again perhaps because it is intermittent and/or there is a better test case. The test case I am running in each test pass turns on security in the RAC, starts the RAC and deploys a simple AGR test suite with one test case in it. Since finding this defect I have run others (more dependent files, larger test suites, IAC, RAC etc.) without any good data on what is producing this resulting.
Hi Bing. I have transferred my bugs to you for triage. Thanks.
In TPTP 4.5, the AGR was moved from a Technology Preview component to an As-Is component. As-Is components are primarily provided for prior users but imply no support (for example, defects, news group, and mailing lists) or commitment to triage or resolve opened defects. As such, we no longer require this defect.
This defect has been shown to not be AGR specific and the change in AGR status does not remove the need for this fix. The text here indicates a much more common and lower level problem. Paul, I believe it should be the component owner or project lead that would mark a defect as wontfix.
(In reply to comment #34) > This defect has been shown to not be AGR specific and the change in AGR status > does not remove the need for this fix. The text here indicates a much more > common and lower level problem. > > Paul, I believe it should be the component owner or project lead that would > mark a defect as wontfix. > I returned this defect since we have not seen this symptom with any other test type and there are no consumers requesting this fix. Given your concerns and the potential for this problem to be reproduced when transferring large files to the target machine, I will reopen this defect. I returned this defect in place of Liz since she is no longer working on the project.
Jonathan, it's related to security code on native side. Can you take a look.
Liz, any objections to lowering the severity to major?
Deferral to future with PMC approval
Mass update of P1 enhancements and defects targetted to future to P2.
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. Since this defect is more than 2 years old, it may be no longer relevant. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this defect is resolved as WONTFIX. If this defect is still relevant and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since the originator of this enhancement/defect has an inactive Bugzilla account and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.
