Bugzilla – Full Text Bug Listing
| Summary: | Serviceability: Improve error handling and reporting during test execution. | ||
|---|---|---|---|
| Product: | z_Archived | Reporter: | Paul Slauenwhite <paulslau> |
| Component: | TPTP | Assignee: | Paul Slauenwhite <paulslau> |
| Status: | CLOSED WONTFIX | QA Contact: | |
| Severity: | major | ||
| Priority: | P2 | CC: | chenbs, scott.schneider |
| Version: | unspecified | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | housecleaned462 | ||
| Bug Depends on: | |||
| Bug Blocks: | 194503 | ||
Joe, can you provide a sizing when time permits. Noteworthy, several of the test runners call org.eclipse.hyades.internal.execution.remote.RemoteComponentSkeleton.sendMessageToAttachedClient(String, long) to log status and exception cases. However, these messages are not consumed by the test harness since the org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.getAgentListener().new AgentListener() {...}.handleCommand(Agent, CommandElement) method body is empty.
*** Bug 220543 has been marked as a duplicate of this bug. *** *** Bug 185363 has been marked as a duplicate of this bug. *** This defect is a general defect to provide improved error handling and reporting during test execution in the execution harness and test runners. See defect #171288 for a general defect to provide Common Base Event/IStatus logging in the Test UI, execution harness, and test runners. In addition: [Defect #220543]: Error reporting does not provide the cause for a failed test execution. For example, when launching a JUnit Plug-in test suite (via ASF) without a valid Workbench Location, a InactiveProcessException exception is thrown: [tptp:execution] adding file BVT/AllBVTJUnitPluginTests.testsuite [tptp:execution] Test started! [tptp:execution] Test 'BVT/AllBVTJUnitPluginTests.testsuite' executing [tptp:execution] org.eclipse.hyades.internal.execution.local.control.InactiveProcessException [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.launch(TestExecutionHarnessExecutorStub.java:422) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness.launchTestExecution(TestExecutionHarness.java:2098) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness.access$2(TestExecutionHarness.java:1830) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness$2.run(TestExecutionHarness.java:750) [tptp:execution] at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) [tptp:execution] java.lang.RuntimeException: org.eclipse.hyades.internal.execution.local.control.InactiveProcessException [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.handleException(TestExecutionHarnessExecutorStub.java:347) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.launch(TestExecutionHarnessExecutorStub.java:434) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness.launchTestExecution(TestExecutionHarness.java:2098) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness.access$2(TestExecutionHarness.java:1830) [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarness$2.run(TestExecutionHarness.java:750) [tptp:execution] at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) [tptp:execution] Caused by: org.eclipse.hyades.internal.execution.local.control.InactiveProcessException [tptp:execution] at org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.launch(TestExecutionHarnessExecutorStub.java:422) [tptp:execution] ... 4 more [tptp:execution] org.eclipse.core.runtime.CoreException: IWAT4039E The Test Execution Harness reported the following problems:org.eclipse.hyades.internal.execution.local.control.InactiveProcessException [tptp:execution] at org.eclipse.hyades.test.core.launch.delegates.AbstractLaunchConfigurationDelegate2.launch(AbstractLaunchConfigurationDelegate2.java:195) [tptp:execution] at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:764) [tptp:execution] at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:614) [tptp:execution] at org.eclipse.debug.internal.core.LaunchConfiguration.launch(LaunchConfiguration.java:607) [tptp:execution] at org.eclipse.hyades.test.core.services.AbstractTestExecutionService.launch(AbstractTestExecutionService.java:797) [tptp:execution] at org.eclipse.hyades.test.core.services.AbstractTestExecutionService.execute(AbstractTestExecutionService.java:585) [tptp:execution] at org.eclipse.hyades.automation.server.AbstractService.execute(AbstractService.java:43) [tptp:execution] at org.eclipse.hyades.automation.server.ServiceProxy$1.run(ServiceProxy.java:141) [tptp:execution] at org.eclipse.hyades.automation.core.utils.ReusableThread.run(ReusableThread.java:131) [tptp:execution] at java.lang.Thread.run(Thread.java:801) BUILD FAILED /home/build/BVT/scripts/build.xml:28: java.lang.NullPointerException The test runner should do a better job at capturing the root cause of the failed test execution and reporting it back to the test harness. In addition, when test runners launch Eclipse applications, the fatal errors recorded in the .log file should be captured and returned to the user. [Defect #211091]: lack of error message when remote execution fail Steps To Reproduce: it would be interesting to log what's really fail during a remote execution at the moment, there is no simple way to get back the following informations : . command line used . remote error log . if needed plugin or feature are not installed on remote machine (of if their version is not the good one) . if local and remote machine don't run with the same JVM . if deployed bundle fail to load for any reason etc... there is lots of things that can happends, and no real way to know what really happened except debugging and a deep knowledge of the deployment mechanism [Defect #185363]: Improve error condition handling in test test execution harness. The test execution harness does not limit the maximum time allowed for operations. For example, when executing a test, the test execution harness executes blocks of code with a limit to the amount of time to execute the block of code. As a result, the user is not made aware of stalled operations. Instead, stalled operations appear as a hang of the test execution. The user should be made aware of stalled operations after a period of time so they can manually cancel the test execution. For instance: block A check if user clicked cancel block B check if user clicked cancel block C . . . We should have a time-out for each of these blocks, and inform the user what possible things have gone wrong and what steps they can take to resolve it. *** Bug 78260 has been marked as a duplicate of this bug. *** In addition: [Defect #78260]: [Regression] Lack of Exception Handling in Deployment (JavaExecutionDeploymentAdapter), causes failure in Deployment to go unnoticed Since this defect requires new strings requiring translation, deferring to the next release of TPTP that will be translated (4.5.3). As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. Since this defect is more than 2 years old, it may be no longer relevant. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this defect is resolved as WONTFIX. If this defect is still relevant and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open. Reopening since still relevant. *** Bug 258949 has been marked as a duplicate of this bug. *** See https://bugs.eclipse.org/bugs/show_bug.cgi?id=258949 for a new required error message. As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. Since this defect is more than 2 years old, it may be no longer relevant. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this defect is resolved as WONTFIX. If this defect is still relevant and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open. Closing. *** Bug 258949 has been marked as a duplicate of this bug. *** |
Test runner error messages are not displayed to the user. When launching a test via the TPTP workbench, errors originating from the test harness are displayed to the user in a dialog. However, errors originating from the test runner (opposed to the test code or system under test) are not displayed to the user. Currently, these errors are logged as message events through the Component Test Agent and captured in the test log. Although this exception handling is suffice for capturing the necessary error information from the test execution, the user should be made aware of the error in the test runner as they are with any errors in the test harness. One possible solution: 1) Provide an API in the org.eclipse.hyades.test.common.agent.ComptestAgent class to log errors originating from test runners. This API will delegate to org.eclipse.hyades.internal.execution.remote.RemoteComponentSkeleton.logErrMsg(String, String, int). 2) Update all of the test runners to log errors using this API. 3) Handle and display an error dialog in the org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub.getAgentListener().new AgentListener() {...}.error(Agent, String, String) method body.