Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 157300

Summary: Test Log Viewer's Verdict Summary includes arbitrated verdict.
Product: z_Archived Reporter: Paul Slauenwhite <paulslau>
Component: TPTPAssignee: Bianca Jiang <bjiang>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P1 CC: paulslau, sleeloy
Version: unspecifiedKeywords: plan
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
Execution History
none
Report Screen Capture
none
Fix patch file for org.eclipse.hyades.test.ui plugin none

Description Paul Slauenwhite CLA 2006-09-14 08:29:11 EDT 
Test Log Viewer's Verdict Summary includes arbitrated verdict.

Using the TPTP-4.2.1-200609071820A driver (including the BIRT dependencies), the enclosed execution history with four verdicts (pass, fail, error and inconclusive) viewed in the Test Log Viewer renders the enclosed Verdict Summary:

Pass: 20%
Error: 40%
Fail: 20%
Inconclusive: 20%
Comment 1 Paul Slauenwhite CLA 2006-09-14 08:29:47 EDT 
Created attachment 50149 [details]
Execution History
Comment 2 Paul Slauenwhite CLA 2006-09-14 08:30:39 EDT 
Created attachment 50150 [details]
Report Screen Capture
Comment 3 Paul Slauenwhite CLA 2006-09-14 08:51:58 EDT 
Also, the Verdict List contains the arbitrated verdict which seems counterintuitive since the arbitrated verdict is in the Common Properties.
Comment 4 Bianca Jiang CLA 2006-09-14 11:29:09 EDT 
The pie chart is just a visual representation of the generated verdict events in the test log. TPTP test runners right now generate a final verdict event with the rollup value from all the verdicts happened during execution. And the generation of this verdict is specific to each test runners.

So the question is whether the test runners should use an extra verdict for the arbitration result.
Comment 5 Paul Slauenwhite CLA 2006-09-22 11:19:09 EDT 
There is also an inconsistency issue we need to address between the Test Log viewer's BIRT graphic and the TPTP BIRT reports (reporting, for example, 4 x 25%).
Comment 6 Paul Slauenwhite CLA 2007-01-17 09:24:23 EST 
Sizing updated as provided by Bianaca.
Comment 7 Paul Slauenwhite CLA 2007-01-22 15:11:21 EST 
Increasing priority and targeting to 4.4 due to a user request.
Comment 8 Paul Slauenwhite CLA 2007-03-23 16:13:22 EDT 
Assigning target.
Comment 9 Bianca Jiang CLA 2007-03-30 12:39:12 EDT 
Created attachment 62518 [details]
Fix patch file for org.eclipse.hyades.test.ui plugin

The fix is to implement verdict providers for extension point "executionHistoryExtension.verdictProvider" for TPTP test types:
"org.eclipse.hyades.test.http.junit.testSuite"
"org.eclipse.hyades.test.java.junit.testSuite"
"org.eclipse.hyades.test.manual.testSuite"
to handle the extra rolled up verdict event in TPTP test logs.
Comment 10 Bianca Jiang CLA 2007-04-25 19:14:19 EDT 
Fix checked in.
Comment 11 Bianca Jiang CLA 2007-04-25 19:24:06 EDT 
update hours
Comment 12 Paul Slauenwhite CLA 2007-06-02 14:52:11 EDT 
Reporter: Please verify and close in preparation for shutting down the TPTP 4.4 release.  Thanks.
Comment 13 Paul Slauenwhite CLA 2007-06-27 14:56:21 EDT 
Verified in TPTP-4.4.0-200706140100C.
Comment 14 Paul Slauenwhite CLA 2009-11-23 10:38:31 EST 
Correcting component.