Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 314806 - Test execution time is inconsistent
Summary: Test execution time is inconsistent
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: TPTP (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: ---   Edit
Assignee: Paul Slauenwhite CLA
QA Contact: Kathy Chan CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-28 02:29 EDT by Ben Chen CLA
Modified: 2016-05-05 10:58 EDT (History)
3 users (show)

See Also:
paulslau: pmc_approved? (oec)
paulslau: pmc_approved? (ernest)
kathy: pmc_approved+
paulslau: pmc_approved? (paulslau)
ewchan: pmc_approved+
paulslau: pmc_approved? (jgwest)
jerome.bozier: review+


Attachments
Patch. (13.17 KB, patch)
2010-05-28 08:52 EDT, Paul Slauenwhite CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Chen CLA 2010-05-28 02:29:56 EDT
Build Identifier: TPTP-4.7.0-201005251601

If user creates a new JUnit/JUnit Plugin test, add 2 test methods and invocations to it, add a Thread.sleep(5000l); to the first test method code and run the test, upon finishing, the start time and the end time appear to be the same on the test log. but if user switches the invocations around(second test method runs first) and run it again, the start/end time have a 5 sec difference.

Reproducible: Always
Comment 1 Paul Slauenwhite CLA 2010-05-28 06:10:42 EDT
This is caused by test log view deriving the start and stop dates/times from the first the last verdict events.  In this case, the first verdict event is generated after the first test case is fully executed, that is after the sleep.  

The fix is to derive the  start and stop dates/times from the first and last events, irrespective of their type.
Comment 2 Paul Slauenwhite CLA 2010-05-28 06:16:58 EDT
Work-around: See the time stamps of the first and last events or the message in the last stop event for the total execution time.
Comment 3 Paul Slauenwhite CLA 2010-05-28 08:52:08 EDT
Created attachment 170334 [details]
Patch.
Comment 4 Paul Slauenwhite CLA 2010-05-28 08:52:32 EDT
Jerome, please review the attached patch.
Comment 5 Paul Slauenwhite CLA 2010-05-28 08:55:11 EDT
Added two new test cases to the following test suites:

/org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite

/org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite
Comment 6 Bozier jerome CLA 2010-05-28 09:05:59 EDT
Patch looks good
Comment 7 Paul Slauenwhite CLA 2010-05-28 09:06:37 EDT
Requesting PMC approval for TPTP 4.7.0: 

  1.   Explain why you believe this is a stop-ship defect. How does the defect manifest itself, and how will users of TPTP / consuming products be affected if the defect is not fixed?

The start and stop time stamps in the Test Log view are incorrect.

   2. Is there a work-around? If so, why do you believe the work-around is insufficient?

No.

   3. Is this a regression or API breakage? Explain.

No.

   4. Does this require new API?

No.

   5. Who performed the code review?

Jerome Bozier

   6. Is there a test case attached to the bugzilla record?

Added two new test cases to the following test suites:

/org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite

/org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite

   7. What is the nature of the fix? What is the scope of the fix? What is the risk associated with this fix?

This is caused by test log view deriving the start and stop dates/times from
the first the last verdict events.  In this case, the first verdict event is
generated after the first test case is fully executed, that is after the sleep. 

The fix is to derive the  start and stop dates/times from the first and last
events, irrespective of their type.

   8. Is this fix related to any standards that TPTP adheres to? If so, who has validated that the fix continues to adhere to the standard?

No.
Comment 8 Paul Slauenwhite CLA 2010-05-28 10:16:31 EDT
The attached patch and test suites are checked in to CVS (HEAD).
Comment 9 Ben Chen CLA 2010-06-01 14:56:22 EDT
verified in TPTP-4.7.0-201006011014
Comment 10 Ben Chen CLA 2010-06-01 14:56:33 EDT
(In reply to comment #9)
> verified in TPTP-4.7.0-201006011014

Closing