Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 312131 - ws performance test JUnits can have inconsistent results due to background jobs running
Summary: ws performance test JUnits can have inconsistent results due to background jo...
Status: CLOSED FIXED
Alias: None
Product: WTP Webservices
Classification: WebTools
Component: jst.ws (show other bugs)
Version: 3.2   Edit
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: 3.2 RC1   Edit
Assignee: Mark Hutchinson CLA
QA Contact: Keith Chong CLA
URL:
Whiteboard: PMC_approved
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-07 16:45 EDT by Mark Hutchinson CLA
Modified: 2010-06-17 14:43 EDT (History)
2 users (show)

See Also:
keith.chong.ca: pmc_approved? (david_williams)
keith.chong.ca: pmc_approved? (raghunathan.srinivasan)
keith.chong.ca: pmc_approved? (naci.dai)
keith.chong.ca: pmc_approved? (deboer)
neil.hauge: pmc_approved+
keith.chong.ca: pmc_approved? (kaloyan)
keith.chong.ca: review+

Attachments
proposed patch (10.13 KB, patch)
2010-05-07 16:55 EDT, Mark Hutchinson CLA 		keith.chong.ca: iplog+
Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Hutchinson CLA 2010-05-07 16:45:23 EDT 
We should try to wait for all background jobs to finish before we start running these tests:
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrBUJavaAxisTC50
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrClientAxisTC50
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrTDJavaAxisTC50
Comment 1 Mark Hutchinson CLA 2010-05-07 16:55:06 EDT 
Created attachment 167576 [details]
proposed patch
Comment 2 Mark Hutchinson CLA 2010-05-11 11:49:35 EDT 
Explain why you believe this is a stop-ship defect. Or, if it is a "hotbug" (requested by an adopter) please document it as such. 
This fix is for the JUnit performance test cases for the axis web services wizards.  Background jobs that have not completed before we start timing can make the performance results quite variable and confound what we are trying to measure.

Is there a work-around? If so, why do you believe the work-around is insufficient? 
There is no workaround.

How has the fix been tested? Is there a test case attached to the bugzilla record? Has a JUnit Test been added? 
This performance JUnit test has been run, and this fix makes the results more consistent.

Give a brief technical overview. Who has reviewed this fix? 
Try to wait for all background jobs to complete before we start timing the test.  Give up waiting after approximately 2 minutes.

What is the risk associated with this fix? 
No risk.  JUnit test case only.
Comment 3 Keith Chong CLA 2010-05-12 12:12:52 EDT 
Patch looks good. We want consistency in our measurements.  This affects only tests and it definitely is a worthwhile change to put into 3.2.

Submitting for PMC approval.
Comment 4 Keith Chong CLA 2010-05-13 11:37:50 EDT 
Resolving bug.  Fix was released for this week's I-build.
Comment 5 Mark Hutchinson CLA 2010-06-17 14:43:58 EDT 
Closing