Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 312131

Summary: ws performance test JUnits can have inconsistent results due to background jobs running
Product: [WebTools] WTP Webservices Reporter: Mark Hutchinson <mahutch>
Component: jst.wsAssignee: Mark Hutchinson <mahutch>
Status: CLOSED FIXED QA Contact: Keith Chong <keith.chong.ca>
Severity: normal    
Priority: P2 CC: keith.chong.ca, neil.hauge
Version: 3.2Flags: keith.chong.ca: pmc_approved? (david_williams)
keith.chong.ca: pmc_approved? (raghunathan.srinivasan)
keith.chong.ca: pmc_approved? (naci.dai)
keith.chong.ca: pmc_approved? (deboer)
neil.hauge: pmc_approved+
keith.chong.ca: pmc_approved? (kaloyan)
keith.chong.ca: review+
Target Milestone: 3.2 RC1   
Hardware: PC   
OS: Windows XP   
Whiteboard: PMC_approved
Attachments:
Description Flags
proposed patch keith.chong.ca: iplog+

Description Mark Hutchinson CLA 2010-05-07 16:45:23 EDT 
We should try to wait for all background jobs to finish before we start running these tests:
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrBUJavaAxisTC50
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrClientAxisTC50
org.eclipse.jst.ws.tests.axis.tomcat.v50.perfmsr.PerfmsrTDJavaAxisTC50
Comment 1 Mark Hutchinson CLA 2010-05-07 16:55:06 EDT 
Created attachment 167576 [details]
proposed patch
Comment 2 Mark Hutchinson CLA 2010-05-11 11:49:35 EDT 
Explain why you believe this is a stop-ship defect. Or, if it is a "hotbug" (requested by an adopter) please document it as such. 
This fix is for the JUnit performance test cases for the axis web services wizards.  Background jobs that have not completed before we start timing can make the performance results quite variable and confound what we are trying to measure.

Is there a work-around? If so, why do you believe the work-around is insufficient? 
There is no workaround.

How has the fix been tested? Is there a test case attached to the bugzilla record? Has a JUnit Test been added? 
This performance JUnit test has been run, and this fix makes the results more consistent.

Give a brief technical overview. Who has reviewed this fix? 
Try to wait for all background jobs to complete before we start timing the test.  Give up waiting after approximately 2 minutes.

What is the risk associated with this fix? 
No risk.  JUnit test case only.
Comment 3 Keith Chong CLA 2010-05-12 12:12:52 EDT 
Patch looks good. We want consistency in our measurements.  This affects only tests and it definitely is a worthwhile change to put into 3.2.

Submitting for PMC approval.
Comment 4 Keith Chong CLA 2010-05-13 11:37:50 EDT 
Resolving bug.  Fix was released for this week's I-build.
Comment 5 Mark Hutchinson CLA 2010-06-17 14:43:58 EDT 
Closing