Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 354544 - PERF: 20110722 vs. 20110805, the update tests in JPAEmulatedIsolatedPerformanceRegressionModel failed
Summary: PERF: 20110722 vs. 20110805, the update tests in JPAEmulatedIsolatedPerforman...
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Eclipselink (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: ---   Edit
Assignee: Nobody - feel free to take it CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-11 15:32 EDT by Edwin Tang CLA
Modified: 2022-06-09 10:20 EDT (History)
2 users (show)

See Also:


Attachments
patch (239.99 KB, patch)
2011-09-01 09:37 EDT, James Sutherland CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Edwin Tang CLA 2011-08-11 15:32:39 EDT
2.4.0.v20110722-r9777 vs. 2.4.0.v20110805-r9842, the update tests in JPAEmulatedIsolatedPerformanceRegressionModel failed.

Error Message:
==============
     [java] 	VERSION: Eclipse Persistence Services - 2.4.0.v20110805-r9842
     [java] 	TEST SUITE NAME: JPAWritingTestSuite
     [java] 	SUITE DESCRIPTION: This suite tests uow/writing performance.
     [java] 
     [java] 		VERSION:   Eclipse Persistence Services - 2.4.0.v20110805-r9842
     [java] 		TEST NAME:   JPAUpdateAddressPerformanceComparisonTest
     [java] 		TEST DESCRIPTION: This test compares the performance of update Address.
     [java] 		##FAILURE##
     [java] 		RUNS:					5
     [java] 
     [java] 		BASE_LINE TEST
     [java] 		TEST: JPAUpdateAddressPerformanceComparisonTest
     [java] 		TEST RUN TIME:				10 seconds
     [java] 		MEAN TEST COUNT:			384927.0
     [java] 		MAX TEST COUNT:				392546
     [java] 		MIN TEST COUNT:				293163
     [java] 		TEST % STANDARD DEVIATION:		0.9789812427529034
     [java] 		BASELINE VERSION:			2.4.0 :20110722
     [java] 		BASELINE VERSION RESULTS:		[470888.0, 470398.0, 496875.0, 467184.0, 471854.0]
     [java] 		BASELINE VERSION % STANDARD DEVIATION:	0.1284172413692
     [java] 		CURRENT VERSION RESULTS:		[384927.0, 385964.0, 423820.0, 412477.0, 411104.0]
     [java] 		CURRENT VERSION % STANDARD DEVIATION:	3.022861981457712
     [java] 		% DIFFERENCE (last run):		-22.331766802536585
     [java] 		% DIFFERENCE (average):			-16.832362582624043
     [java] 
     [java] 		RESULT:      				Error
     [java] 		Exception [EclipseLink-0] (Eclipse Persistence Services - 2.4.0.v20110805-r9842): org.eclipse.persistence.testing.framework.TestErrorException
     [java] 		Exception Description: Test is -16% slower than last successful execution.
     [java] 
     [java] 		VERSION:   Eclipse Persistence Services - 2.4.0.v20110805-r9842
     [java] 		TEST NAME:   JPAUpdateEmployeePerformanceComparisonTest
     [java] 		TEST DESCRIPTION: This test compares the performance of update Employee.
     [java] 		##FAILURE##
     [java] 		RUNS:					5
     [java] 
     [java] 		BASE_LINE TEST
     [java] 		TEST: JPAUpdateEmployeePerformanceComparisonTest
     [java] 		TEST RUN TIME:				10 seconds
     [java] 		MEAN TEST COUNT:			215045.33333333334
     [java] 		MAX TEST COUNT:				216887
     [java] 		MIN TEST COUNT:				184048
     [java] 		TEST % STANDARD DEVIATION:		0.10405422933413866
     [java] 		BASELINE VERSION:			2.4.0 :20110722
     [java] 		BASELINE VERSION RESULTS:		[249647.0, 261961.0, 258654.0, 253249.0, 253246.0]
     [java] 		BASELINE VERSION % STANDARD DEVIATION:	0.9992755016069953
     [java] 		CURRENT VERSION RESULTS:		[215045.0, 217872.0, 230940.0, 225514.0, 228825.0]
     [java] 		CURRENT VERSION % STANDARD DEVIATION:	2.0469411521211898
     [java] 		% DIFFERENCE (last run):		-16.090585691366925
     [java] 		% DIFFERENCE (average):			-13.825718412819777
     [java] 
     [java] 		RESULT:      				Error
     [java] 		Exception [EclipseLink-0] (Eclipse Persistence Services - 2.4.0.v20110805-r9842): org.eclipse.persistence.testing.framework.TestErrorException
     [java] 		Exception Description: Test is -13% slower than last successful execution.
Comment 1 James Sutherland CLA 2011-09-01 09:37:44 EDT
Created attachment 202604 [details]
patch
Comment 2 James Sutherland CLA 2011-09-01 09:41:58 EDT
svn trunk commit: Bug#354544 update perf regression

https://bugs.eclipse.org/bugs/show_bug.cgi?id=354544

Our update isolated performance regression tests were failing due to an error in descriptor cache policy initialization.

Changes:
- fixed cache isolation flags on projects to be set correctly during descriptor initialization
Comment 3 James Sutherland CLA 2011-11-07 10:29:55 EST
this was resolved
Comment 4 Eclipse Webmaster CLA 2022-06-09 10:20:48 EDT
The Eclipselink project has moved to Github: https://github.com/eclipse-ee4j/eclipselink