Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 343297

Summary:	Hardware/Software differences between epwin2 and epwin3 and effects on performance tests
Product:	[Eclipse Project] Platform	Reporter:	Oleg Besedin <ob1.eclipse>
Component:	Releng	Assignee:	Kim Moir <kim.moir>
Status:	RESOLVED WONTFIX	QA Contact:
Severity:	normal
Priority:	P3	CC:	bokowski, daniel_megert, kim.moir, ob1.eclipse
Version:	3.7
Target Milestone:	---
Hardware:	PC
OS:	Windows 7
Whiteboard:

Description Oleg Besedin

2011-04-19 14:03:46 EDT

What exactly are the differences in hardware and software between Windows performance test machines epwin2 and epwin3?

For most performance tests CPU & Elapsed timing between those machines are very close and within normal test variability.

Some test however, show puzzling differences with epwin3 being 2 to 10 *times* slower then epwin2:

- 10 times slower: UI's OpenCloseViewTest#showView:BookmarkView()
- 3 times slower:  ANT's SeparateVMTests#testBuildWithLotsOfLinks() 
- 2 times slower: SWT's Draw strings using GC.drawString()
... and many more.

From the descriptions, test machines are supposed to be identical. Both supposed to be WinXP 2 x 3.00GHz - 3GB RAM, it is unclear if both machines use Sun 1.6.0_17 or one uses Sun 1.5?

This needs to be cleared up as we are spending significant amount of effort based on those performance numbers.

Comment 1 Oleg Besedin

2011-04-19 14:08:23 EDT

(And, by the way, if two machines are identical why we are running tests on both?)

Suggestions on actions:

(a) Decommission epwin3 and forget about it
(b) Run Windows benchmark software on both machines and compare results  
(c) Get a complete list of hardware pieces on both machines and see if there are differences
(d) Run whatever tests are available for both machines to ascertain that epwin3 does not have a hardware problem

Comment 2 Boris Bokowski

2011-04-19 15:06:46 EDT

Is the graphics card the same on both machines?

Comment 3 Kim Moir

2011-04-19 15:14:31 EDT

As far as I know they are the same hardware.  I'll ask Karl (our sysadmin) if the patch level is the same.  They both run the tests on the same VMs.

The reason that we have two machines is that Frederic wanted two as a sanity check. If one machine had strange results, you could look to the other machine. I agree that the test results are inconsistent and it wastes a time.  As an aside, he rebooted both machines yesterday and applied patches so the machines should (in theory :-) be in a consistent state for today's build.

Comment 4 Dani Megert

2011-04-20 02:12:03 EDT

Looking at the Java system properties there is a difference: on 'epwin2' the java.library.path has this additional entry: "C:\\buildtest\\jdk6_04\\bin" at the end.

No idea whether this affects the result or not.

Comment 5 Oleg Besedin

2011-04-20 15:07:10 EDT

What was the last time clean images had been installed on those machines? Is ti possible to re-image them in time for M7 milestone week? If not, RC1?

Comment 6 Kim Moir

2011-04-20 15:26:07 EDT

We don't reimage the machines anymore.  IT doesn't provide the facilities for us to reimage the machines anymore.

Comment 7 Kim Moir

2012-03-23 15:05:54 EDT

Doesn't apply anymore.  We are moving the build to eclipse.org hardware.