Community
Participate
Working Groups
3.6 M1 (but is a general issue). Bug 323439 in our performance test infrastructure revealed that we run slower on 1.6 than on 1.5 (blocked bugs for details). We need to profile some of those scenarios to see whether they have a common root cause and whether there is something we could do about (change vm arguments, use different APIs, etc.). Interesting part is that it happens on Linux and Windows.
Frédéric, can you please do that investigation for us.
*** Bug 323018 has been marked as a duplicate of this bug. ***
*** Bug 323022 has been marked as a duplicate of this bug. ***
*** Bug 323010 has been marked as a duplicate of this bug. ***
*** Bug 323009 has been marked as a duplicate of this bug. ***
> (blocked bugs for details). See duplicates.
(In reply to comment #1) > Frédéric, can you please do that investigation for us. Sure
Hmmm, really weird... Doing the same test on my Linux box, I got the opposite result: the JDT/Core tests are faster using 1.6 than 1.5!? (BTW, not so weird as I would say it should have been the expected results... :-S) I'll redo all my tests to verify that I did not make any mistake while running them as I definitely want to be sure of these results... Kim, A question while waiting for the new results: was the switch from 1.5 to 1.6 the only change you made on those machine or were there any other ones?
Very strange, since performance was supposedly one of the big goals in Java 6 development for HotSpot: http://java.sun.com/performance/reference/whitepapers/6_performance.html
It can also be a memory issue, e.g. with our current memory VM args it might trigger more GCs on 1.6. Frédéric, are you sure you used the same VM args as our test scripts do? Kim, are they the same on HEAD and the baseline?
See bug 321142 for the patches applied to the builder while changing from 1.5 to 6.0...
regarding comment #8 Yes, the switch to the new VMs was made on August 4, 2010. See bug 321142 for details. The new perf database machine started being used August 6, 2010 but this shouldn't impact the measurements in the tests themselves. Bug 321124.
I have interesting results while running manually the JDT/Core Type Hierarchy test FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes() on a Linux and Windows machines with the different JVM setup... Windows (Vista): run1 run2 run3 mean jvm 1.5.0_14: 2.07s 2.06s 2.06s 2.06s jvm 6.0_04: 1.92s 1.89s 1.89s 1.90s => +7.68% (faster) jvm 6.0_17: 2.42s 2.38s 2.38s 2.39s => -16.18% (slower) Linux (RedHat 5.5): run1 run2 run3 mean jvm 1.5.0_14: 456ms 482ms 439ms 459ms jvm 6.0_04: 518ms 509ms 530ms 519ms => -13.07% (slower) jvm 6.0_17: 516ms 562ms 531ms 536ms => -16.78% (slower) Note that my initial test (comment 8) was made with different vm builds... So, it seems that the new used JDK6 version has some performance troubles. I'll come shortly with similar tests using JDK 6.0_20 and JDK 6.0_21 to see if they still have this perf issue...
Here are the complete results including more recent JDK 1.5.0 and 6.0 builds for my Linux machine: Linux (RedHat 5.5): run1 run2 run3 mean jvm 1.5.0_14: 456ms 482ms 439ms 459ms jvm 1.5.0_22: 440ms 404ms 415ms 420ms => +8.50% (faster) jvm 6.0_04: 518ms 509ms 530ms 519ms => -13.07% (slower) jvm 6.0_17: 516ms 562ms 531ms 536ms => -16.78% (slower) jvm 6.0_20: 565ms 527ms 559ms 536ms => -16.78% (slower) jvm 6.0_21: 485ms 524ms 507ms 505ms => -10.02% (slower) I finally gave up to dump the results on the Windows machine as the dispersion is really too high to make the numbers meaningful with so less runs... E.g. for the 1.5.0_14 jvm build, I got numbers between 1.42s and 5.01s!!! Note that on releng perf machines, there were several JDKs updates this year: 1) on 6.0 machines, the JDK was upgraded from 6.0_04 to 6.0_17 on 2010/01/14 2) on 1.5.0 machines, the JDK was upgraded: a) from 1.5.0_14 to 1.5.0_22 on 2010/01/14 b) from 1.5.0_22 to 6.0_17 on 2010/07/28 These numbers would confirm the observed regression which is around 30% on the test. As the baseline uses a JDK 1.5.0_22 and HEAD a JDK 6.0_17, we get respectively 420ms and 519 which makes a regression of 23.57%... I continue to test all these different jvm builds but this time with doing 20 runs which will allow me to smooth the numbers and then get a more significant result.
1.6 VM are checking stack maps so this can take a couple of milliseconds.
Satyam, this is not high prio and can also be done even after 3.7 ships as most likely even if we know the cause there won't be much we could do at this stage. We should then also compare with 1.7.
Now that Java 1.8 is required I suggest closing this bug and opening another one when someone has real data for comparison.
