Community
Participate
Working Groups
Actually, they are running in today's I-build! But 1) completely forgot I need to also run them with the base line code too! And 2) in the end, the results will all be "merged together" on one "performance results" page, though the "short running" results should be available 4 to 7 hours after the build finished (that's running the "short runnings tests" against baseline, and the current build) and then the "long running tests" should be available 8 or 12 hours after that. So -- the point is -- I should have some indication of "long versus short". Probably, where I now say View the performance test results for the current build. ... and, hey, you can see them right now, at http://download.eclipse.org/eclipse/downloads/drops4/I20141118-0830/performance.php I will say View the (initial) performance test results for the current build. and then once long running ones are done, change that to just say simply View the performance test results for the current build.
(Oh, and in case not obvious ... pretty sure I've said this before ... at least for now plan to run the "short running" tests for every build (I and N) but the long running ones only for I-builds. If nothing else, I hope, running the others for N builds will help "get the bugs out" of running those tests better. And, I would think it would be useful to committers ... if I can figure out how to get the "over time" charts to display as expected.
The "long running" tests did complete ... "without errror" (but, remember, there are two suites "that run with no error" but also produce no results ... for some reason). https://hudson.eclipse.org/perftests/view/Eclipse%20and%20Equinox/job/ep45ILR-perf-lin64/1/ They took 5 and a half hours, and that's for "current build" only ... not "baseline" ... so, typically I'd expect 11 hours total _after_ the short running performance tests finish. And that's with two suites "doing nothing": org.eclipse.equinox.p2.tests.ui https://hudson.eclipse.org/perftests/view/Eclipse%20and%20Equinox/job/ep45ILR-perf-lin64/1/artifact/workarea/I20141118-0830/eclipse-testing/results/html/org.eclipse.equinox.p2.tests.ui_linux.gtk.x86_64_8.0.html org.eclipse.pde.api.tools.tests https://hudson.eclipse.org/perftests/view/Eclipse%20and%20Equinox/job/ep45ILR-perf-lin64/1/artifact/workarea/I20141118-0830/eclipse-testing/results/html/org.eclipse.pde.api.tools.tests_linux.gtk.x86_64_8.0.html My guess is I don't have "previous release" version specified correctly ... or, something? But, it's good news the other three do well on "current build". Not sure they will do so well on "baseline" .. I believe there has been some "breaking changes" for some of the low level internal class hierarchies? org.eclipse.jdt.core.tests.performance org.eclipse.jdt.text.tests org.eclipse.jdt.ui.tests.refactoring
The main mechanics of this have been done for M4, but still need to better display "which are which" ... to add "short set" to nightlies, or something. Even for I-builds, they come in two waves, sometimes the short set being done soon after unit tests are done ... but "long set" still pending" for a many hours (if not a full day) until they show up.
I'm going to count this as fixed. The remaining things I mentioned fall into the category of "improve reporting" ... which will be an ongoing effort.
book keeping ... they do appear in http://download.eclipse.org/eclipse/downloads/drops4/M-4.4.2RC2-201501281000/performance/performance.php
