Community
Participate
Working Groups
+++ This bug was initially created as a clone of Bug #405811 +++ I'm splitting off the "documentation" part of bug 405811, since it is likely be be "long lived" ... being fixed during some of the RCs instead of before M7. There are two documents "owned" by the platform that are in general need updating. Our general strategy will be to greatly shorten those documents (if not eliminate them) and provide the relevant information on a wiki. As mentioned in bug 405811 there is some newer "community maintained" document at http://wiki.eclipse.org/Platform-releng/Platform_Build#running_the_tests That is oriented towards local CBI builds, but has been reported not to work for some (bug 405849). The main reason this comes up now, is that we have moved two important files "out of" the test framework (bundle org.eclipse.tests) so in some cases, people who use that framework for their own testing will need to make adjustments. For example, they will have to get their own versions of the files, such as pull from http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/library.xml http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/JUNIT.XSL And then set relevant properties, from command line, or ant script, before they invoke their own test.xml file. For example <property name="library-file" value="${basedir}/library.xml" /> <property name="junit-stylesheet" value="${basedir}/JUNIT.XSL" /> To be clear, this doesn't effect Eclipse Project Platform committers and "production runs" since we already do this, but will likely effect cases where even Eclipse Project Platform committers want to run their own test suite in isolation on their local machine. Hence, we need to update the documentation to make this clear (and come up with best recommendation) as well as clean up the documentation to remove much of the far outdated stuff.
I forgot to paste location of the two documents we need to fix, eliminate, and/or move more to wiki. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/testframework.html http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/automatedtesting.html
Am I reading correctly that you recommend that each project using the test framework should maintain their own copy of the two library files/scripts? Interesting move :)
(In reply to comment #2) > Am I reading correctly that you recommend that each project using the test > framework should maintain their own copy of the two library files/scripts? > Interesting move :) Good question. You could make a copy for yourself, or, its easy to "get" these from git, with commands such as the following wget http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/library.xml wget http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/JUNIT.XSL (Or, if you have an ant script to run your build, it'd be similar, but using the <get ant task. HTH
Status: I've created the initial "wiki versions", at http://wiki.eclipse.org/Platform-releng/Automated_Testing and http://wiki.eclipse.org/Platform-releng/Eclipse_Test_Framework So far these are mostly cleaned up versions of what was available ... that is, I've not checked for correctness or if examples are current. I'll improve a bit more, but thought I'd update this status in cases others want to improve. I will like the "git versions" in git until slightly before RC1, in case anyone wants to refer to them while editing/checking with wiki versions but then plan to remove them, so they are not in any final builds for Kepler. (Of course, they will always bit in git, but to get at them you'll have to look at history, and not master). To repeat those URLs here, for reference, the one for "Automated Testing" is at http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/automatedtesting.html and the one for "Test Framework" is at http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/plain/eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/src/main/scripts/testframework.html
*** Bug 191789 has been marked as a duplicate of this bug. ***
*** Bug 344285 has been marked as a duplicate of this bug. ***
Created attachment 230580 [details] old version removed from git repo
Created attachment 230581 [details] old version removed from git repo For ease of access, I'm attaching the old, out-dated versions here, as I remove them from Git repo, just in case that helps anyone "check" anything.
Besides removing the old 'html' documents from repo, there was one place in library.xml on "usage" that referred to the old document, so I changed that to point to the wiki version. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=5cdb0a96bf073fd0b6b7e0295533a4d05e18055c
As a result of removing the files, there were error messages in log where are "helper" script used to copy them. While minor by itself, I find error messages in the log are easy to misinterpret in months ahead, so removed that no-longer required copy. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=54c68d32f9f2a77954880793c6515c75ae62ccc1
Still need to update wiki pages and test some scenarios, but hope to find nothing that actually needs changed in test framework, so moving to "RC3".
Due to other scheduled tasks and appointments., unlikely I'll finish this "doc and testing" by Friday, so admitting that now and adjusting to RC4.
I did add some additional info to the test framework document, http://wiki.eclipse.org/index.php?title=Platform-releng%2FEclipse_Test_Framework&diff=339349&oldid=335129 But further improvements (and testing various scenarios) will have to wait.
This may be a little bit controversial - but I believe that the test framework is a dinosaur and should die together with the pde-based Eclipse build. Mickael is slowly but steadily pushing the platform build towards a state where it will be possible to execute all tests during the build (such as it can be done for p2), and I believe that it offers much more value than the test framework - you just start the build and tests are run automagically *without* additional setup.
(In reply to Krzysztof Daniel from comment #14) > This may be a little bit controversial - but I believe that the test > framework is a dinosaur and should die together with the pde-based Eclipse > build. > > Mickael is slowly but steadily pushing the platform build towards a state > where it will be possible to execute all tests during the build (such as it > can be done for p2), and I believe that it offers much more value than the > test framework - you just start the build and tests are run automagically > *without* additional setup. I love the idea of getting rid of things! But, how does that work if someone simply wants to run the tests in a different environment ... say ... take an existing build and test it on Redhat OS using Java 8 ... or any of the other many combinations of things we don't "build on"? I'm sure there's an answer ... just need to be sure all these use-cases are documented well. Thanks for your help.
First, let's say that we're far from having all Platform tests working with Surefire ;) So test framework will still be there for some time. Now, the cool part: (In reply to David Williams from comment #15) > I love the idea of getting rid of things! But, how does that work if someone > simply wants to run the tests in a different environment ... say ... take an > existing build and test it on Redhat OS using Java 8 ... or any of the other > many combinations of things we don't "build on"? > > I'm sure there's an answer ... just need to be sure all these use-cases are > documented well. If one wants to use Tycho to run tests on an existing build, they can use this: http://wiki.eclipse.org/Tycho/Testing_with_Surefire#p2Installed_on_provisioned_RCP_application We use it for JBoss Developer Studio, and we've put this configuration in a profile. It allows us to run tests from a Maven module into our full JBDS with a "mvn clean verify -Ptest-on-installed-app". We like it. A lot. https://github.com/jbosstools/jbosstools-build/blob/master/parent/pom.xml#L777
FWIW, the Object Teams build is still PDE based and makes heavy use of the test framework. Needless to say that I have no time to migrate that build during the Luna time frame. OTOH, I don't need any new documentation at this point :)
Moving "out". I'd still like to improve documentation for those who want to use this dinosaur ... at least to the point where I understand it :) Though I'm sure if I put it off long enough, Krzysztof and Mickael will get SureFire (or other tools) working every where :)
David, is this still planned for 4.4?
(In reply to Dani Megert from comment #19) > David, is this still planned for 4.4? Yeah, let's leave on list for another week or two, ... since some improvements may just be wiki changes, I'm hoping to take a pass.
While others have made some improvements, there's still more I'd like to do with the doc (and framework itself) so will re-target this tracking bug.
This really needs to be done, but am removing target, since I've deferred so many times, there must not be a burning need to do it.
Doing a mass "reset to default assignee" of 52 bugs to help make clear it will (very likely) not be me working on things I had previously planned to work on. I hope this will help prevent the bugs from "getting lost" in other people's queries. Feel free to "take" a bug if appropriate.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug. If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
