This Bugzilla instance is deprecated, and most Eclipse projects now use GitHub or Eclipse GitLab. Please see the deprecation plan for details.
Bug 140503 - How to automate the running of Eclipse PDE Unit tests using ant
Summary: How to automate the running of Eclipse PDE Unit tests using ant
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Articles (show other bugs)
Version: unspecified   Edit
Hardware: PC All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: community.articles-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-05-07 13:21 EDT by Brian Joyce CLA
Modified: 2013-01-10 04:17 EST (History)
27 users (show)

See Also:


Attachments
1st draft of article on how to run PDE unit tests using ant (118.20 KB, application/zip)
2007-05-20 18:53 EDT, Brian Joyce CLA
no flags Details
2nd Draft of article on how to run PDE unit tests using ant (150.13 KB, application/zip)
2008-03-04 16:38 EST, Brian Joyce CLA
no flags Details
3rd Draft of article on how to run PDE unit tests using ant (108.93 KB, application/zip)
2008-03-30 19:59 EDT, Brian Joyce CLA
no flags Details
4th Draft of article on how to run PDE unit tests using ant (185.74 KB, application/zip)
2008-04-04 20:59 EDT, Brian Joyce CLA
no flags Details
mylyn/context/zip (3.68 KB, application/octet-stream)
2008-06-13 15:24 EDT, Wayne Beaton CLA
no flags Details
printer friendly PDF version (from HTML via OpenOffice) (278.83 KB, application/octet-stream)
2008-08-07 05:06 EDT, Jörg von Frantzius CLA
no flags Details
Reformatted OpenOffice version (from HTML) (39.00 KB, application/vnd.oasis.opendocument.text)
2008-08-07 05:08 EDT, Jörg von Frantzius CLA
no flags Details
Junit Util plugin modified (12.46 KB, application/x-tar)
2009-07-06 04:25 EDT, Manuel Selva CLA
no flags Details
PDETestExampleProjects-3.4.zip for Eclipse 3.4 (25.38 KB, application/zip)
2009-08-10 20:06 EDT, Paul Thomas CLA
no flags Details
PDETestListener / Collector for 3.5 + 3.6 (8.79 KB, application/x-zip-compressed)
2010-01-22 14:47 EST, Elias Volanakis CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Joyce CLA 2006-05-07 13:21:42 EDT
This article will outline how to integrate PDE unit tests into an automated build system, using a simple eclipse plugin as an example, some simple java classes, basic eclipse plugin techniques and some standard ant tasks.

This article is intended to be a much simpler article than Markus Barchfeld's existing article at : http://www.eclipse.org/articles/Article-PDE-Automation/automation.html. This article concentrates only on the automation of the PDE unit tests and would provide a very useful lightweight mechanism for integrating automated PDE tests into existing ant based build systems, without having to adopt the full eclipse headless build system.


The article would provide a full working example, java code and ant build files showing how PDE unit tests are automated by detailing each of the following steps:

   Prerequisites:

   1. A dedicated PDE test eclipse installation that is only used by the automated build system.
   2. An eclipse plugin A that is to be tested.
   3. An eclipse plugin fragment A-test, that is a plugin fragment for A and that only contains the PDE unit tests for plugin A.
   4. A predefined PDE test port to be used for the PDE test run, this port is used by the PDE test run in the PDE test eclipse installation to output the test results and by the PDE test listener process to listen to the results of the PDE test run.

   Running the PDE unit tests:

   1. Start a PDE test listener process that listens on the PDE test port.
   2. Deploy plugin A and plugin fragment A-test into the PDE test eclipse installation .
   3. Startup the PDE test eclipse installation, specifying the list of unit test classes to be run.
   4. The PDE test eclipse installation starts up and automatically runs the specified tests and outputs the results to the PDE test port  where the PDE test listener process captures the results and outputs them in standard ant junit task xml format.
   5. Remove the deployed plugin and fragment (A and A-test) from the PDE test eclipse installation.
   6. Run the regular ant Junitreport task on the generated test results xml files.
   7. That's it - you have a typical Junit html report containing your PDE unit test results.



I would hope to have my 1st draft completed by end June 2006 at the latest.
Comment 1 Wayne Beaton CLA 2006-05-08 11:01:32 EDT
(In reply to comment #0)
> 
> I would hope to have my 1st draft completed by end June 2006 at the latest.
> 

Sounds good. I'll look for it.
Comment 2 Wayne Beaton CLA 2006-11-27 15:20:49 EST
I'm hardly one to pass judgment on missed self-imposed deadlines, but.... any progress?
Comment 3 Wayne Beaton CLA 2007-01-17 15:52:07 EST
Due to apparent inactivity, I'm marking this as LATER. Feel free to reopen.
Comment 4 Brian Joyce CLA 2007-05-20 18:53:41 EDT
Created attachment 67925 [details]
1st draft of article on how to run PDE unit tests using ant

1st draft of article on how to run PDE unit tests using ant.

Better late than never
Comment 5 Brian Joyce CLA 2007-05-20 18:56:02 EDT
Hi All,

So sorry for leaving this so late but I finally managed to get some time for this article.

Really interested in any and all feedback.

Cheers
Brian
Comment 6 Wayne Beaton CLA 2007-06-04 14:07:20 EDT
Can somebody from PDE build review this article?
Comment 7 David Bosschaert CLA 2007-08-23 07:07:44 EDT
I'm going to review the article. I'm not on the PDE build, but I was involved in developing the described system when it was originally developed.
Comment 8 Andrew Niefer CLA 2007-08-23 10:53:13 EDT
Sorry, I did not notice this before, I'll take a look at it.
Comment 9 David Bosschaert CLA 2007-09-06 05:35:49 EDT
Hi Brian,

Finally had a bit of time to look at your article. Although its very well written and nicely detailed I have a few concerns...

* You say it only relates to Eclipse 3.1. With Eclipse 3.3 released last June I think it is quite outdated. Before you publish I think you should update it to Eclipse 3.3.
* In the introduction you say that there is no other technique to automatically run tests. I actually think that the PDE Build today can also do this. For this it uses a file called test.xml. Have a look at this article and see whether you think this statement is still true: http://www.eclipse.org/articles/Article-PDE-Automation/automation.html
Comment 10 Brian Joyce CLA 2007-10-01 12:06:17 EDT
Hi David,

Thank you for taking the time to review the draft - your feedback is much appreciated.

> * You say it only relates to Eclipse 3.1. With Eclipse 3.3 released last June I
> think it is quite outdated. Before you publish I think you should update it to
> Eclipse 3.3.

I have done some work on the Eclipse 3.3 version and I will indeed update the article to cover Eclipse 3.3.

> * In the introduction you say that there is no other technique to automatically
> run tests. I actually think that the PDE Build today can also do this. For this
> it uses a file called test.xml. Have a look at this article and see whether you
> think this statement is still true:
> http://www.eclipse.org/articles/Article-PDE-Automation/automation.html

The PDE Test method described in this article is intended to be a lightweight version of Markus Barchfeld's method (described in the article you mention). I accept your point and I will update the article to include a reference to Markus's article. 

Thanks again,
Brian
Comment 11 Andrew Niefer CLA 2007-10-01 13:32:47 EDT
(In reply to comment #6)
> Can somebody from PDE build review this article?
> 

I just took a quick look at the article.  There is nothing particular to pde.build here, and I have no comments on the test aspects of the article, but there are a couple of small things:
1) Running PDE Unit Tests in Ant, Marker 5 : 3.3 no longer has a startup.jar.  The class name will be org.eclipse.equinox.launcher.Main, the jar is the plugins/org.eclipse.equinox.launcher_<version>.jar.  See http://wiki.eclipse.org/Starting_Eclipse_Commandline_With_Equinox_Launcher for some example ant.

2) There is nothing in the article about connecting the tests to an automated headless build.  pde.build does provide opportunity to run custom ant after the build is complete using the customTargets.xml postBuild target.  Perhaps a paragraph on automatically extracting build results and running the tests on them would be appropriate.
Comment 12 Brian Joyce CLA 2007-10-15 09:11:46 EDT
(In reply to comment #11)
Hi Andrew,

Many thanks for taking the time to review the draft article - I appreciate your feedback.

> (In reply to comment #6)
> > Can somebody from PDE build review this article?
> > 
> 
> I just took a quick look at the article.  There is nothing particular to
> pde.build here, and I have no comments on the test aspects of the article, but
> there are a couple of small things:
> 1) Running PDE Unit Tests in Ant, Marker 5 : 3.3 no longer has a startup.jar. 
> The class name will be org.eclipse.equinox.launcher.Main, the jar is the
> plugins/org.eclipse.equinox.launcher_<version>.jar.  See
> http://wiki.eclipse.org/Starting_Eclipse_Commandline_With_Equinox_Launcher for
> some example ant.

I will include an Eclipse 3.3 version in the revised draft that will use the Eclipse 3.3 mechanism you refer to for launching the test run.

> 
> 2) There is nothing in the article about connecting the tests to an automated
> headless build.  pde.build does provide opportunity to run custom ant after the
> build is complete using the customTargets.xml postBuild target.  Perhaps a
> paragraph on automatically extracting build results and running the tests on
> them would be appropriate.
> 

My aim in this article is to provide a generic mechanism for running PDE Tests from ant, so that ant users can either integrate with the headless build or any ant build system. I am not an expert on the Eclipse headless build so I don't intend to provide a concrete example of how to integrate it into the headless build, but I will be adding a link to Markus Barchfeld's article (http://www.eclipse.org/articles/Article-PDE-Automation/automation.html) in my article. Andrew, will that suffice to cover this point ?

Thanks,
Brian
Comment 13 Brian Joyce CLA 2008-03-04 16:38:44 EST
Created attachment 91575 [details]
2nd Draft of article on how to run PDE unit tests using ant

Apologies for the delay in updating the article with the review comments.

The article and examples are now updated for Eclipse 3.3 (tested using Eclipse 3.3.1.1), and the original Eclipse 3.1 version is also included.
Comment 14 Brian Joyce CLA 2008-03-30 19:59:48 EDT
Created attachment 94157 [details]
3rd Draft of article on how to run PDE unit tests using ant  

This version of the article and examples have been enhanced so that the Eclipse 3.3 examples should work with any Eclipse 3.3 installation (not just Eclipse 3.3.1.1 as in the previous draft). The Eclipse 3.3 examples have been tested using Eclipse
3.3.2 and 3.3.1.1.

The original Eclipse 3.1 version of the article and examples are still included.

Any and all feedback gratefully received.

Regards
Brian
Comment 15 Wayne Beaton CLA 2008-04-01 15:35:01 EDT
Hi Brian. It's starting to look good. I'm nothing close to a PDE Build expert; please keep this in mind as you read over my comments.

Your hints and tips section at the end might be better as an appendix. It'd be nice to see some discussion around using fragments for test bundles. The discussion of the example application might also belong in an appendix.

I think that a little more discussion about the listener/collector is necessary. These are running in separate JVMs (which you imply in step 4). I think this relationship needs to be discussed.

The first note in "Running PDE Unit Tests in Ant" indicates that the example code "should not be expanded into an eclipse installation". Do you mean "workspace"? (I tend to think of the "installation" as the "eclipse" directory with all the bundles and such).

The exerpt of the build file seems mis-labeled. Maybe it's enough to indicate that the file *builds* and runs the tests.

It needs a conclusion.
Comment 16 Brian Joyce CLA 2008-04-03 07:34:04 EDT
(In reply to comment #15)

Hi Wayne,

Thanks for taking the time to review the article and for the excellent feedback, I will rework the article based on your comments. See more replies to your comments in-line below.

> Your hints and tips section at the end might be better as an appendix. 

I agree, I'll move these to an appendix.

> It'd be nice to see some discussion around using fragments for test bundles. 

That's a good idea I will add a section on why we use fragments for PDE test bundles.

> The discussion of the example application might also belong in an appendix.

OK, I'll move those sections to an appendix too.
 
> I think that a little more discussion about the listener/collector is
> necessary. These are running in separate JVMs (which you imply in step 4). I
> think this relationship needs to be discussed.

Again I agree, I will put more effort into detailing the interactions between the PDE test listener process and the PDE test eclipse process.

> 
> The first note in "Running PDE Unit Tests in Ant" indicates that the example
> code "should not be expanded into an eclipse installation". Do you mean
> "workspace"? (I tend to think of the "installation" as the "eclipse" directory
> with all the bundles and such).

What I mean here is that the example zip is a full project complete with an ant build system, so the advice means unzip the examples into a directory that is separate from your eclipse installation (I agree with your interpretation of eclipse installation).

I'll try to clarify that in the article.

> 
> The exerpt of the build file seems mis-labeled. Maybe it's enough to indicate
> that the file *builds* and runs the tests.

Could you let me know which parts you think are mis-labeled?

I know that labels 4. and 5. appear twice (first where the targets are called and also where the targets are defined).  I think that this section is needed as this article concerns ant too and the section details the actual ant script that runs the pde tests and collects the results, which is the main purpose of the article.

> 
> It needs a conclusion.
> 

I agree - I'll add one.


Many thanks again for the feedback.

Regards,
Brian
Comment 17 Brian Joyce CLA 2008-04-04 20:59:54 EDT
Created attachment 94945 [details]
4th Draft of article on how to run PDE unit tests using ant  

The article has been updated in response to Wayne Beaton's review comments.

> Your hints and tips section at the end might be better as an appendix. 
Done.

> It'd be nice to see some discussion around using fragments for test bundles. 
Done.

> The discussion of the example application might also belong in an appendix.
Done.

> I think that a little more discussion about the listener/collector is
> necessary. These are running in separate JVMs (which you imply in step 4). I
> think this relationship needs to be discussed.
Done, I've added more detailed sections on each of the processes involved and I've also added an overview diagram to help describe the steps involved.

> The first note in "Running PDE Unit Tests in Ant" indicates that the example
> code "should not be expanded into an eclipse installation". Do you mean
> "workspace"? (I tend to think of the "installation" as the "eclipse" directory
> with all the bundles and such).
Done, I hope the new wording is clearer.

> It needs a conclusion.
Done.


I have also removed the 3.1 version of the article (but have kept the 3.1 example code).

As ever all comments and feedback gratefully received.

Regards
Brian
Comment 18 Wayne Beaton CLA 2008-05-13 16:35:05 EDT
Can I either get some comments about this article from the community, or a couple of +1s indicating that it's ready for publishing?
Comment 19 Wayne Beaton CLA 2008-05-27 16:50:04 EDT
(In reply to comment #18)
> Can I either get some comments about this article from the community, or a
> couple of +1s indicating that it's ready for publishing?
> 

Anyone?
Comment 20 David Bosschaert CLA 2008-05-30 03:28:53 EDT
Nice article!
Comment 21 Wayne Beaton CLA 2008-05-30 10:24:54 EDT
(In reply to comment #20)
> Nice article!
> 

Sorry if this comes across as pedantic, but--per our convention--does this translate into a "+1"?
Comment 22 David Bosschaert CLA 2008-05-30 10:39:20 EDT
+1!
Comment 23 Wayne Beaton CLA 2008-05-30 16:22:23 EDT
I've copied the content into Eclipse CVS. It now appears on the website [1], but has not been added to our database (so it won't show up on /articles or /resources yet).

I'll try and take a pass over it this weekend (or early next week).

Any new changes should (please) be submitted in the form of a patch to:

:pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse:/www/articles/Article-PDEJUnitAntAutomation

[1]http://www.eclipse.org/articles/article.php?file=Article-PDE-JUnitAntAutomation/index.html
Comment 24 Edwin Dankert CLA 2008-06-12 15:18:39 EDT
That is looking really good.

For what it's worth +1 from me.
Comment 26 Wayne Beaton CLA 2008-06-13 15:24:30 EDT
I have committed an update that fixes up some spelling and grammar errors.

The article is live and is accessible from the home page, /resources, and /articles.

Brian, can you please send me your coordinates (mailing address, phone number) along with your shirt size and I'll see if I can find some nice appreciation swag for you? Send it by personal e-mail (I assume you don't want that information published publically)
Comment 27 Wayne Beaton CLA 2008-06-13 15:24:42 EDT
Created attachment 104910 [details]
mylyn/context/zip
Comment 28 Conor Carr CLA 2008-07-03 06:32:56 EDT
I wonder would it be possible to update the example to use junit4 ?
Comment 29 Jason Hocker CLA 2008-07-03 12:05:51 EDT
Could this be updated for Eclipse 3.4?  I am able to run the example in 3.3 but wouldn't know what else I need to do.  I'd appreciate it if someone could email me assistance.
Comment 30 Brian Joyce CLA 2008-07-03 13:30:11 EDT
(In reply to comment #28)
> I wonder would it be possible to update the example to use junit4 ?
> 

Hi Conor,

I have no plans to support Junit 4 for now as Junit 3 is just fine for the kinds of PDE tests we're writing. 

But please try it yourself, it should be possible to upgrade to Junit 4 but I'm not sure what possible issues there could be - especially in the PDETestListener.

If I do ever upgrade to Junit 4 then I will let you know.

Cheers
Brian
Comment 31 Brian Joyce CLA 2008-07-03 13:47:16 EDT
(In reply to comment #29)
> Could this be updated for Eclipse 3.4?  I am able to run the example in 3.3 but
> wouldn't know what else I need to do.  I'd appreciate it if someone could email
> me assistance.
> 

Hi Jason,

As you can imagine the article itself cannot keep up with every release of Eclipse, and unfortunately I haven't had a chance to update things for Eclipse 3.4 yet.

If you want to update to 3.4 then I would advise that you have a look at all the build.xml files (specifically projects\PhoneBookExample-test\build.xml) and update the path definitions e.g. "build.class.path" to point to the relevant Eclipse 3.4 jars, note you may need to add new 3.4 jars here.

It should work assuming that the Eclipse Junit api classes (i.e. org.eclipse.jdt.internal.junit.model.ITestRunListener2 and
import org.eclipse.jdt.internal.junit.model.RemoteTestRunnerClient) haven't changed in 3.4.

There may also be issues if the launching mechanism has changed in 3.4.

Anyway hope that helps.

Good luck.
Brian
Comment 32 Jason Hocker CLA 2008-07-03 14:10:30 EDT
(In reply to comment #31)
> There may also be issues if the launching mechanism has changed in 3.4.
> Anyway hope that helps.

I believe the problems I'm having is that I cannot launch the test Eclipse to run the junit tests.
Comment 33 Kaloyan Raev CLA 2008-07-28 05:33:54 EDT
This is great article. Thanks!
It would be very valuable if it is extended with performance tests and their automation. May be this should be a separate article. 
Comment 34 Hans Schwaebli CLA 2008-07-30 04:28:47 EDT
I have difficulties applying that to our project.

We have a RCP product to test. So a parameter -product is required. But this is not all. I need to pass the location of the configuration folder with -configuration. Otherwise the product is not started before the tests but a Eclipse runtime workbench.

But the configuration folder is very tricky because of the update configuration plugin. As it seems, every product needs to use that bundle. And so there is a file /org.eclipse.update/platform.xml which contains features and plugins. It seems that the entries in this must match exactly the features and plugins which are contained in the features and plugins folders. Since I have added a lot of them because of test environment, the product does not start but a runtime workbench.

I am struggeling with this -configuration parameter for a while. I need to use it, otherwise the test starts a runtime workbench and not the product to be tested.

Does anyone know how to solve these problems?
Comment 35 Jörg von Frantzius CLA 2008-08-07 05:06:15 EDT
Created attachment 109388 [details]
printer friendly PDF version (from HTML via OpenOffice)

The HTML on http://www.eclipse.org/articles/article.php?file=Article-PDEJUnitAntAutomation/index.html doesn't print well from any browser I tried. I manually reformatted it a bit in OpenOffice and produced this PDF from it (also contains clickable bookmarks for headings)
Comment 36 Jörg von Frantzius CLA 2008-08-07 05:08:36 EDT
Created attachment 109389 [details]
Reformatted OpenOffice version (from HTML)
Comment 37 Brian Joyce CLA 2008-08-07 05:38:26 EDT
(In reply to comment #35)

Thank you Jorg for doing this, the PDF version looks really good.

Regards
Brian
Comment 38 Ben Vitale CLA 2008-08-21 16:13:19 EDT
I tracked down the pieces that were breaking this example on Eclipse 3.4. Without much understanding of Equinox, I can't assert that these are 100% correct solutions, but they allowed me to run the example tests against 3.4.

#1 - After extracting a clean installation of Eclipse 3.4 for test purposes, declare the plugins and fragment in the <eclipse_home>/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info file.

PhoneBookExample,1.0.0,file:plugins/PhoneBookExample_1.0.0.jar,4,true
pde.test.utils,3.3,file:plugins/pde.test.utils_3.3.jar,4,true	PhoneBookExample_test,1.0.0,file:plugins/PhoneBookExample_test_1.0.0.jar,4,false

#2 - Don't wipe out the "configuration" folder every time the tests run.  Line 80 in the old PhoneBookExample-test/build.xml needs to be commented out (or at least refined). Otherwise the changes made to the bundles.info file will be lost.
Comment 39 Mykola Dzyuba CLA 2008-09-04 19:18:43 EDT
I've got it working on Eclipse 3.4 on Mac OS X 10.5. Here is details on changes I made to the code and test run script.

I am running the test listener in a separate instance of Eclipse 3.4. I had to update pde.test.utils MANIFEST.MF with these plugins:
Require-Bundle: org.eclipse.core.runtime;bundle-version="3.4.0",
 org.eclipse.ui;bundle-version="3.4.0",
 org.eclipse.jdt.junit,
 org.eclipse.jdt.junit.runtime;bundle-version="3.3.0",
 org.apache.ant,
 org.junit

That fixed a couple of exceptions that reported missing classes. 

I run the test run from a command line with this command:

cd /Users/mykola/Code/projects/testuirun/eclipse

java -classpath ./plugins/org.eclipse.equinox.launcher_1.0.100.v20080509-1800.jar \
org.eclipse.equinox.launcher.Main \
-application org.eclipse.pde.junit.runtime.uitestapplication \
-data /Users/mykola/Code/projects/testuirun/ws1	\
-dev bin \
-clean \
-port 1623 \
-testpluginname PhoneBookExample \
-classnames phonebookexample.dialogs.PhoneBookEntryEditorDialogTest \
-vmargs \
-XstartOnFirstThread \
-Xms512m \
-Xmx1024m \
-XX:MaxPermSize=256m \
-Dos=macosx \
-Darch=x86 \
-Dws=carbon \
-Dorg.eclipse.swt.internal.carbon.smallFonts

Note, the Mac run requires -XstartOnFirstThread option. 

The test run fine. I saw Eclipse UI came up, the dialogs displayed, etc. The test results listener printed:

Listening on port 1623 for test suite PhoneBookExample results ...
PDE Test Run Started - running 6 tests ...
Test Tree Entry - Description: 1,phonebookexample.dialogs.PhoneBookEntryEditorDialogTest,true,6
Test Tree Entry - Description: 2,testDialogTitle(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
Test Tree Entry - Description: 3,testInitialDialog(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
Test Tree Entry - Description: 4,testInvalidNumberDisablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
Test Tree Entry - Description: 5,testValidNumberEnablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
Test Tree Entry - Description: 6,testChangingNameUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
Test Tree Entry - Description: 7,testChangingNumberUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest),false,1
  Test Started - 1 - testDialogTitle(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 1 - testDialogTitle(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Started - 2 - testInitialDialog(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 2 - testInitialDialog(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Started - 3 - testInvalidNumberDisablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 3 - testInvalidNumberDisablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Started - 4 - testValidNumberEnablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 4 - testValidNumberEnablesOKButton(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Started - 5 - testChangingNameUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 5 - testChangingNameUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Started - 6 - testChangingNumberUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
  Test Ended   - 6 - testChangingNumberUpdatesPhoneBookEntryData(phonebookexample.dialogs.PhoneBookEntryEditorDialogTest)
Test Run Ended   - PASSED - Total: 6 (Errors: 0, Failed: 0, Passed: 6), duration 794ms.

I hope this information will be helpful.

Comment 40 Barbara Rosi-Schwartz CLA 2008-11-14 12:01:24 EST
Hi.

I have been trying to run the sample code on Eclipse 3.4.1 and Windows XP and was having an SWT "Widget disposed" type exception which ultimately caused a Java Result 13 error to be produced by ant.

I have followed Ben's advice (comment #38) and all now works. I thought I would share for other people that may come across the same error.

Thanks Ben, great contribution.
Comment 41 Andreas Buehler CLA 2009-01-20 06:15:37 EST
Hi Brian, 
apparently the link to the PDETestExampleProjects-3.3.zip file in your article is dead meanwhile. Can you post us to the new location since I wasn't able to find it anywhere.
Maybe you can attach it to this issue (if it's not too big)
Greetings Andi
Comment 42 Brian Joyce CLA 2009-01-20 06:34:22 EST
(In reply to comment #41)
> Hi Brian, 
> apparently the link to the PDETestExampleProjects-3.3.zip file in your article
> is dead meanwhile. Can you post us to the new location since I wasn't able to
> find it anywhere.
> Maybe you can attach it to this issue (if it's not too big)
> Greetings Andi
> 

Hi Andi,

Just tested the link and it's working fine.

Cheers
Brian
Comment 43 Andreas Buehler CLA 2009-01-20 10:32:31 EST
Shit, sorry, you're right. Aparently I used the wrong link. I used this one: http://www.eclipse.org/articles/Article-PDEJUnitAntAutomation/article.html#PDE_Test_Utilities 
Sorry for the trouble - by the way, great article, just got the headless tests to run ;-)
Comment 44 Manuel Selva CLA 2009-02-18 04:36:57 EST
Hi,

I successfully integrated my J-Unit tests into my PDE headless build thanks to this great article. Many thanks !

Nevertheless I have some little troubles with the generated HTML files. Using the 3 provided PDE Junit utility classes the html report doesn't correctly reflect Packages and Classes used to run the test.

In fact in the report I only have one package and one class where the package name and class name comes form the test plugin name given as parameter to the PDEResultsCollector main method. For example if my test plug-in is com.mycompany.test the report will show one package called com.mycompany and one class called test containing all my tests.

I noticed that only one XML file containing tests results is generated by the PDETestListener class and it seems junireport needs one xml file for each executed TestCase ... Am I wrong ?

Does any body reproduced this problem ? For information As test.list.class variable I am using a single AllTests class (with a suite() method). I also tried with a list of TestCase classes and the result is the same ...

I am investigating more on that issue but I can't find the exact format of XML files needed as input of the Ant junitreport task ... ??!!! I will let you know about  any news

Regards

Manu
Comment 45 Manuel Selva CLA 2009-03-02 11:40:38 EST
Hi all,

As said here: ttp://manuelselva.wordpress.com/2009/03/02/headless-build-final-step/ I was finally able to have reports correctly generated.

If anyone is interested to know how please let me know.

Moreover I think it can be nice to have classes like the ones provided in this article directly provided by the PDE. 

What do you think about that ? May be the place for a contribution ?

Regards

Manu
Comment 46 Manuel Selva CLA 2009-03-02 13:16:10 EST
The good link is 

http://manuelselva.wordpress.com/2009/03/02/headless-build-final-step/

Moreover maybe the PDE could provide an option to the application used to run PDE tests in order to say that we want to run test AND generates XML report files.

Manu
 
(In reply to comment #45)
> Hi all,
> 
> As said here:
> http://manuelselva.wordpress.com/2009/03/02/headless-build-final-step/ I was
> finally able to have reports correctly generated.
> 
> If anyone is interested to know how please let me know.
> 
> Moreover I think it can be nice to have classes like the ones provided in this
> article directly provided by the PDE. 
> 
> What do you think about that ? May be the place for a contribution ?
> 
> Regards
> 
> Manu
> 

Comment 47 Christian Schneider CLA 2009-07-03 11:16:30 EDT
Hi Manu,

I am interested in your fixes for the reports - can't you just post them here as a patch? It would be awesome to have nice reports from this headless build solution.

Best regards and thanks to all for the effort put into this,
Christian
Comment 48 Manuel Selva CLA 2009-07-06 04:25:25 EDT
Created attachment 140842 [details]
Junit Util plugin modified
Comment 49 Manuel Selva CLA 2009-07-06 04:27:07 EDT
I added in attachment the modified plugin I am using to generate correct reports.

Let me know if you have any questions.

Regards

Manu

(In reply to comment #47)
> Hi Manu,
> 
> I am interested in your fixes for the reports - can't you just post them here
> as a patch? It would be awesome to have nice reports from this headless build
> solution.
> 
> Best regards and thanks to all for the effort put into this,
> Christian
> 

Comment 50 Keynan CLA 2009-07-30 18:02:34 EDT
Hi Brian,

I like your article. However, It doesn't seem to work. I recently tried downloading it and running each ant build in turn. An error occurred durring 
run_pde_tests: [java] Java Result: 13.

Would you please be able to update the example code for modern eclipse versions.
Comment 51 Brian Joyce CLA 2009-07-31 13:26:17 EDT
(In reply to comment #50)
> Hi Brian,
> 
> I like your article. However, It doesn't seem to work. I recently tried
> downloading it and running each ant build in turn. An error occurred durring 
> run_pde_tests: [java] Java Result: 13.
> 
> Would you please be able to update the example code for modern eclipse
> versions.
> 

Hi Keynan,

I'd love to be able to update the example for the latest versions of Eclipse but my family life is demanding all my spare time at the moment and I don't see that changing anytime soon.

Have a look at some of the excellent contributions in the earlier comments these might help you to get it working for your version of Eclipse, if and when you do could you update the example ?

Regards
Brian
Comment 52 Paul Thomas CLA 2009-08-10 20:06:23 EDT
Created attachment 143986 [details]
PDETestExampleProjects-3.4.zip for Eclipse 3.4

I got the examples working with Eclipse 3.4.2 using some of the comments above then ran 'ant zip_example -Dtarget.eclipse.version=3.4' to create the PDETestExampleProjects-3.4.zip that I am attaching.

If I use the updated examples in this zip, modify <eclipse_home>/configuration/org.eclipse.equinox.simpleconfigurator/bundles.info as described in comment #38, and specify -Dtarget.eclipse.version=3.4 on the command line, I am able to build and run the examples with Eclipse 3.4.2.

Caveat: I'm not an Ant or Eclipse expert. It worked for me. Your mileage may vary.
Comment 53 SamehTawfik CLA 2009-09-25 17:28:48 EDT
I downloaded PDETestExampleProjects-3.4.zip and run it successfully with Eclipse 3.4, but it failed when I tried to run it with Eclipse 3.5.

This example runs fine from Eclipse 3.5 IDE, but not from the command line? 

I知 not sure, but it seems the syntax of the following command in \projects\PhoneBookExample-test\build.xml needs to be modified?

    <java classname="org.eclipse.equinox.launcher.Main"
          classpathref="equinox.launcher.class.path"
          dir="${plugin.dir}"
          fork="yes">
      <arg line="-application org.eclipse.pde.junit.runtime.uitestapplication -data ${test.reports.dir}/output/ws -dev bin -clean -port ${pde.test.port} -testpluginname PhoneBookExample -classnames ${test.classes.list}" />
    </java>

Unfortunately, I could not find enough information about org.eclipse.pde.junit.runtime.uitestapplication process.

So, does anyone know how to fix this problem?
Comment 54 SamehTawfik CLA 2009-09-28 16:27:38 EDT
Sorry, but I forgot to specify the error message that gets generated when I tried to run PDETestExampleProjects-3.4.zip for Eclipse 3.4 with Eclipse 3.5:

run_pde_tests:
    [mkdir] Skipping C:\tmp\Eclipse_PDE_Unit_Tests\projects\PhoneBookExample-test\target\reports\test\output\ws because it already exists.
     [java] Executing 'C:\Program Files\Java\jdk1.6.0_13\jre\bin\java.exe' with arguments:
     [java] '-classpath'
     [java] 'C:\tmp\Eclipse_PDE_Unit_Tests\projects\eclipse-3.5\eclipse\plugins\org.eclipse.equinox.launcher_1.0.200.v20090520.jar'
     [java] 'org.eclipse.equinox.launcher.Main'
     [java] '-application'
     [java] 'org.eclipse.pde.junit.runtime.uitestapplication'
     [java] '-data'
     [java] 'C:\tmp\Eclipse_PDE_Unit_Tests\projects\PhoneBookExample-test\target\reports\test/output/ws'
     [java] '-dev'
     [java] 'bin'
     [java] '-clean'
     [java] '-port'
     [java] '52831'
     [java] '-testpluginname'
     [java] 'PhoneBookExample'
     [java] '-classnames'
     [java] 'phonebookexample.dialogs.PhoneBookEntryEditorDialogTest'
     [java]
     [java] The ' characters around the executable and arguments are
     [java] not part of the command.
     [java] Exception in thread "WorkbenchTestable" org.eclipse.swt.SWTException: Failed to execute runnable (java.lang.IllegalArgumentException
: No Classloader found for plug-in PhoneBookExample)
     [java]     at org.eclipse.swt.SWT.error(SWT.java:3884)
     [java]     at org.eclipse.swt.SWT.error(SWT.java:3799)
     [java]     at org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:195)
     [java]     at org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:150)
     [java]     at org.eclipse.swt.widgets.Display.syncExec(Display.java:4312)
     [java]     at org.eclipse.ui.internal.testing.WorkbenchTestable.runTest(WorkbenchTestable.java:112)
     [java]     at org.eclipse.pde.internal.junit.runtime.UITestApplication.runTests(UITestApplication.java:112)
     [java]     at org.eclipse.ui.internal.testing.WorkbenchTestable$1.run(WorkbenchTestable.java:71)
     [java]     at java.lang.Thread.run(Thread.java:619)
     [java] Caused by: java.lang.IllegalArgumentException: No Classloader found for plug-in PhoneBookExample
     [java]     at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.getClassLoader(RemotePluginTestRunner.java:77)
Comment 55 Elias Volanakis CLA 2010-01-22 14:47:52 EST
Created attachment 156982 [details]
PDETestListener / Collector for 3.5 + 3.6

This provides a PDETestListener, PDETestResultsCollector that works with 3.5 and 3.6.
Comment 56 Elias Volanakis CLA 2010-01-22 15:02:36 EST
Brian,

I would like to use some code from your article in the Eclipse Riena Project - http://eclipse.org/riena

However, I found that there is no licensing information in the code. I'm hoping that you can help me clarify the licensing for this code, so that I get it past the Eclipse Foundation's legal team:

1) Can you confirm (via bugzilla reply), that you are the only author for code in 
https://bugs.eclipse.org/bugs/attachment.cgi?id=94945 
(PDETestExampleProjects-*.zip)

2) Do you have the rights to place the code under the Eclipse Public License?

3) If the answer to the above is yes, could you kindly change the copyright headers and re-attach? Templates for the copyright headers can be found here:
http://eclipse.org/legal/copyrightandlicensenotice.php

If you want to save time it would be enough for me to do it for these three files only: PDETestListener.java, PDETestPortLocator.java, PDETestResultsCollector.java in PDETestExampleProjects-3.3.zip)

I am sorry for the inconvenience and hope you can help.

Thanks,
Elias.
Comment 57 Brian Joyce CLA 2010-01-22 15:59:19 EST
(In reply to comment #56)

Hi Elias,

Comments below ...

> 
> I would like to use some code from your article in the Eclipse Riena Project -
> http://eclipse.org/riena

That's nice to hear, thanks.

> 
> However, I found that there is no licensing information in the code. I'm hoping
> that you can help me clarify the licensing for this code, so that I get it past
> the Eclipse Foundation's legal team:
> 
> 1) Can you confirm (via bugzilla reply), that you are the only author for code
> in 
> https://bugs.eclipse.org/bugs/attachment.cgi?id=94945 
> (PDETestExampleProjects-*.zip)

Yes, I am the only author.

> 
> 2) Do you have the rights to place the code under the Eclipse Public License?

Yes.

> 
> 3) If the answer to the above is yes, could you kindly change the copyright
> headers and re-attach? Templates for the copyright headers can be found here:
> http://eclipse.org/legal/copyrightandlicensenotice.php
> 
> If you want to save time it would be enough for me to do it for these three
> files only: PDETestListener.java, PDETestPortLocator.java,
> PDETestResultsCollector.java in PDETestExampleProjects-3.3.zip)

You can freely copy the source and add any headers you need to them for your work, but I won't be doing that for you. This article is here to provide code and examples to be freely copied, modified and reused as users see fit.

I hope that helps you.

Regards
Brian
Comment 58 Elias Volanakis CLA 2010-01-29 13:53:00 EST
Brian, thanks for the help. 

I've obtained the IP team's "seal of approval" for the pde.test.utils plugin. :-)

Interested Eclipse committers can refer to this CQ: http://dev.eclipse.org/ipzilla/show_bug.cgi?id=3705
Comment 59 minh.nguyen CLA 2011-03-10 17:38:45 EST
Hi, I'm having problems building the PDETestListener/Collector for Eclipse 3.6.

Eclipse displays the following warning:
Discouraged access: The type ITestRunListener2 is not accessible due to restriction on required library C:\eclipse\plugins\org.eclipse.jdt.junit.core_3.6.1.r361_v20100825-0800.jar
Comment 60 minh.nguyen CLA 2011-03-10 17:46:03 EST
Oh, and here's the build output:

Buildfile: C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\build.xml
build:
    [javac] Compiling 3 source files to C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\target\classes
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:13: package org.eclipse.jdt.internal.junit.model does not exist
    [javac] import org.eclipse.jdt.internal.junit.model.ITestRunListener2;
    [javac]                                            ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:15: cannot find symbol
    [javac] symbol: class ITestRunListener2
    [javac] public class PDETestListener implements ITestRunListener2 {
    [javac]                                         ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestResultsCollector.java:3: package org.eclipse.jdt.internal.junit.model does not exist
    [javac] import org.eclipse.jdt.internal.junit.model.ITestRunListener2;
    [javac]                                            ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestResultsCollector.java:4: package org.eclipse.jdt.internal.junit.model does not exist
    [javac] import org.eclipse.jdt.internal.junit.model.RemoteTestRunnerClient;
    [javac]                                            ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:118: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		if (status == ITestRunListener2.STATUS_OK) {
    [javac] 		              ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:121: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		} else if (status == ITestRunListener2.STATUS_FAILURE) {
    [javac] 		                     ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:125: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		} else if (status == ITestRunListener2.STATUS_ERROR) {
    [javac] 		                     ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:137: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		if (status == ITestRunListener2.STATUS_OK) {
    [javac] 		              ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:139: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		} else if (status == ITestRunListener2.STATUS_FAILURE) {
    [javac] 		                     ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestListener.java:141: cannot find symbol
    [javac] symbol  : variable ITestRunListener2
    [javac] location: class pde.test.utils.PDETestListener
    [javac] 		} else if (status == ITestRunListener2.STATUS_ERROR) {
    [javac] 		                     ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestResultsCollector.java:17: cannot find symbol
    [javac] symbol  : class ITestRunListener2
    [javac] location: class pde.test.utils.PDETestResultsCollector
    [javac] 		new RemoteTestRunnerClient().startListening(new ITestRunListener2[] { pdeTestListener }, port);
    [javac] 		                                                ^
    [javac] C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\src\pde\test\utils\PDETestResultsCollector.java:17: cannot find symbol
    [javac] symbol  : class RemoteTestRunnerClient
    [javac] location: class pde.test.utils.PDETestResultsCollector
    [javac] 		new RemoteTestRunnerClient().startListening(new ITestRunListener2[] { pdeTestListener }, port);
    [javac] 		    ^
    [javac] 12 errors

BUILD FAILED
C:\Users\minguyen\Desktop\pdetestexample\projects\pde.test.utils\build.xml:34: Compile failed; see the compiler error output for details.

Total time: 543 milliseconds
Comment 61 Gopal Kumar CLA 2013-01-10 04:17:50 EST
I am Also getting the above error mentioned by minh.nguyen@viasat.com 
Any resolution for this? quick help appreciated.
Thanks,
Gopal