Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 478485

Summary: Make it easier to run the Eclipse UI unit tests
Product: [Eclipse Project] Platform Reporter: Stefan Xenos <sxenos>
Component: UIAssignee: Platform-UI-Inbox <Platform-UI-Inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: david_williams, Lars.Vogel, psuzzi
Version: 4.6   
Target Milestone: ---   
Hardware: PC   
OS: Windows NT   
Whiteboard:

Description Stefan Xenos CLA 2015-09-27 02:45:59 EDT
If you want to develop for Eclipse UI, most plugins are fairly easy to work on:
- Install Eclipse with PDE
- Clone the Eclipse UI git repository
- Import the projects
- Start coding

Not so for the org.eclipse.ui.tests and org.eclipse.ui.tests.performance plugins. Every time I want to run the tests on a new computer, it takes me at least 4-5 hours to get configured. Assuming that others have similar difficulties, the likely consequence is that many developers won't write or run the unit tests, which is bad.

This bug requests that we make it possible to write and run the Eclipse UI unit tests using only what comes preinstalled in a PDE build of Eclipse and the contents of the Eclipse UI git repository. We can either add new things to the PDE install, add stuff to the UI repository, remove our dependency on third-party libs in order to make this possible.

org.eclipse.ui.tests depends on org.eclipse.core.tests.harness, which is in the runtime repository requires setting yourself up for development on core runtime. This should definitely not be a requirement for UI testing.

It also depends on hamcrest and some other custom tooling. Fortunately, our How To Contribute wiki has a script that makes this easier to install, but that requires finding the wiki.

org.eclipse.tests.harness requires org.eclipse.pde.junit.runtime... and I have no idea where to find that one. I'm currently digging through the PDE repositories to find its source, but it definitely should not be this difficult.
Comment 1 Lars Vogel CLA 2015-09-27 06:26:23 EDT
4-5 hours sounds a bit long, I do the setup approximately 2-3 times per Werk and it usually takes 5-10 Minuten.

But +1 for every improvement in this area.
Comment 2 Stefan Xenos CLA 2015-09-27 12:03:17 EDT
> 4-5 hours sounds a bit long

I started at 7:30pm last night and at around midnight I was still hunting around for org.eclipse.pde.junit.runtime and gave up. This morning I found org.eclipse.pde.junit.runtime after locating and cloning the PDE repositories, which took about 45 minutes... so 4-5 hours seems about right for someone who hasn't done it recently.

I think if you do it frequently enough that you've memorized what git repositories to clone and what plugins to install, it's probably a lot faster -- but I only switch computers a few times a year and need to figure it out from scratch each time, which takes most of the time. I believe my own experience is farily representative of the time taken for someone who hasn't done it before.
Comment 3 Lars Vogel CLA 2015-09-27 14:26:00 EDT
I'm actually unaware of the PDE dependencies. I check that tomorrow.
Comment 4 Stefan Xenos CLA 2015-09-27 22:38:11 EDT
I'm currently fighting with this one:

Bundle 'org.hamcrest' cannot be resolved
Bundle 'org.mockito' cannot be resolved

I see both declared in MANIFEST.MF:

org.hamcrest;bundle-version="1.1.0",
org.mockito;bundle-version="1.8.4"

...and I see both installed in my host:

Hamcrest Library of Matchers	1.1.0.v20090501071000	org.hamcrest	Eclipse.org
Java Mocking and Stubbing Framework	1.8.4.v201303031500	org.mockito	Eclipse.org

...so I'm baffled as to why they aren't resolving.
Comment 5 Stefan Xenos CLA 2015-09-27 22:51:33 EDT
Would it be possible to bundle hamcrest and mockito with the PDE builds of Eclipse so they come correctly preconfigured?

Perhaps we could deal with the org.eclipse.core.tests.harness dependency by including a renamed copy of it somewhere in the UI's git repository. It doesn't seem very big.
Comment 6 Lars Vogel CLA 2015-09-28 05:17:36 EDT
(In reply to Stefan Xenos from comment #5)
> Hamcrest Library of Matchers	1.1.0.v20090501071000	org.hamcrest	Eclipse.org
> Java Mocking and Stubbing Framework	1.8.4.v201303031500	org.mockito
> Eclipse.org

We are using Mockito 1.9.5.
Comment 7 Lars Vogel CLA 2015-09-28 05:23:32 EDT
(In reply to Lars Vogel from comment #3)
> I'm actually unaware of the PDE dependencies. I check that tomorrow.

I think it this dependency is not required, see Bug 478516.
Comment 8 David Williams CLA 2015-09-29 09:55:47 EDT
I agree all unit tests should be easier to run. 

But, am curious, does it help if you download and install (ether in workspace, or target) one of the latest "eclipse-test-framework" zip's, such as 

http://download.eclipse.org/eclipse/downloads/drops4/I20150922-0800/download.php?dropFile=eclipse-test-framework-I20150922-0800.zip

It is a small repo that contains the bundles 

org.easymock_2.4.0.v20090202-0900.jar
org.eclipse.ant.optional.junit_3.3.100.v20140518-1929.jar
org.eclipse.test_3.3.200.v20150327-1546.jar
org.eclipse.test.performance_3.11.0.v20150223-0658.jar
org.eclipse.test.performance.win32_3.1.200.v20140518-1929.jar
org.eclipse.test.source_3.3.200.v20150327-1546.jar
org.hamcrest.core_1.3.0.v201303031735.jar
org.junit_4.12.0.v201504281640.jar

Hope that helps ... and, if not ... it would be good to know why not. 

Thanks,
Comment 9 David Williams CLA 2015-09-29 09:57:09 EDT
(In reply to David Williams from comment #8)
> I agree all unit tests should be easier to run. 
> 
> But, am curious, does it help if you download and install (ether in
> workspace, or target) 

Not workspace, I meant "dev environment, or target".
Comment 10 Lars Vogel CLA 2015-09-29 10:08:41 EDT
(In reply to David Williams from comment #8)
> I agree all unit tests should be easier to run. 
> 
> But, am curious, does it help if you download and install (ether in
> workspace, or target) one of the latest "eclipse-test-framework" zip's, such
> as 
> 
> http://download.eclipse.org/eclipse/downloads/drops4/I20150922-0800/download.
> php?dropFile=eclipse-test-framework-I20150922-0800.zip
> 
> It is a small repo that contains the bundles 

Do we have also an update site for this download?

> org.easymock_2.4.0.v20090202-0900.jar
> org.eclipse.ant.optional.junit_3.3.100.v20140518-1929.jar
> org.eclipse.test_3.3.200.v20150327-1546.jar
> org.eclipse.test.performance_3.11.0.v20150223-0658.jar
> org.eclipse.test.performance.win32_3.1.200.v20140518-1929.jar
> org.eclipse.test.source_3.3.200.v20150327-1546.jar
> org.hamcrest.core_1.3.0.v201303031735.jar
> org.junit_4.12.0.v201504281640.jar

I think Mockito is missing. I opened Bug 478633 to see it that can be added.
Comment 11 David Williams CLA 2015-09-29 12:23:44 EDT
Another easy way to "add all pre-reqs" is to import, from Git, 
the eclipse.platform.releng.aggregator project (just the project, do not pick to clone submodules, or you will get more than you want, probably ... since that would be *everything*). 

In that project is a folder named eclipse.platform.releng.prereqs.sdk. 
It contains a eclipse-sdk-prereqs.target file that a) we use during builds, but b) can also be used by PDE as the target. 

Under preferences, Plugin-in Development, Target Platform is where you would see it listed (automatically) so you could select it there. Might have to hit "reload", or something to get it to "take". I sometimes have to also say "edit" to get it to load everything ... but, don't actually edit it). 

Assuming you do not have *all* of the eclipse platform loaded from Git, but assuming your "dev. environment" is current then at the bottom of that 
eclipse-sdk-prereqs.target  
file, is a bit of code that is normally commented out (since, we would not want it active for builds). If you uncomment it, *using a text editor*, it also adds "current dev. environment" to that target definition, which should give you all you need. 

(As far as I know).
Comment 12 Stefan Xenos CLA 2015-09-29 12:57:12 EDT
Who do we have to talk to in order to get more stuff into the PDE builds? IMO, if you download and install the "Eclipse for Committers" build, you should have all the prereqs you need to run the unit tests. I'd suggest that we add these plugins (mockito, hamcrest, org.eclipse.core.tests.harness) to the Eclipse for Committers build.

> We are using Mockito 1.9.5.

Where do you get that from? When I look in org.eclipse.ui.tests/META-INF/MANIFEST.MF, I see this:

 org.hamcrest;bundle-version="1.1.0",
 org.mockito;bundle-version="1.8.4"

I've been trying to follow these instructions, which don't result in an environment in which org.eclipse.ui.tests compiles:

https://wiki.eclipse.org/Platform_UI/How_to_Contribute

It suggests that you install these plugins:

http://git.eclipse.org/c/platform/eclipse.platform.ui.git/plain/releng/org.eclipse.ui.releng/platformUiTools.p2f

...which installs this:

    <iu id='org.mockito' name='org.mockito' version='1.8.4.v201303031500'>
      <repositories size='1'>
        <repository location='http://download.eclipse.org/tools/orbit/downloads/drops/R20130827064939/repository'/>
      </repositories>
    </iu>
    <iu id='org.hamcrest' name='org.hamcrest' version='1.1.0.v20090501071000'>
      <repositories size='1'>
        <repository location='http://download.eclipse.org/tools/orbit/downloads/drops/R20130827064939/repository'/>
      </repositories>
    </iu>

...so if we're using mockito 1.9.5, the instructions on the wiki are pointing people to the wrong version.

BTW, if someone could update the wiki with a set of steps that actually work, that would be very helpful. I only have one machine that can run the unit tests at the moment and I can't replicate whatever I did to make it work.
Comment 13 Lars Vogel CLA 2015-09-29 13:08:28 EDT
(In reply to Stefan Xenos from comment #12)
> > We are using Mockito 1.9.5.

I might have that wrong, in Android land 1.9.5 is necessary as this is the first version that works on Dalvik. For Eclipse I install the version from Orbit. 

For the platform tests I usually follow my own description: http://www.vogella.com/tutorials/EclipsePlatformDevelopment/article.html#unitTesting 

I just noted that Mockito and Hamcrest is missing here, I update the tutorial shortly. 

But the wiki should also be correct, I think Patrik (cc) uses it and update it on a regular basis.
Comment 14 Stefan Xenos CLA 2015-09-29 13:37:15 EDT
Ugh. The problems with hamcrest and mockito just disappeared on this machine. Presumably it's PDE misbehaving?

Seems like it might be reproducable.
Comment 15 David Williams CLA 2015-09-29 14:24:52 EDT
(In reply to Stefan Xenos from comment #14)
> Ugh. The problems with hamcrest and mockito just disappeared on this
> machine. Presumably it's PDE misbehaving?
> 
> Seems like it might be reproducable.

There is a bug in Orbit about some incompatibility issue with 1.9.5 Mockito ... I don't really understand it ... just giving the cross-reference: 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=403635
Comment 16 Stefan Xenos CLA 2016-05-17 10:27:30 EDT
Marking as FIXED. With the new Oomph setup scripts, it's really easy to set up a workspace that will compile and run the unit tests.