Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 544472 - [JUnit 5] test plugin must depend on junit runtime
Summary: [JUnit 5] test plugin must depend on junit runtime
Status: VERIFIED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.10   Edit
Hardware: PC Windows 7
: P3 major (vote)
Target Milestone: 4.13 M1   Edit
Assignee: Julian Honnen CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 544790 (view as bug list)
Depends on:
Blocks: 546114
  Show dependency tree
 
Reported: 2019-02-15 04:27 EST by Julian Honnen CLA
Modified: 2019-07-09 06:36 EDT (History)
5 users (show)

See Also:


Attachments
JUnit5 Test Plugin (2.48 KB, application/x-zip-compressed)
2019-02-15 04:27 EST, Julian Honnen CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Julian Honnen CLA 2019-02-15 04:27:17 EST
Created attachment 277583 [details]
JUnit5 Test Plugin

I created a JUnit 5 test plugin (attached) according to the Tycho How To [1].

The tests run fine in tycho, but Run as JUnit * Test doesn't show up in eclipse. It's only available when I add a package import for org.junit.platform.commons.annotation.

It can then be run as JUnit Plug-in test [2], but not as regular JUnit Test:
java.lang.NoClassDefFoundError: org/junit/platform/engine/TestEngine
...
Caused by: java.lang.ClassNotFoundException: org.junit.platform.engine.TestEngine

For that to work, I have to also add dependencies to
- org.junit.jupiter.engine
- org.junit.platform.engine
- org.opentest4j


Adding a couple of build dependencies so that I can run tests from eclipse doesn't seem right when tycho works fine without them. Shouldn't PDE also add a JUnit5 classpath container like JDT?


[1] https://wiki.eclipse.org/Tycho/How_Tos/JUnit5
[2] in 2018-12 this still fails with Bundle "org.eclipse.jdt.junit5.runtime" not found.


Version: 2019-03 (4.11)
Build id: I20190214-1800
Comment 1 Vikas Chandra CLA 2019-02-15 04:38:40 EST
>>PDE also add a JUnit5 classpath container

PDE adds that if you create a plugin project and create a JUnit5 test case.
Comment 2 Julian Honnen CLA 2019-02-15 05:00:44 EST
(In reply to Vikas Chandra from comment #1)
> PDE adds that if you create a plugin project and create a JUnit5 test case.

Ah right. That didn't work, because I had already added the jupiter.api dependency manually. With a fresh plugin PDE inserts the following:

Require-Bundle: org.junit,
 org.junit.jupiter.api,
 org.junit.jupiter.engine,
 org.junit.jupiter.migrationsupport,
 org.junit.jupiter.params,
 org.junit.platform.commons,
 org.junit.platform.engine,
 org.junit.platform.launcher,
 org.junit.platform.runner,
 org.junit.platform.suite.api,
 org.junit.vintage.engine,
 org.hamcrest.core,
 org.opentest4j,
 org.apiguardian


Even though, I think my point still stands. Is it correct to add plugin dependencies that are only required at runtime and only in eclipse?
Comment 3 Vikas Chandra CLA 2019-02-15 05:16:31 EST
>>that are only required at runtime and only in eclipse?

With JUnit4, it worked in a similar manner.
Comment 4 Julian Honnen CLA 2019-02-15 05:21:32 EST
True, but api & runtime were a single dependency then. Splitting that up was a major motivation for JUnit5 as I understand. Forcing tests to be dependent on the runtime feels like a step backwards IMHO.
Comment 5 Martin Huter CLA 2019-02-25 08:19:25 EST
(In reply to Julian Honnen from comment #2)
> (In reply to Vikas Chandra from comment #1)
> > PDE adds that if you create a plugin project and create a JUnit5 test case.
> 
> Ah right. That didn't work, because I had already added the jupiter.api
> dependency manually. With a fresh plugin PDE inserts the following:
> 
> Require-Bundle: org.junit,
>  org.junit.jupiter.api,
>  org.junit.jupiter.engine,
>  org.junit.jupiter.migrationsupport,
>  org.junit.jupiter.params,
>  org.junit.platform.commons,
>  org.junit.platform.engine,
>  org.junit.platform.launcher,
>  org.junit.platform.runner,
>  org.junit.platform.suite.api,
>  org.junit.vintage.engine,
>  org.hamcrest.core,
>  org.opentest4j,
>  org.apiguardian
> 
> 
> Even though, I think my point still stands. Is it correct to add plugin
> dependencies that are only required at runtime and only in eclipse?

I just tried it out with 2018-12 RCP and just needed " org.junit.platform.commons".
In addition to Junit 4 (org.junit) and Junit 5 (org.junit.jupiter.api as Import-Package) as described in https://wiki.eclipse.org/Tycho/How_Tos/JUnit5
Comment 6 Martin Huter CLA 2019-02-25 09:21:32 EST
(In reply to Martin Huter from comment #5)
> (In reply to Julian Honnen from comment #2)
> > (In reply to Vikas Chandra from comment #1)
> > > PDE adds that if you create a plugin project and create a JUnit5 test case.
> > 
> > Ah right. That didn't work, because I had already added the jupiter.api
> > dependency manually. With a fresh plugin PDE inserts the following:
> > 
> > Require-Bundle: org.junit,
> >  org.junit.jupiter.api,
> >  org.junit.jupiter.engine,
> >  org.junit.jupiter.migrationsupport,
> >  org.junit.jupiter.params,
> >  org.junit.platform.commons,
> >  org.junit.platform.engine,
> >  org.junit.platform.launcher,
> >  org.junit.platform.runner,
> >  org.junit.platform.suite.api,
> >  org.junit.vintage.engine,
> >  org.hamcrest.core,
> >  org.opentest4j,
> >  org.apiguardian
> > 
> > 
> > Even though, I think my point still stands. Is it correct to add plugin
> > dependencies that are only required at runtime and only in eclipse?
> 
> I just tried it out with 2018-12 RCP and just needed "
> org.junit.platform.commons".
> In addition to Junit 4 (org.junit) and Junit 5 (org.junit.jupiter.api as
> Import-Package) as described in https://wiki.eclipse.org/Tycho/How_Tos/JUnit5

Need to report, my approach only worked for junit5 plugin tests to execute just junit 5 tests you need that configuration
Comment 7 Julian Honnen CLA 2019-02-25 14:31:08 EST
*** Bug 544790 has been marked as a duplicate of this bug. ***
Comment 8 Gunnar Wagenknecht CLA 2019-02-26 04:09:55 EST
FWIW, I'm not sure what has changed but with Tycho 1.3 this seems to be broken as well (bug 544789).

However, adding required bundles still produces an error because Tycho Surefire brings in its own runtime.
Comment 9 Martin Huter CLA 2019-02-26 04:12:39 EST
I figured out a currently running environment for tests:

Require-Bundle: org.junit,
 org.junit.platform.commons,
 org.junit.platform.engine,
 org.opentest4j
Import-Package: org.junit.jupiter.api,
 org.junit.jupiter.engine.execution,
 org.junit.jupiter.params,
 org.junit.jupiter.params.provider

As far as I can see, Tycho is able to run the tests in a maven build, and eclipse is able to run junit 4 and 5 tests.
Comment 10 Gunnar Wagenknecht CLA 2019-02-26 04:19:55 EST
(In reply to Martin Huter from comment #9)

> As far as I can see, Tycho is able to run the tests in a maven build, and
> eclipse is able to run junit 4 and 5 tests.

Just tried with adapting the sample project posted in bug 544789:
https://bugs.eclipse.org/bugs/attachment.cgi?id=277681

Doesn't work, i.e. Tycho is not finding any tests. Can you share your target platform, JRE and Tycho version as well?
Comment 11 Martin Huter CLA 2019-02-26 04:46:23 EST
(In reply to Gunnar Wagenknecht from comment #10)
> (In reply to Martin Huter from comment #9)
> 
> > As far as I can see, Tycho is able to run the tests in a maven build, and
> > eclipse is able to run junit 4 and 5 tests.
> 
> Just tried with adapting the sample project posted in bug 544789:
> https://bugs.eclipse.org/bugs/attachment.cgi?id=277681
> 
> Doesn't work, i.e. Tycho is not finding any tests. Can you share your target
> platform, JRE and Tycho version as well?

We are currently using Tycho 1.3, it is not a pom-less build.

mvn --version   
Apache Maven 3.6.0 (NON-CANONICAL_2018-11-06T03:14:22+01:00_root; 2018-11-06T03:14:22+01:00)
Maven home: /opt/maven
Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-8-openjdk/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux", version: "4.19.20-1-manjaro", arch: "amd64", family: "unix"

I tried out the repository from the referenced bug, but in my case the my.plugin.tests could not resolve its host bundle. In the maven build and just failed - so I was not able to reproduce the behavior of not found tests.
Comment 12 Gunnar Wagenknecht CLA 2019-02-26 05:44:01 EST
(In reply to Martin Huter from comment #11)
> I tried out the repository from the referenced bug, but in my case the
> my.plugin.tests could not resolve its host bundle. In the maven build and
> just failed - so I was not able to reproduce the behavior of not found tests.

Did you run the root pom? I just tried locally with downloading the ZIP and it worked for me (Mac).
Comment 13 Martin Huter CLA 2019-02-28 07:38:27 EST
I discovered a new flaw.
We now got rid of all junit4 tests, and I wanted to remove the dependency to it now, but Eclipse is not showing Junit Plug-In tests anymore for test files.

Should I open a new Ticket for this? - Couldn't find a matching one.
Comment 14 Julian Honnen CLA 2019-02-28 08:09:32 EST
(In reply to Martin Huter from comment #13)
> I discovered a new flaw.
> We now got rid of all junit4 tests, and I wanted to remove the dependency to
> it now, but Eclipse is not showing Junit Plug-In tests anymore for test
> files.
> 
> Should I open a new Ticket for this? - Couldn't find a matching one.

Does it work when you add the dependencies listed in comment 2?
Comment 15 Martin Huter CLA 2019-02-28 08:11:28 EST
(In reply to Julian Honnen from comment #14)
> (In reply to Martin Huter from comment #13)
> > I discovered a new flaw.
> > We now got rid of all junit4 tests, and I wanted to remove the dependency to
> > it now, but Eclipse is not showing Junit Plug-In tests anymore for test
> > files.
> > 
> > Should I open a new Ticket for this? - Couldn't find a matching one.
> 
> Does it work when you add the dependencies listed in comment 2?

Yes, but the first dependency org.junit is the junit4 dependency that I want to get rid of, because I'd like to eradicate the possibility to create junit4 tests.
Comment 16 Julian Honnen CLA 2019-02-28 09:26:09 EST
(In reply to Martin Huter from comment #15)
> Yes, but the first dependency org.junit is the junit4 dependency that I want
> to get rid of, because I'd like to eradicate the possibility to create
> junit4 tests.

You don't need to create a new bug, that's exactly this issue.

@Vikas: I think we should add necessary junit runtime dependencies the PDE cp container automatically when a plugin as an API dependency. WDYT?
Comment 17 Vikas Chandra CLA 2019-03-01 01:23:32 EST
>>@Vikas: I think we should add necessary junit runtime dependencies the PDE cp >>container 

If you create a patch, we can try that out. If no regression, then we should be good.
Comment 18 Eclipse Genie CLA 2019-03-15 10:23:48 EDT
New Gerrit change created: https://git.eclipse.org/r/138829
Comment 19 Julian Honnen CLA 2019-03-15 10:33:52 EDT
(In reply to Julian Honnen from comment #16)
> (In reply to Martin Huter from comment #15)
> > Yes, but the first dependency org.junit is the junit4 dependency that I want
> > to get rid of, because I'd like to eradicate the possibility to create
> > junit4 tests.
> 
> You don't need to create a new bug, that's exactly this issue.

I was wrong, tycho also needs an org.junit dependency to execute tests. Otherwise it fails with

java.lang.ClassNotFoundException: org.junit.runners.model.RunnerBuilder cannot be found by org.eclipse.tycho.surefire.osgibooter_1.3.0

@Martin, can you file a bug with tycho for that?
Comment 21 Julian Honnen CLA 2019-04-03 03:04:16 EDT
Verified on I20190402-1800
Comment 22 Dani Megert CLA 2019-04-06 04:13:55 EDT
This caused bug 546114. The change got reverted (see bug 546114 comment 7 ).
Comment 23 Julian Honnen CLA 2019-04-08 02:35:03 EDT
(In reply to Dani Megert from comment #22)
> This caused bug 546114. The change got reverted (see bug 546114 comment 7 ).
> A cycle was detected in the build path of project 'org.eclipse.e4.ui.tests.css.core'.
> The cycle consists of projects {
> org.junit.jupiter.api, 
> org.junit.jupiter.engine, 
> Eclipse ZRH ALL Tests, 
> org.eclipse.jdt.text.tests, 
> org.eclipse.jdt.ui.tests, 
> org.eclipse.jdt.junit5.runtime, 
> org.junit.jupiter.migrationsupport, 
> org.junit.jupiter.params, 
> org.eclipse.jdt.ui.tests.refactoring, 
> org.eclipse.e4.ui.tests.css.core, 
> org.eclipse.test}

Moving out of M1. Looks like the JUnit5 plugins themselves get the "runtime" dependencies added, which is obviously wrong.
Comment 24 Eclipse Genie CLA 2019-04-30 10:03:21 EDT
New Gerrit change created: https://git.eclipse.org/r/141399
Comment 25 Vikas Chandra CLA 2019-05-14 02:48:58 EDT
Lets take up this in 4.13. Change in RequiredPluginsClasspathContainer is risky at very close to last iteration of 4.12.
Comment 26 Julian Honnen CLA 2019-05-20 03:09:50 EDT
Incrementing prio - apparently it's not possible to create a test bundle that works in both PDE and Tycho without manually messing with the classpath.

See Bug 534587: Tycho can't execute test from a bundle with runtime dependencies: 

org.apache.maven.surefire.util.SurefireReflectionException: java.util.ServiceConfigurationError: org.junit.platform.engine.TestEngine: Provider org.junit.jupiter.engine.JupiterTestEngine not a subtype

The suggested workaround is to manually add JDT's Junit5 CP container to the .classpath
Comment 28 Julian Honnen CLA 2019-06-25 06:43:20 EDT
(In reply to Eclipse Genie from comment #27)
> Gerrit change https://git.eclipse.org/r/141399 was merged to [master].
> Commit:
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=6c43f9a21e1984cb3dda485637b6f5b9c2d7a7c2

Dani, please try the next I-Build with your large workspace.
Comment 29 Vikas Chandra CLA 2019-07-09 04:45:58 EDT
Julian, Can you please verify this bug?
Comment 30 Julian Honnen CLA 2019-07-09 06:36:26 EDT
The example from comment 0 works in I20190708-1800.