Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 567358

Summary: [JUnit 5] Add JUnit 5.7 bundles to Orbit
Product: [Tools] Orbit Reporter: Noopur Gupta <noopur_gupta>
Component: bundlesAssignee: Roland Grunberg <rgrunber>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: P3 CC: akurtakov, carsten.hammer, daniel_megert, rgrunber
Version: unspecified   
Target Milestone: 2020-12 M2   
Hardware: All   
OS: All   
See Also: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/169982
https://git.eclipse.org/c/orbit/orbit-recipes.git/commit/?id=9c0f989f9eb2b85cc5290e5b420c9e28a87b21d5
Whiteboard:
Bug Depends on:    
Bug Blocks: 567357    

Description Noopur Gupta CLA 2020-09-25 09:43:49 EDT
We should replace JUnit 5.6 JARs with JUnit 5.7 JARs for bug 567357.
Comment 1 Noopur Gupta CLA 2020-09-25 09:45:47 EDT
Roland, will you be able to update the JUnit 5 JARs in Orbit this time and/or document the steps with the new process so that others can pick this up when you are not available?
Comment 2 Roland Grunberg CLA 2020-09-25 11:43:36 EDT
I'll try to get to this eventually, but in the event I don't here's the process I generally use. Let me know if there's a better place to put such info.

The change itself is usually the easiest part since it can effectively be done by a search/replace. In fact, that will more than likely work. The difficult part is confirming that the dependencies haven't changed. It can happen so it's good to verify.

https://git.eclipse.org/r/c/orbit/orbit-recipes/+/156762 is an example of a pretty straightforward change. We went from 5.5.1 to 5.6.0.

org.apiguardian, and org.opentest4j did not need to be upgraded but junit dependence went from 4.12 to 4.13 [1]. So, how did I know this was required ? Here's the shell script I usually run :

(Ideal to run in an empty directory)

for art in \
  junit-platform-{commons,engine,launcher,runner,suite-api}; do
  for v in "1.6.0" "1.7.0"; do
    curl -s -O https://search.maven.org/remotecontent?filepath=org/junit/platform/${art}/${v}/${art}-${v}.pom
  done
  diff -u ${art}*
done

art=junit-vintage-engine
for v in "5.6.0" "5.7.0"; do
  curl -s -O https://search.maven.org/remotecontent?filepath=org/junit/vintage/${art}/${v}/${art}-${v}.pom
done
diff -u ${art}*

for art in \
  junit-jupiter-{api,engine,migrationsupport,params}; do
  for v in "5.6.0" "5.7.0"; do
    curl -s -O https://search.maven.org/remotecontent?filepath=org/junit/jupiter/${art}/${v}/${art}-${v}.pom
  done
  diff -u ${art}*
done

This essentially compares the pom files of the old and new artifacts and is a good way to determine what may have changed.

There may be a few false positives, like pom dependencies being modified around, but generally, you're looking for version numbers of dependency artifacts that are changed/added/removed . Obviously ignore the version numbers of the poms themselves (eg. 5.5.1 -> 5.6.0).

So you would just do the same thing but for 5.6.0 -> 5.7.0. From running the script above on the new version I think a search/replace should suffice.

Once a change gets pushed, (and CQs get updated acccordingly in ip_log.xml) gerrit-orbit-recipes should generate a p2 repo that you can even try out with the change before it gets merged). After it's merged you can try it out in the next I-build, until a milestone release.

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=558875#c5
Comment 3 Carsten Hammer CLA 2020-09-25 13:07:37 EDT
Maybe at one point a recent version of hamcrest library can be added?

see http://hamcrest.org/JavaHamcrest/distributables (2.2)

Orbit has 1.3.0

see Bug 562290
Comment 4 Noopur Gupta CLA 2020-09-28 04:48:41 EDT
(In reply to Roland Grunberg from comment #2)
> I'll try to get to this eventually, but in the event I don't here's the
> process I generally use. 

Thanks, Roland. Assigning it to you for now. Please revert if you won't be able to work on it.

> Let me know if there's a better place to put such info.

You can add the steps/details to a new Eclipse Wiki page.
Comment 5 Eclipse Genie CLA 2020-09-28 10:55:58 EDT
New Gerrit change created: https://git.eclipse.org/r/c/orbit/orbit-recipes/+/169982
Comment 6 Roland Grunberg CLA 2020-09-30 09:34:48 EDT
Are you able to verify that https://ci.eclipse.org/orbit/job/gerrit-orbit-recipes/1435/artifact/releng/repository/target/repository/ (containing JUnit 5.7.0) works as you would expect.

I think if so, we can merge and expect it to be in M1.
Comment 7 Noopur Gupta CLA 2020-10-01 08:16:53 EDT
(In reply to Roland Grunberg from comment #6)
> Are you able to verify that
> https://ci.eclipse.org/orbit/job/gerrit-orbit-recipes/1435/artifact/releng/repository/target/repository/
> (containing JUnit 5.7.0) works as you would expect.
> 
> I think if so, we can merge and expect it to be in M1.

I replaced the 5.6 JARs with 5.7 JARs from above link in IDE plugins folder but I am seeing some errors on creating/running JUnit 5 test in a modular project. It could be some setup issue as well. I need more time for checking.
Comment 8 Roland Grunberg CLA 2020-10-14 17:22:08 EDT
Any update here ? Just making sure not merging yet is the right call.
Comment 9 Noopur Gupta CLA 2020-10-15 03:19:19 EDT
(In reply to Roland Grunberg from comment #8)
> Any update here ? Just making sure not merging yet is the right call.

These JARs don't seem to work on mac.

I have been using Windows earlier for this.

Let me see if I can access a Windows machine now or if there's a way to get the mac compatible JARs from Gerrit repo.
Comment 10 Noopur Gupta CLA 2020-10-15 04:08:54 EDT
Talked to Sravan: these JARs are not signed so won't be picked up on mac. 

I will verify on a Wondows machine.
Comment 11 Noopur Gupta CLA 2020-10-20 03:16:00 EDT
(In reply to Noopur Gupta from comment #10)
> Talked to Sravan: these JARs are not signed so won't be picked up on mac. 
> 
> I will verify on a Windows machine.
Sorry, this is taking longer. If anyone else has a Windows/Linux machine and can verify the JARs sooner then we can release the patch. I hope to verify this in this week.
Comment 12 Noopur Gupta CLA 2020-10-23 05:56:40 EDT
I tried the JARs from https://ci.eclipse.org/orbit/job/gerrit-orbit-recipes/1484/artifact/releng/repository-all/target/repository/plugins/ on a Windows machine and I am still getting the exceptions in Error log on trying to create a JUnit 5 test. The Eclipse Installation Details > Plug-ins doesn't show the new JARs loaded into Eclipse.

Roland, can you please check if you see the same and if there's any difference in 5.6 vs 5.7 JARs which can cause this? 

I just replaced the 5.6 JARs with 5.7 JARs in the Eclipse plugins (or dropins) folder which used to work earlier.
Comment 13 Roland Grunberg CLA 2020-10-23 10:12:51 EDT
(In reply to Noopur Gupta from comment #12)
> I tried the JARs from
> https://ci.eclipse.org/orbit/job/gerrit-orbit-recipes/1484/artifact/releng/
> repository-all/target/repository/plugins/ on a Windows machine and I am
> still getting the exceptions in Error log on trying to create a JUnit 5
> test. The Eclipse Installation Details > Plug-ins doesn't show the new JARs
> loaded into Eclipse.
> 
> Roland, can you please check if you see the same and if there's any
> difference in 5.6 vs 5.7 JARs which can cause this? 
> 
> I just replaced the 5.6 JARs with 5.7 JARs in the Eclipse plugins (or
> dropins) folder which used to work earlier.

I'll try this out. The jars not even showing up indicates this could be a pure installation issue. I'll try on Linux and maybe I can get a hold of a Windows box if they work there.
Comment 14 Roland Grunberg CLA 2020-10-23 12:20:42 EDT
Tested on Linux with I20201023-0250 and it seems to work there. I assume no changes needed to be made in jdt.ui to support the new jars.

- I just placed the JUnit 5.7.0/1.7.0 jars in dropins (test4j and apiguardian are not being updated so they were already in place) and all the dropins content was RESOLVED on startup
- Created a new JUnit 5 testcase and confirmed the referenced libraries were from 5.7.0/1.7.0
- A basic launch with the Junit 5 test runner ran without issue

I can try getting access to a windows box but not sure what could be different.
Comment 15 Noopur Gupta CLA 2020-10-25 06:05:14 EDT
(In reply to Roland Grunberg from comment #14)
> Tested on Linux with I20201023-0250 and it seems to work there. I assume no
> changes needed to be made in jdt.ui to support the new jars.
> 
> - I just placed the JUnit 5.7.0/1.7.0 jars in dropins (test4j and
> apiguardian are not being updated so they were already in place) and all the
> dropins content was RESOLVED on startup

Did you also delete the corresponding 5.6/1.6 jars from plugins? I think I used the Oct 16 I-build on Windows 10. I will try with the latest. Not sure what else could be the difference other than the OS. It will be good to find out the reason for the future version updates.

If it's working for you on Linux then we can release the patch and test the I-build on other platforms.
Comment 16 Roland Grunberg CLA 2020-10-26 00:15:16 EDT
(In reply to Noopur Gupta from comment #15) 
> Did you also delete the corresponding 5.6/1.6 jars from plugins? I think I
> used the Oct 16 I-build on Windows 10. I will try with the latest. Not sure
> what else could be the difference other than the OS. It will be good to find
> out the reason for the future version updates.

Initially I just placed the jars in dropins keeping the old ones in place, but now I've also removed the older 5.6.0/1.6.0 ones along with the additions and that works. I can still try to test on Windows to see if it works there, unless you confirm a newer build succeeds.

> If it's working for you on Linux then we can release the patch and test the
> I-build on other platforms.

Sure, I can push the change in Orbit for the M2 contribution. If for some reason, there are issues on Windows or some other regression, could JDT just use Orbit's M1 until it gets resolved ?
Comment 17 Noopur Gupta CLA 2020-10-27 07:54:50 EDT
This has something to do with the org.opentest4j_1.2.0.v20190826-0900 and org.apiguardian_1.1.0.v20190826-0900 JARs.

If I keep them in plugins then it works. If I remove them from plugins and place them in dropins then I see the problem. This was the other difference between our steps. It needs investigation if reproducible on your system as well.

Without touching these two JARs, the rest of them work fine on Windows in dropins. So, we can release the patch.
Comment 18 Noopur Gupta CLA 2020-10-27 09:58:40 EDT
(In reply to Roland Grunberg from comment #16)
> Sure, I can push the change in Orbit for the M2 contribution. If for some
> reason, there are issues on Windows or some other regression, could JDT just
> use Orbit's M1 until it gets resolved ?
Yes, we can do that.
Comment 20 Roland Grunberg CLA 2020-10-27 12:15:32 EDT
I'll verify the issue with using opentest4j and apiguardian in dropins, though these bundles have not needed an update as part of this contribution.
Comment 21 Roland Grunberg CLA 2020-10-27 19:23:12 EDT
I would think this is an issue with dropins itself.

apiguardian and opentest4j are unchanged (unlike the other JUnit bundles). When moved from plugins/ to dropins/ they do get correctly detected but are never added as IUs (hence why they're not visible in the host OSGi console). I think this happens because once added, it's determined that they already exist as part of the root profile, since removing them from plugins/ doesn't really get rid of them.

As an experiment, I removed the JUnit 5.6 bundles and placed the new JUnit 5.7 ones in dropins/ . I then moved apiguardian and opentest4j from plugins/ to dropins and bumped their manifest Bundle-Version qualifier 'v20190826-0900 -> v20190826-0901' (as well as in the file name). The result was that everything loaded as expected. I wasn't able to test junit 5 launch because it detected my modification to the signed manifest but it does show this seems more like a dropins limitation.
Comment 22 Noopur Gupta CLA 2020-10-28 05:10:05 EDT
Thanks for checking that, Roland. I was able to see this dropins limitation with the mentioned JARs even when JUnit 5.7 is not involved. 

Closing the bug. Please add the steps (CQ creation, comment #2, etc.) involved in this process to a wiki page and add the wiki link here when you get some time.
Comment 23 Roland Grunberg CLA 2020-10-28 21:59:28 EDT
I've created the document for this : https://wiki.eclipse.org/JUnit5_Update_Process .
Comment 24 Noopur Gupta CLA 2020-10-29 04:21:56 EDT
(In reply to Roland Grunberg from comment #23)
> I've created the document for this :
> https://wiki.eclipse.org/JUnit5_Update_Process .

Thanks, Roland.