Comment 1 Thomas Schindl 2018-04-10 03:18:08 EDT

So the javax.annotation disaster is back! This is a very critical bug we need to fix for Photon otherwise we won't be able to run on > Java 11 - we already have trouble compiling against Java 9 because of the module with javax.annotation is not exported by default :-(

The only solution is to add back javax.annotation-bundle (maybe with a EE for JavaSE-1.9?) the real problem although is stuff who does not do an import-package for javax.annotation as it won't consult the repackaged version.

The best idea I have to get this fixed is that we provide an adapter-hook. Tom is there anything OSGi has planned to do? It's not only javax.annotation that is gone in JavaSE-11.

Comment 2 Dani Megert 2018-04-10 05:36:57 EDT

(In reply to Thomas Schindl from comment #1)
> It's not only javax.annotation that is gone in JavaSE-11.

Correct. For details see

https://bugs.openjdk.java.net/browse/JDK-8189188
http://openjdk.java.net/jeps/320

Comment 3 Lakshmi P Shanmugam 2018-04-10 07:53:48 EDT

(In reply to Dr. Michael Paus from comment #0)
> The reason is that Eclipse by itself tries to run with Java
> 11, which I have installed on my system but which is not the default. When I
> type "java -version" in a shell I get Java 10 and not 11.
Have you installed the JRE or JDK for Java 10 and Java 11?

Comment 4 Dr. Michael Paus 2018-04-10 07:56:26 EDT

As a developer I have the JDK installed of course :-)

Comment 5 Thomas Watson 2018-04-10 08:52:18 EDT

(In reply to Thomas Schindl from comment #1)
> So the javax.annotation disaster is back! This is a very critical bug we
> need to fix for Photon otherwise we won't be able to run on > Java 11 - we
> already have trouble compiling against Java 9 because of the module with
> javax.annotation is not exported by default :-(
> 
> The only solution is to add back javax.annotation-bundle (maybe with a EE
> for JavaSE-1.9?) the real problem although is stuff who does not do an
> import-package for javax.annotation as it won't consult the repackaged
> version.
> 
> The best idea I have to get this fixed is that we provide an adapter-hook.
> Tom is there anything OSGi has planned to do? It's not only javax.annotation
> that is gone in JavaSE-11.

What would an adaptor hook do here?  Anything doing require-bundle on org.eclipse.osgi and depending on it giving them stuff from javax package space is in error and should be fixed.  I agree, it does look like we should add back the javax.annotation bundle.  If the community can not react to fixing their bugs with require-bundle and we think we need to cater to this issue you could consider making the javax.annotation bundle a system.bundle fragment.  It would seem that you should make the required osgi.ee JavaSE version 11 though.  No need for an adaptor hook for that though.  But I think we should avoid doing that since it is a work around to allow others to keep their buggy require-bundle for javax.* packages.

From the OSGi perspective the framework must report the correct set of packages available from the running platform and make them available for resolving Import-Package.  Using system.bundle fragments to export javax.* stuff would be considered an anti-pattern I think.

One last thing on the required osgi.ee.  If we do make that JavaSE version 11 I am unsure how p2 will react when try to install a feature with that bundle on Java 8,9,10.  Can features have optional bundles?

Comment 6 Thomas Schindl 2018-04-10 09:05:08 EDT

Well an adapter-hook could act as the last resort safenet. 

Suppose one has written a bundle using JavaSE-1.8 as the EE and has NOT added any import-package for javax.annotation (because everything works fine there, no warning, no compilation problem, ...). 

To mitigate the problem when running in JavaSE-11 we could add an adapter-hook who IIRC is consulted when the all classloading attempts of Equinox failed and do the following:
* check that we are running JavaSE-11 or higher
* log a warning telling that one should upate the bundle importing javax.annotation
* locate javax.annotation bundle and load the class from there

Comment 7 Thomas Watson 2018-04-10 10:33:06 EDT

(In reply to Thomas Schindl from comment #6)
> Well an adapter-hook could act as the last resort safenet. 
> 
> Suppose one has written a bundle using JavaSE-1.8 as the EE and has NOT
> added any import-package for javax.annotation (because everything works fine
> there, no warning, no compilation problem, ...). 
> 
> To mitigate the problem when running in JavaSE-11 we could add an
> adapter-hook who IIRC is consulted when the all classloading attempts of
> Equinox failed and do the following:
> * check that we are running JavaSE-11 or higher
> * log a warning telling that one should upate the bundle importing
> javax.annotation
> * locate javax.annotation bundle and load the class from there

That should be possible, I would not want that bundle to live in Equinox.  If this is important for the Platform then the bundle can exist there.  Platform is the thing that needs javax.annotation (not Equinox) so I think Platform should handle this if needed.

Comment 8 Thomas Watson 2018-04-10 11:32:02 EDT

(In reply to Thomas Watson from comment #7)
> That should be possible, I would not want that bundle to live in Equinox. 
> If this is important for the Platform then the bundle can exist there. 
> Platform is the thing that needs javax.annotation (not Equinox) so I think
> Platform should handle this if needed.

We discussed this at the PMC call today.  We decided not to do the class loading trick here.  An announcement should be made to make others aware of a possible breakage for bundles that have bugs in their require-bundle usages to access javax.annotation package from the JRE.  They may break when moving to Java 11.

We should try to pull in the latest javax.annotation bundle version 1.2.0 from orbit and use that from the platform.  It should be added to some platform feature.  Someone needs to test that this works when running on Java 11.  We also need to revisit the issues that can happen with the bundle being present when running on Java 8 which still includes the javax.annotation package.  IIRC there were some possible linkage errors that could occur if bundles got wired to inconsistent providers for the javax.annotation package (the JRE provided one vs. the javax.annotation bundle provided one).  The orbit bundle javax.annotation version 1.2.0 requires the system.bundle:

Require-Bundle: system.bundle

In theory this should clear up any class space consistency issues when running on java 8 for bundles being wired to different providers of javax.annotation.  The framework will treat the javax.annotation packages from the javax.annotation bundle as split packages and delegate to the JRE provided package first.  This should make for a consistent class space for the bundles even when they are wired to different providers of the package.

Moving to releng to pull in the javax.annotation bundle.

Comment 9 Lakshmi P Shanmugam 2018-04-11 02:23:55 EDT

(In reply to Dr. Michael Paus from comment #4)
> As a developer I have the JDK installed of course :-)

Can you please provide more information about your java installation?
Did you install both JDK 10 & 11 using the dmg file or tar.gz? How did you set Java 10 as the default? Have you added Java 10 & 11 locations to the PATH?

Eclipse will consult the eclipse.ini file for the java location and if not found looks in the system path variable. Eclipse doesn't consult the JAVA_HOME environment variable. 

As a workaround, you can explicitly point to Java 10 in the eclipse.ini - Please see https://wiki.eclipse.org/Eclipse.ini

Comment 10 Dr. Michael Paus 2018-04-11 02:57:11 EDT

(In reply to Lakshmi Shanmugam from comment #9)
> (In reply to Dr. Michael Paus from comment #4)
> Did you install both JDK 10 & 11 using the dmg file or tar.gz?

I installed them both via the DMG files provided by Oracle.

> How did you set Java 10 as the default?

I set this in my .bash_profile file

export JAVA_HOME="$(/usr/libexec/java_home -version 10)"

> Have you added Java 10 & 11 locations to the PATH?

I add $JAVA_HOME/bin to the PATH.

Calling java -version in a shell returns

$ java -version
java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

so the PATH is set up correctly.

In the system setting panel Java 11 is shown though but I thought that this is
only relevant for the browser plugin (applets, webstart,...) which I have not activated anyway. When you install from the DMG files it cannot be prevented to also install the separate JRE (unless you later delete it manually).

> As a workaround, you can explicitly point to Java 10 in the eclipse.ini -
> Please see https://wiki.eclipse.org/Eclipse.ini

I have done that already but I find it annoying and a bad user experience if I have to fiddle around with the ini file and not every user will probably know what to do.

So, it all boils down to the question what mechanism Eclipse uses to find Java. It neither uses JAVA_HOME nor does it use the PATH.

Comment 11 Lakshmi P Shanmugam 2018-04-11 05:47:43 EDT

(In reply to Dr. Michael Paus from comment #10)

Thanks for the details.

> So, it all boils down to the question what mechanism Eclipse uses to find
> Java. It neither uses JAVA_HOME nor does it use the PATH.
As per https://wiki.eclipse.org/FAQ_How_do_I_run_Eclipse%3F#Find_the_JVM, Eclipse uses the system PATH variable to find the JVM. It's strange that it finds Java 11 which is not in the system PATH.

Comment 12 Sravan Kumar Lakkimsetti 2018-04-11 07:11:30 EDT

I am able to reproduce this on my Mac 10.13.3

In Mac launcher uses this command to retrieve JAVA_HOME
   /usr/libexec/java_home -a x86_64

We do not use JAVA_HOME environment variable at all in cocoa launcher

This command returns latest java version's path(in this case path to java 11) though default is set to java 8. 

One solution could be modify getJavaHome() method (link http://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/features/org.eclipse.equinox.executable.feature/library/cocoa/eclipseCocoa.c#n371) to use JAVA_HOME from the environment if defined, else continue with old behaviour. 

I will not be able to take-up this immediately. If any one from community wants to take up please go ahead and provide me a Gerrit patch for review.

I raised bug 533462 to trace launcher problem

Comment 13 Dani Megert 2018-04-17 05:19:19 EDT

(In reply to Thomas Watson from comment #8)
> We should try to pull in the latest javax.annotation bundle version 1.2.0
> from orbit and use that from the platform.

Tom, something else seems to go wrong here. We actually do ship javax.annotation_1.2.0.v201602091430.jar with our builds, including 4.7.3a.

Can you have a look?

Comment 14 Sravan Kumar Lakkimsetti 2018-04-17 06:22:41 EDT

Sarika and I tested javax.annotation problem on 4.7.3, 4.7.3a and 4.8 on Mac and Windows. Here are the results

         Windows    Mac OSX
4.8      Pass       Pass
4.7.3a   Fail       Fail
4.7.3    Fail       Fail

Javax.annotation problem is reproduced in 4.7.x. Not in 4.8

Comment 15 Dani Megert 2018-04-17 08:06:17 EDT

(In reply to Sravan Kumar Lakkimsetti from comment #14)
> Sarika and I tested javax.annotation problem on 4.7.3, 4.7.3a and 4.8 on Mac
> and Windows. Here are the results
> 
>          Windows    Mac OSX
> 4.8      Pass       Pass
> 4.7.3a   Fail       Fail
> 4.7.3    Fail       Fail
> 
> Javax.annotation problem is reproduced in 4.7.x. Not in 4.8

Tom, it started to work in Photon with M4. Could it be that your fix for bug 527353 is making it work?

Comment 16 Thomas Watson 2018-04-17 08:44:08 EDT

(In reply to Dani Megert from comment #15)
> (In reply to Sravan Kumar Lakkimsetti from comment #14)
> > Sarika and I tested javax.annotation problem on 4.7.3, 4.7.3a and 4.8 on Mac
> > and Windows. Here are the results
> > 
> >          Windows    Mac OSX
> > 4.8      Pass       Pass
> > 4.7.3a   Fail       Fail
> > 4.7.3    Fail       Fail
> > 
> > Javax.annotation problem is reproduced in 4.7.x. Not in 4.8
> 
> Tom, it started to work in Photon with M4. Could it be that your fix for bug
> 527353 is making it work?

I guess it is possible.  There is no profile for Java 11 in 4.7.3, and that version of Equinox does not auto-detect VM packages.  That means we fall back to the latest Java profile available, which is JavaSE-9 and JavaSE-9 includes the javax.annotation package.  So when the framework is launched it exports the javax.annotation package, but there is no content for it.  If bundles get wired to that export they will get NoClassDefFoundErrors.

I'll try to have a look today to confirm the issue.  If that is the issue, I am unsure why the bundle with org.eclipse.e4.core.internal.di.InjectorImpl is getting wired to the system.bundle export of javax.annotation.  I would think it would get wired to the versioned one if we include a javax.annotation bundle already.

Comment 17 Thomas Watson 2018-04-17 12:35:41 EDT

I can confirm that the fix for bug 527353 is what fixes this in Photon.  The interesting part is why the resolver picks the system.bundle export of javax.annotation package over the versioned one exported by the javax.annotation bundle.

This is because of the spec'ed preference order specified by the OSGi core specification:

https://osgi.org/specification/osgi.core/7.0.0/framework.module.html#framework.module.resolvingprocess

At the bottom of this section is the following:

The following list defines the preferences, if multiple choices are possible, in order of decreasing priority:

    A resolved exporter must be preferred over an unresolved exporter.

    An exporter with a higher version is preferred over an exporter with a lower version.

    An exporter with a lower bundle ID is preferred over a bundle with a higher ID.

The first rule applies here.  Since the system bundle is always resolved first it becomes the preferred provider when we are resolving the rest of the bundles.

I see two options if we want to provide a fix for Oxygen

1) Backport the fix for bug 527353 to Oxygen
2) Add a JavaSE-11.profile to org.eclipse.osgi for Oxygen

I prefer to backport bug 527353 because it better adapts to when they decide to remove more packages in Java 12.  But first we need to decide if this should be fixed, or just have developers wait for Photon.

Comment 18 Dani Megert 2018-04-17 12:43:00 EDT

(In reply to Thomas Watson from comment #17)
> I prefer to backport bug 527353.

4.7.x is done and Java 11 is still EA. So, nothing planned here.

*** This bug has been marked as a duplicate of bug 527353 ***

Comment 19 Dani Megert 2018-04-18 09:57:17 EDT

(In reply to Dani Megert from comment #18)
> (In reply to Thomas Watson from comment #17)
> > I prefer to backport bug 527353.
> 
> 4.7.x is done and Java 11 is still EA. So, nothing planned here.

Tom, can you think of a workaround that users could apply, e.g. by adding some config file(s) or options to the eclipse.ini?

Comment 20 Thomas Watson 2018-04-18 11:57:36 EDT

(In reply to Dani Megert from comment #19)
> (In reply to Dani Megert from comment #18)
> > (In reply to Thomas Watson from comment #17)
> > > I prefer to backport bug 527353.
> > 
> > 4.7.x is done and Java 11 is still EA. So, nothing planned here.
> 
> Tom, can you think of a workaround that users could apply, e.g. by adding
> some config file(s) or options to the eclipse.ini?

Yes, but it is awkward.  It would involve setting the property `org.osgi.framework.system.packages` to the correct value for the JVM version being run.  For example, you could take the value from the javaSE-9.profile for the property `org.osgi.framework.system.packages` and remove the javax.annotation packages from it and use that value in your config.ini

Comment 21 Dani Megert 2018-04-18 12:00:32 EDT

(In reply to Thomas Watson from comment #20)
> (In reply to Dani Megert from comment #19)
> > (In reply to Dani Megert from comment #18)
> > > (In reply to Thomas Watson from comment #17)
> > > > I prefer to backport bug 527353.
> > > 
> > > 4.7.x is done and Java 11 is still EA. So, nothing planned here.
> > 
> > Tom, can you think of a workaround that users could apply, e.g. by adding
> > some config file(s) or options to the eclipse.ini?
> 
> Yes, but it is awkward.  It would involve setting the property
> `org.osgi.framework.system.packages` to the correct value for the JVM
> version being run.  For example, you could take the value from the
> javaSE-9.profile for the property `org.osgi.framework.system.packages` and
> remove the javax.annotation packages from it and use that value in your
> config.ini

But at least something we could give to our developers and clients. Could you give exact steps?

Comment 22 Thomas Watson 2018-04-18 12:49:03 EDT

(In reply to Dani Megert from comment #21)
> But at least something we could give to our developers and clients. Could
> you give exact steps?

It just involves one step, but it is an ugly step :)

Add this to your config.ini (note that I removed the javax.annotation and javax.transaction packages)

org.osgi.framework.system.packages = \
 javax.accessibility,\
 javax.activation,\
 javax.activity,\
 javax.crypto,\
 javax.crypto.interfaces,\
 javax.crypto.spec,\
 javax.imageio,\
 javax.imageio.event,\
 javax.imageio.metadata,\
 javax.imageio.plugins.bmp,\
 javax.imageio.plugins.jpeg,\
 javax.imageio.spi,\
 javax.imageio.stream,\
 javax.jws,\
 javax.jws.soap,\
 javax.lang.model,\
 javax.lang.model.element,\
 javax.lang.model.type,\
 javax.lang.model.util,\
 javax.management,\
 javax.management.loading,\
 javax.management.modelmbean,\
 javax.management.monitor,\
 javax.management.openmbean,\
 javax.management.relation,\
 javax.management.remote,\
 javax.management.remote.rmi,\
 javax.management.timer,\
 javax.naming,\
 javax.naming.directory,\
 javax.naming.event,\
 javax.naming.ldap,\
 javax.naming.spi,\
 javax.net,\
 javax.net.ssl,\
 javax.print,\
 javax.print.attribute,\
 javax.print.attribute.standard,\
 javax.print.event,\
 javax.rmi,\
 javax.rmi.CORBA,\
 javax.rmi.ssl,\
 javax.script,\
 javax.security.auth,\
 javax.security.auth.callback,\
 javax.security.auth.kerberos,\
 javax.security.auth.login,\
 javax.security.auth.spi,\
 javax.security.auth.x500,\
 javax.security.cert,\
 javax.security.sasl,\
 javax.sound.midi,\
 javax.sound.midi.spi,\
 javax.sound.sampled,\
 javax.sound.sampled.spi,\
 javax.sql,\
 javax.sql.rowset,\
 javax.sql.rowset.serial,\
 javax.sql.rowset.spi,\
 javax.swing,\
 javax.swing.border,\
 javax.swing.colorchooser,\
 javax.swing.event,\
 javax.swing.filechooser,\
 javax.swing.plaf,\
 javax.swing.plaf.basic,\
 javax.swing.plaf.metal,\
 javax.swing.plaf.multi,\
 javax.swing.plaf.nimbus,\
 javax.swing.plaf.synth,\
 javax.swing.table,\
 javax.swing.text,\
 javax.swing.text.html,\
 javax.swing.text.html.parser,\
 javax.swing.text.rtf,\
 javax.swing.tree,\
 javax.swing.undo,\
 javax.tools,\
 javax.xml,\
 javax.xml.bind,\
 javax.xml.bind.annotation,\
 javax.xml.bind.annotation.adapters,\
 javax.xml.bind.attachment,\
 javax.xml.bind.helpers,\
 javax.xml.bind.util,\
 javax.xml.crypto,\
 javax.xml.crypto.dom,\
 javax.xml.crypto.dsig,\
 javax.xml.crypto.dsig.dom,\
 javax.xml.crypto.dsig.keyinfo,\
 javax.xml.crypto.dsig.spec,\
 javax.xml.datatype,\
 javax.xml.namespace,\
 javax.xml.parsers,\
 javax.xml.soap,\
 javax.xml.stream,\
 javax.xml.stream.events,\
 javax.xml.stream.util,\
 javax.xml.transform,\
 javax.xml.transform.dom,\
 javax.xml.transform.sax,\
 javax.xml.transform.stax,\
 javax.xml.transform.stream,\
 javax.xml.validation,\
 javax.xml.ws,\
 javax.xml.ws.handler,\
 javax.xml.ws.handler.soap,\
 javax.xml.ws.http,\
 javax.xml.ws.soap,\
 javax.xml.ws.spi,\
 javax.xml.ws.spi.http,\
 javax.xml.ws.wsaddressing,\
 javax.xml.xpath,\
 org.ietf.jgss,\
 org.omg.CORBA,\
 org.omg.CORBA_2_3,\
 org.omg.CORBA_2_3.portable,\
 org.omg.CORBA.DynAnyPackage,\
 org.omg.CORBA.ORBPackage,\
 org.omg.CORBA.portable,\
 org.omg.CORBA.TypeCodePackage,\
 org.omg.CosNaming,\
 org.omg.CosNaming.NamingContextExtPackage,\
 org.omg.CosNaming.NamingContextPackage,\
 org.omg.Dynamic,\
 org.omg.DynamicAny,\
 org.omg.DynamicAny.DynAnyFactoryPackage,\
 org.omg.DynamicAny.DynAnyPackage,\
 org.omg.IOP,\
 org.omg.IOP.CodecFactoryPackage,\
 org.omg.IOP.CodecPackage,\
 org.omg.Messaging,\
 org.omg.PortableInterceptor,\
 org.omg.PortableInterceptor.ORBInitInfoPackage,\
 org.omg.PortableServer,\
 org.omg.PortableServer.CurrentPackage,\
 org.omg.PortableServer.POAManagerPackage,\
 org.omg.PortableServer.POAPackage,\
 org.omg.PortableServer.portable,\
 org.omg.PortableServer.ServantLocatorPackage,\
 org.omg.SendingContext,\
 org.omg.stub.java.rmi,\
 org.w3c.dom,\
 org.w3c.dom.bootstrap,\
 org.w3c.dom.css,\
 org.w3c.dom.events,\
 org.w3c.dom.html,\
 org.w3c.dom.ls,\
 org.w3c.dom.ranges,\
 org.w3c.dom.stylesheets,\
 org.w3c.dom.traversal,\
 org.w3c.dom.views,\
 org.w3c.dom.xpath,\
 org.xml.sax,\
 org.xml.sax.ext,\
 org.xml.sax.helpers

Comment 23 Dani Megert 2018-04-18 13:18:44 EDT

(In reply to Thomas Watson from comment #22)

So, we could provide

1. a new version of the default/standard config.ini.
2. steps to fix a custom config.ini.

Comment 24 Sravan Kumar Lakkimsetti 2018-04-19 05:15:54 EDT

(In reply to Dani Megert from comment #23)
> (In reply to Thomas Watson from comment #22)
> 
> So, we could provide
> 
> 1. a new version of the default/standard config.ini.
> 2. steps to fix a custom config.ini.

I tested this on Mac using 4.7.3a. I am able to launch eclipse without any problems.

Comment 25 Thomas Watson 2018-04-19 08:45:32 EDT

(In reply to Dani Megert from comment #23)
> (In reply to Thomas Watson from comment #22)
> 
> So, we could provide
> 
> 1. a new version of the default/standard config.ini.
> 2. steps to fix a custom config.ini.

Using the config.ini to workaround this is only a short-term workaround.  It locks the packages in for all versions of Java you may run on which as we can see does not always expand in future versions (it sometimes shrinks!).

Comment 26 Nathan John Sowatskey 2018-10-26 03:58:48 EDT

I tested this with Neon.3, OSX 10.13.6 and java -version
java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)


It seems to work.

Comment 27 Nathan John Sowatskey 2018-10-26 04:00:31 EDT

Created attachment 276382 [details]
The .log file with the error that led ne to this bug.

For the record, the .log file with the error that led ne to this bug.

Comment 28 Guillaume Hillairet 2018-11-28 09:59:35 EST

We have a RCP product based on Eclipse 4.7.3 and we want users that have only Java 11 to be able to run it. 

So far the above fix seems to work fine and also work when other versions of Java are installed (8, 9 or 10). 

Is it correct that this fix will not cause any issues when other versions of Java are installed (i.e. not 11)?

Comment 29 Guillaume Hillairet 2018-11-28 10:01:51 EST

We have a RCP product based on Eclipse 4.7.3 and we want users that have only Java 11 to be able to run it. 

So far the above fix seems to work fine and also work when other versions of Java are installed (8, 9 or 10). 

Is it correct that this fix will not cause any issues when other versions of Java are installed (i.e. not 11)?

Comment 30 Theodore Epstein 2018-11-30 11:03:53 EST

@(In reply to Thomas Watson from comment #25)

Thomas, about your earlier comment:

> Using the config.ini to workaround this is only a short-term workaround.  It
> locks the packages in for all versions of Java you may run on which as we
> can see does not always expand in future versions (it sometimes shrinks!).

I'm working with Guillaume, who asked yesterday: 

> We have a RCP product based on Eclipse 4.7.3 and we want users 
> that have only Java 11 to be able to run it. 
> 
> So far the above fix seems to work fine and also work when other 
> versions of Java are installed (8, 9 or 10). 
>
> Is it correct that this fix will not cause any issues when other versions 
> of Java are installed (i.e. not 11)?

A bit more background: 

We make our product available on Eclipse Marketplace, and also packaged as an Eclipse RCP application. Migrating our RCP target platform to Photon is an option to get around the Java 11 compatibility issue, but that would likely make our product incompatible with Oxygen distributions. It seems there are still quite a few Eclipse users who haven't yet upgraded to Photon or subsequent releases.

We rely on Eclipse Marketplace for visibility to Eclipse users, but the Eclipse Marketplace Client filters out incompatible solutions. So removing compatibility with Oxygen would not only present a barrier for Oxygen users, who would need to upgrade to Photon to use our product; it would also make our product much less visible. Those users wouldn't even know our product exists unless they learn about it outside of Eclipse Marketplace.

So it's critical for us to ascertain whether it's safe to use the config.ini hack. I have only the most basic understanding of OSGi, but it seems that the org.osgi.framework.system.packages setting specifies a list of packages that application code can access through OSGi; and that at least two packages (javax.annotation and javax.transaction) that were formerly included in this for prior Java versions are now excluded for Java 11. 

We've done some superficial testing that suggests our RCP application runs fine with this config.ini change, running on Java 8, 9, 10, or 11. We don't see (yet) any detrimental effects of including that change. But since I don't really understand how this works, I wouldn't know how to focus our testing or code analysis to find potential problems. 

From your comment, it sounds like there's a risk that some of the packages currently on the list would need to be removed for compatibility with future Java releases; and that failure to remove them could result in compatibility issues like the one that was originally reported here. 

Is that the only risk? If so, we can probably live with it for now, and deal with future Java incompatibilities as and when they arise. 

OTOH, if including this config.ini hack in our RCP distribution is going to cause problem when running on _earlier_ releases of Java, we really need to understand that. 

Thanks for your work on Eclipse, and thanks in advance for any pointers or background references you can give us.

Comment 31 Mat Booth 2018-11-30 12:26:31 EST

> Migrating our RCP target platform to Photon is an option to get around the Java 11 compatibility issue, but that would likely make our product incompatible with Oxygen distributions.

I'd be interested in knowing more about this. What incompatibilities do you anticipate, or is it just speculation?

> It seems there are still quite a few Eclipse users who haven't yet upgraded to Photon or subsequent releases.

It seems like a strange situation to me that your users who want to use the latest and greatest JRE don't also want to use the latest and greatest version of the Eclipse Platform... :-)

Comment 32 Thomas Watson 2018-11-30 16:34:26 EST

(In reply to Theodore Epstein from comment #30)
> Is that the only risk? If so, we can probably live with it for now, and deal
> with future Java incompatibilities as and when they arise. 
> 

In later versions of Eclipse we no longer depend on hard coded lists of packages available from the JVM.  This is because now with Java 9+ the JVM may be launched with a subset of the available modules.  So we do not know ahead of time what list of packages to use.  Now we correctly determine the packages based on the modules being used.  If you have a consistent set of modules from the JVM being used then there probably is not much risk as long as your JVM version is constant.  But for each consistent set of modules there may be a different set of available packages with each Java version released.  So the list today that I posted in comment 22 is really only valid for a single release of Java.  As soon as new release of Java is available that you must support then you will need to validate the list of packages again.  If they differ then you will have to figure out how to launch with the correct config.ini based on the version of Java used.

I also can only speak from the core Equinox POV on support for Java versions.  Oxygen should run fine on Java 11.  But I can make no guarantees on the rest of the Eclipse platform, let alone everything available in Eclipse Market place.

Comment 33 Theodore Epstein 2018-11-30 17:28:10 EST

(In reply to Mat Booth from comment #31)
> > Migrating our RCP target platform to Photon is an option to get around the Java 11 compatibility issue, but that would likely make our product incompatible with Oxygen distributions.
> 
> I'd be interested in knowing more about this. What incompatibilities do you
> anticipate, or is it just speculation?

Guillaume can probably explain this in more detail. But generally, when we move to a newer target platform, we have to make sure our dependencies on common libraries are in sync (or at least not in conflict) with the new Eclipse platform and associated Orbit release.

Usually, this means updating our dependencies to newer versions of various components - JARs, plugins, bundles, or whatever form they may take. Once we do this, it puts our dependencies ahead of the previous Eclipse release, and we will usually find that there are conflicts that make our product incompatible with earlier Eclipse releases.

> > It seems there are still quite a few Eclipse users who haven't yet upgraded to Photon or subsequent releases.
> 
> It seems like a strange situation to me that your users who want to use the
> latest and greatest JRE don't also want to use the latest and greatest
> version of the Eclipse Platform... :-)

That's the thing: they're actually different groups of users. 

From our single codebase, we build an Eclipse RCP application and an Eclipse Update Site (for Marketplace installs and RCP updates). The RCP app is often downloaded and installed by developers who aren't Eclipse users at all. If they have recently installed or upgraded Java, they'll have Java 11. In fact, Java 11 is the only version Oracle currently provides on its main download page. You have to go to the archive to get previous versions. 

If all of our users were RCP users, sure, we would move our codebase eagerly to the latest available Eclipse release. 

But Eclipse developers are a different user group. If they're installing our solution from Marketplace, Java version compatibility is not an issue; they're already running a JVM that works with their Eclipse distro. But Eclipse release compatibility is an issue for us with this group. If we jump ahead to the latest Eclipse release, we'll be leaving behind a significant percentage of the Eclipse developer community.

Comment 34 Theodore Epstein 2018-11-30 18:59:39 EST

(In reply to Thomas Watson from comment #32)
> (In reply to Theodore Epstein from comment #30)

Thanks for your reply. Trying to get from your description to a few important conclusions...

> In later versions of Eclipse we no longer depend on hard coded lists of
> packages available from the JVM.  This is because now with Java 9+ the JVM
> may be launched with a subset of the available modules.

OK, that makes sense. 

> So we do not know
> ahead of time what list of packages to use.  Now we correctly determine the
> packages based on the modules being used.  

... and in case a given module is not available, its associated packages won't be exported from org.eclipse.osgi. But correctly configured dependencies on those packages will still resolve to system.bundle, javax.annotation.bundle, or another one of the provided bundles. Is that right? 

> If you have a consistent set of
> modules from the JVM being used then there probably is not much risk as long
> as your JVM version is constant.

IIUC, the set of modules will be the default set, unless the user does something to configure the JRE differently, which is not something we expect in normal usage. So in that sense, the set of modules should be consistent, at least within any given Java version. 

But the JVM version isn't constant; our RCP application supports Java 8, 9, or 10. And with the config.ini setting (for now), it should run on Java 11. 

> But for each consistent set of modules
> there may be a different set of available packages with each Java version
> released.  So the list today that I posted in comment 22 is really only
> valid for a single release of Java.  As soon as new release of Java is
> available that you must support then you will need to validate the list of
> packages again.  If they differ then you will have to figure out how to
> launch with the correct config.ini based on the version of Java used.

My concern isn't about future releases of Java; it's with the pre-Java 11 versions. Our very preliminary testing hasn't shown any problem running Java 8, 9, or 10 with org.osgi.framework.system.packages set in config.ini as you specified in comment 22. 

So my questions are: 

1. What could go wrong if we use the prescribed config.ini setting for Java 11, but run our Eclipse RCP application with Java 8, 9, or 10? 

2. If the Eclipse workbench starts normally with this config.ini setting, and appears to be running normally, does that mean everything is OK? Or could this configuration be vulnerable to runtime errors (ClassNotFoundException or others) that could be anywhere, and could occur only when a specific code branch is executed? 

If we have to ensure that this config.ini setting is only used for Java 11, we can try to have our installer add it dynamically, and add a -vm parameter in our reprezen.ini file (equivalent to eclipse.ini) to hard-wire the intended JRE version to our application. I'm sure there's a way to do this.

It's just much easier to have that setting in config.ini unconditionally, as long as it doesn't cause problems when running with Java 8-10.

Comment 35 Thomas Watson 2018-12-03 10:44:34 EST

(In reply to Theodore Epstein from comment #34)
> 
> 1. What could go wrong if we use the prescribed config.ini setting for Java
> 11, but run our Eclipse RCP application with Java 8, 9, or 10?

The javax.annotation and javax.transaction will no longer be exported by the system.bundle on Java 8.  That means you have to ensure these packages are provided by other bundles.  This isn't an issue for the javax.annotation package because Oxygen ships bundles that export this package.  But not for javax.transaction package.

> 
> 2. If the Eclipse workbench starts normally with this config.ini setting,
> and appears to be running normally, does that mean everything is OK? Or
> could this configuration be vulnerable to runtime errors
> (ClassNotFoundException or others) that could be anywhere, and could occur
> only when a specific code branch is executed?

All I can suggest is that you test with this extensively.  We certainly never did any extensive testing with the workaround offered with comment 22.  I do *think* much could go wrong, but it feels wrong for me to make any more definitive statement when the community simply has not testing this work around.

> 
> If we have to ensure that this config.ini setting is only used for Java 11,
> we can try to have our installer add it dynamically, and add a -vm parameter
> in our reprezen.ini file (equivalent to eclipse.ini) to hard-wire the
> intended JRE version to our application. I'm sure there's a way to do this.
> 
> It's just much easier to have that setting in config.ini unconditionally, as
> long as it doesn't cause problems when running with Java 8-10.

I agree it is much easier for you, but I am not sure how you do this with Eclipee Marketplace content, and I am not sure I would want to have stuff in Eclipse Marketplace configuring my config.ini in such ways.  But perhaps that is not what you are suggesting.

Comment 36 Theodore Epstein 2018-12-03 13:11:57 EST

(In reply to Thomas Watson from comment #35)


> The javax.annotation and javax.transaction will no longer be exported by the
> system.bundle on Java 8.  That means you have to ensure these packages are
> provided by other bundles.  This isn't an issue for the javax.annotation
> package because Oxygen ships bundles that export this package.  But not for
> javax.transaction package.

Thanks, that helps us understand the impact more specifically. That's what I was looking for.

> > It's just much easier to have that setting in config.ini unconditionally, as
> > long as it doesn't cause problems when running with Java 8-10.
> 
> I agree it is much easier for you, but I am not sure how you do this with
> Eclipee Marketplace content, and I am not sure I would want to have stuff in
> Eclipse Marketplace configuring my config.ini in such ways.  But perhaps
> that is not what you are suggesting.

We would only implement this in our RCP application installer. It wouldn't affect Eclipse users who install from the marketplace. And in any case, it isn't needed for current Eclipse users, because they must already have a compatible combination of Java + Eclipse as a prerequisite to installing our marketplace solution.

> All I can suggest is that you test with this extensively.  We certainly
> never did any extensive testing with the workaround offered with comment 22.
> I do *think* much could go wrong, but it feels wrong for me to make any more
> definitive statement when the community simply has not testing this work
> around.

Makes sense. Thanks again for your advice on this.

Comment 37 Thomas Watson 2018-12-03 16:00:13 EST

(In reply to Thomas Watson from comment #35)
> All I can suggest is that you test with this extensively.  We certainly
> never did any extensive testing with the workaround offered with comment 22.
> I do *think* much could go wrong, but it feels wrong for me to make any more
> definitive statement when the community simply has not testing this work
> around.
> 

Sigh, I really dislike when I leave out a critical NOT in a statement:

I do NOT *think* much could go wrong, but ...

Sorry if that caused some confusion.

Comment 38 Theodore Epstein 2018-12-03 16:18:44 EST

(In reply to Thomas Watson from comment #37)
> > All I can suggest is that you test with this extensively.  We certainly
> > never did any extensive testing with the workaround offered with comment 22.
> > I do *think* much could go wrong, but it feels wrong for me to make any more
> > definitive statement when the community simply has not testing this work
> > around.
> > 
> 
> Sigh, I really dislike when I leave out a critical NOT in a statement:
> 
> I do NOT *think* much could go wrong, but ...
> 
> Sorry if that caused some confusion.

No worries, I thought that might have been what you intended.