Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 459415 - [1.9] Support modular run time images in JDT
Summary: [1.9] Support modular run time images in JDT
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.4.1   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: BETA J9   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact: Jay Arthanareeswaran CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 457413 461372
  Show dependency tree
 
Reported: 2015-02-09 04:43 EST by Jay Arthanareeswaran CLA
Modified: 2015-05-26 02:56 EDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jay Arthanareeswaran CLA 2015-02-09 04:43:20 EST
The JEP 220 of the project jigsaw defines the new module run-time images. This needs to be supported in JDT.

At the moment, this is the list (not complete by any means) of things that need to evolve:

1. First and foremost, loading classes from a JDK9. JDT should be able to read classes from the new jrt format. We need to come up with a strategy for this - whether to use the provided jrt-fs that will be bundled with the JDK9 (even when eclipse itself runs on <= JRE 8) or something else (home-grown?). Without this, it's impossible to use JDK9 in project build path.

2. JREContainer should evolve to support the new VM installs. Some of the changes known at this point are, removal of /ext and /endorsed. Instead the new module system will be used for extensions.

3. Removal of certain JAR files: rt.jar and tools.jar have been removed. If we are hard-coding dependency anywhere, that should be addressed.
Comment 1 Jay Arthanareeswaran CLA 2015-02-11 00:21:58 EST
From the limited documentation available to me, it's not clear how a program running on < JDK 8 can load the modular images and get .class contents etc.. For people interested in such things, there are two things provided by the JDK.

1. A file system provider that supports the new jrt URI scheme that can be used to get the contents of a particular .class in the JDK. But one can do that only from within the JVM 9.

2. There's supposedly a tool in bin folder (jimage.exe or similar) to help one look inside the modular image. Of course, we can't use this for our purpose.

Most people probably knew this already, but adding here for documentation purpose.
Comment 2 Dani Megert CLA 2015-02-11 03:09:09 EST
(In reply to Jay Arthanareeswaran from comment #1)
> From the limited documentation available to me, it's not clear how a program
> running on < JDK 8 can load the modular images and get .class contents etc..
> For people interested in such things, there are two things provided by the
> JDK.
> 
> 1. A file system provider that supports the new jrt URI scheme that can be
> used to get the contents of a particular .class in the JDK. But one can do
> that only from within the JVM 9.
> 
> 2. There's supposedly a tool in bin folder (jimage.exe or similar) to help
> one look inside the modular image. Of course, we can't use this for our
> purpose.
> 
> Most people probably knew this already, but adding here for documentation
> purpose.

I *think* we will have to use (put on classpath) the file system provider that comes with the JRE 9 that's used by the project.
Comment 3 Jay Arthanareeswaran CLA 2015-02-11 03:25:47 EST
(In reply to Dani Megert from comment #2)
> I *think* we will have to use (put on classpath) the file system provider
> that comes with the JRE 9 that's used by the project.

Yep, but my bigger worry (based on my limited knowledge) is if we can read the classes from a JDK9 modular image when eclipse is running in JRE 8. But I do remember from the mailing group discussions, they have plans to make the file system provider to work with a supplied "java.home". But we'll have to wait and watch how that pans out.

On other related note, we may have to consider moving to 1.7 compliance as some of the APIs mentioned by JEP 220 (FilesSystems, Files, Path) are at 1.7 only.
Comment 4 Markus Keller CLA 2015-02-12 11:09:50 EST
The Jigsaw people are aware of the problems with accessing another JDK install's modules, and they explicitly said they're working on it but they don't have a solution yet:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-January/004153.html 
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-February/004189.html

Moving JDT Core to 1.7 compliance is OK (already for Mars).
Comment 5 Jay Arthanareeswaran CLA 2015-02-12 21:47:05 EST
(In reply to Markus Keller from comment #4)
> The Jigsaw people are aware of the problems with accessing another JDK
> install's modules, and they explicitly said they're working on it but they
> don't have a solution yet:
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-January/004153.html 
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-February/004189.html

Thanks for the links, Markus. Yep, this is what I was referring to in comment #3. But wonder how long it's going to take. But we can only wait.
Comment 6 Jay Arthanareeswaran CLA 2015-02-24 04:21:06 EST
I am working on an initial prototype, whose immediate requirements being:

1. Support projects with JDK 9 in the build path. Full build, incremental build and reconciler should work with JDK 9.
2. Support from the package explorer to view, expand/explore the packages and classes in a JDK modular image.
3. Open class files from the module image (in the build path), with support for source attachments.
4. Existing features such as Outline view, quick outline view, AST view, hover, hierarchies, Javadoc etc. should be supported for binary types from JDK module images.
5. Batch compiler support with JDK 9.
6. Support for projects with JDK 9 as well as < JDK 9 in the same workspace.
7. Support for content assist.
8. Support for search.
9. All existing tests should pass when run with <= JRE8

To be considered in successive phase:
1. Performance (at the moment) for certain use cases that involve opening Jimage/JAR could be different (better or worse) than before. To be studied.
2. Update the test framework to support JDK 9.
3. At the moment, above features will only work when run with JRE 9 or with jrt file system in the path. This limitation should be removed when things become clearer.
Comment 7 Andrew Clement CLA 2015-03-04 17:35:34 EST
Hi Jay,

I've actually merged the relevant bits from BETA_JAVA9 into AspectJ so that AspectJ can build on a java9 jdk.  This is working for simple cases but I find if any of my code uses the non 'core' types - i.e. if I refer to something in java.awt or something in javax.naming then I get problems with types not being found.  It might be that this works for you and I've made a mistake porting the changes - so thought I'd ask: do test programs with those kinds of type in work for you with this change?

I assumed it was because they were not in the java.base module and currently the BETA_JAVA9 branch seems to only look at that module. (For example javax.naming.Context is found if I search in the java.naming 'module' within the image, instead of 'java.base')

Thanks for starting this effort !
Comment 8 Jay Arthanareeswaran CLA 2015-03-04 22:47:43 EST
(In reply to Andrew Clement from comment #7)
> I assumed it was because they were not in the java.base module and currently
> the BETA_JAVA9 branch seems to only look at that module. (For example
> javax.naming.Context is found if I search in the java.naming 'module' within
> the image, instead of 'java.base')

Indeed, we started this as a prototype and I just assumed that the base contains all that we may need. It won't be a problem to include all of them. In any case the code will change when we know how to read contents of a particular image file. At the moment we are only reading from the current runtime. I will fix this for now. And thanks for the trying out!
Comment 9 Jay Arthanareeswaran CLA 2015-03-05 00:17:41 EST
Hmm.. It's going to be interesting to read the class file content since there is no apparent mapping between a package and a (sub)module. Ideally we don't want to iterate over all modules to find the match. But I don't see a better way at the moment.
Comment 10 Andrew Clement CLA 2015-03-05 01:58:46 EST
For now, that is exactly how I changed the JDT code when dropping it into AspectJ to get it going and what I did in the AspectJ weaver, iterating over all the modules - it felt very non-optimal.
Comment 11 Jay Arthanareeswaran CLA 2015-03-05 04:40:44 EST
I have fixed this now to look for all modules. Have introduced a rudimentary mapping cache between packages and modules. But this is likely to change.

On another note, we should start using the Zip file system provider (also available since Java 7) so it's all consistent.
Comment 12 Markus Keller CLA 2015-03-05 06:21:53 EST
(In reply to Jay Arthanareeswaran from comment #9)
> Hmm.. It's going to be interesting to read the class file content since
> there is no apparent mapping between a package and a (sub)module. Ideally we
> don't want to iterate over all modules to find the match. But I don't see a
> better way at the moment.

This has been discussed here:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2014-November/004044.html

But there was also opposition to providing this via the jrt protocol:
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2014-November/004049.html
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2014-November/004076.html

I haven't heard of a conclusion.
Comment 13 Jay Arthanareeswaran CLA 2015-03-24 04:34:18 EDT
(In reply to Jay Arthanareeswaran from comment #11)
> On another note, we should start using the Zip file system provider (also
> available since Java 7) so it's all consistent.

Gave it a shot, but looks like replicating the caching that Java Model Manager does for zip files is bit difficult. So, we will revisit this later if found necessary.
Comment 14 Jay Arthanareeswaran CLA 2015-05-26 02:56:35 EDT
I think we can call this done, but for the following two open issues:

1. Rewrite the image visitor part to use the proposed package-module mapping that Oracle is working on (Bug 468260).

2. Make JDT Core tests work with both a Java 9 as well as older JREs. Ref: Bug 466944.