Community
Participate
Working Groups
There are API changes in APT in Java 8 and JDT's implementation must support the changes. In essence, we must provide implementation for the following interface methods: Note: '->' stands for "must implement" ElementsImpl -> Elements.isFunctionalInterface(TypeElement) ExecutableElementImpl -> ExecutableElement.isDefault() ExecutableTypeImpl -> ExecutableType.getReceiverType() ExecutableElementImpl -> ExecutableElement.getReceiverType() Following types must implement AnnotatedConstruct.getAnnotationsByType(Class<A>) : TypeElementImpl TypeParameterElementImpl PackageElementImpl VariableElementImpl TypeMirrorImpl ExecutableElementImpl NoTypeImpl Following types must implement AnnotatedConstruct.getAnnotation(Class<A>) and AnnotatedConstruct.getAnnotationMirrors(): TypeMirrorImpl NoTypeImpl
I think we can provide the implementation but still keep the compliance of compiler.apt at 1.6. As long as we provide the Java8 specific implementation, we should be compatible with both SE 8 and below. I am still doing my experiments, though. Walter/Stephan, Do you see any issues with this?
Jay, many thanks for leading the APT work for Java 8. I suggest you use this bug as the umbrella bug for APT work for Java 8 and raise other blocking bugs for actual work at unit task level. Walter, we are grateful for your weighing in any design questions/aspects. We understand your other commitments may allow very limited participation, we are happy to take what we get.
Ideally we would make it so that an annotation processor that calls Java 8 methods cannot be compiled when the selected JDK is <8; also, a processor that calls Java 8 methods should fail to run (with something like a NoSuchMethodError) when executed in Java 6 or 7 compliance mode. The compilation part works as long as people compile against the JDK rather than against the org.eclipse.jdt code; and that's desirable anyway. If they compile against org.eclipse.jdt they are not guaranteed compatibility with javac. The execution part can be handled at runtime; we just need to check compliance mode and throw an exception if the call is inappropriate. So that leaves only the question of how to make the APT classes, like ElementsImpl, implement the Java 8 interfaces even though they must be compiled against the Java 1.6 JDK themselves. I think the only solution is to incorporate the Oracle interface code into Eclipse, as we did for the Java 5 APT interfaces in Eclipse 3.1. I don't know whether the current Oracle open-source licensing is easier to work with, but they were willing to let us do this even back before they'd open-sourced so I don't think it should be too hard now.
(In reply to comment #3) > So that leaves only the question of how to make the APT classes, like > ElementsImpl, implement the Java 8 interfaces even though they must be > compiled against the Java 1.6 JDK themselves. I think the only solution is > to incorporate the Oracle interface code into Eclipse, as we did for the > Java 5 APT interfaces in Eclipse 3.1. I don't know whether the current > Oracle open-source licensing is easier to work with, but they were willing > to let us do this even back before they'd open-sourced so I don't think it > should be too hard now. Jay, let us close on what needs to happen asap, while the actual happening can take its due course. If this requires interfacing with lawyers (I don't know) and such we need to kick start the process real soon.
(In reply to comment #3) > So that leaves only the question of how to make the APT classes, like > ElementsImpl, implement the Java 8 interfaces even though they must be > compiled against the Java 1.6 JDK themselves. I think the only solution is > to incorporate the Oracle interface code into Eclipse, as we did for the > Java 5 APT interfaces in Eclipse 3.1. I don't know whether the current > Oracle open-source licensing is easier to work with, but they were willing > to let us do this even back before they'd open-sourced so I don't think it > should be too hard now. Just thinking aloud - what would stop us have the TypeMirrorImpl etc. to provide the implementation (without the @Override) for the new methods and yet compile them with JDK 6 ? Here's a little experiment I tried: 1. Added the Java 8 specific implementation to JDT's APT. I did this by changing the JDK of jdt.compiler.apt to 1.8 version, provided the missing implementations and after making sure there were no compiler errors, moved the project back to 1.6 JDK. So, we have APT compiled against 1.6 but still have the 1.8 specifics in them. No compiler errors, but that's obvious. 2. Wrote a annotation processor for 1.6 specifications and everything works as ever. No surprises here either. 3. Moved the annotation processor to 1.8 spec and tried accessing the 1.8 specific APIs. I wasn't sure if this would work, but it did work. For e.g., I was able to cast the VariableElementImpl to AnnotatedConstruct even though the former wasn't aware of nor 'implements' the latter when it was compiled. But VariableElementImpl does provide the methods that were introduced in AnnotatedConstruct. Even though all this works, I am not sure if we can seriously consider this approach ever. I would like Srikanth to weigh in on this one, esp. on the last point. Can we safely assume that the casting I talked about is guaranteed to work?
(In reply to comment #5) > For e.g., I was able to cast the VariableElementImpl to AnnotatedConstruct > even though the former wasn't aware of nor 'implements' the latter when it > was compiled. But VariableElementImpl does provide the methods that were > introduced in AnnotatedConstruct. > > Even though all this works, I am not sure if we can seriously consider this > approach ever. I would like Srikanth to weigh in on this one, esp. on the > last point. Can we safely assume that the casting I talked about is > guaranteed to work? The compiler will always allow a cast from an object whose compile time type is some non-final type to an interface type. In this case the compiler will emit a checkCast instruction to request the VM to actually check that the operation is type safe. At run time if it is not type safe, a CCE would result. So it is not necessary for VariableElementImpl to be "aware" of or implement AnnotatedConstruct. The runtime cast will succeed if the dynamic type of the object referred to by the reference statically typed to be VariableElementImpl implements AnnotatedConstruct.
"The compiler will always allow a cast from an object whose compile time type is some non-final type to an interface type." I didn't know that! If that's so then yes, this approach should work. My criterion would be: if an annotation processor can pass: assert (e instanceof Elements) then whatever we're doing is okay.
(In reply to comment #7) > My criterion would be: if an annotation processor can pass: > > assert (e instanceof Elements) I was staring at this snippent for a while and then it occurred to me that you probably meant Element and not Elements. Remember, Elements is part of the API too?
Well, comment 1 talks about Elements.isFunctionalInterface(), which is why I mentioned Elements rather than Element. But the statement applies either way - basically, as long as all of our implementation classes look, to the annotation processor, like the interfaces it's expecting, then I think we're good. The annotation processors run in the same JVM as the rest of the compiler, and are loaded by the same classloader. So if org.eclipse.[...].SomeElementImpl implements javax.[...].Element, and the annotation processor gets an instance of SomeElementImpl and asks whether it is instanceof Element, the answer will be 'true'. But if SomeElementsImpl just happens to have methods that have the same signature as those of Element, without actually stating in its declaration that it implements Element, then I would *not* expect it to be instanceof Element. In Comment 5 and Comment 6 it seemed like you were suggesting a path that would let you *cast* to Element, but that might not support instanceof. I don't know enough about the JVM to know whether that is even possible, but if it is, it seems like a bad idea.
(In reply to comment #9) > In Comment 5 and Comment 6 it seemed like you were suggesting a path that > would let you *cast* to Element, but that might not support instanceof. I > don't know enough about the JVM to know whether that is even possible, but > if it is, it seems like a bad idea. Thanks for the explanation, Walter! I think we are saved by the fact that we don't have to implement AnnotatedConstruct ourselves as this is only indirectly implemented through Element or TypeMirror. So, I guess we are fine. The instaceof checks work fine.
But looks like we have a problem with NameImpl as two new methods have been added to java.lang.CharSequence, which NameImpl indirectly implements. I feel that the bigger problem is with the new type IntStream that has found its way into Java 8 and new method signatures: public IntStream codePoints() public IntStream chars() On closer look, these are default methods in CharSequence and I am seeing compilation errors (when jdt.compiler.apt has JRE 8 in it's classpath) only because my host eclipse doesn't yet have Java 8 support. I should worry about this, shouldn't I?
(In reply to comment #11) > I should worry about this, shouldn't I? Actually, wanted to say more on this. The newly introduced type IntStream means we have to either compile this project or have IntStream included in the compiler.apt as source. Sigh!
(In reply to comment #12) > (In reply to comment #11) > > I should worry about this, shouldn't I? > > Actually, wanted to say more on this. The newly introduced type IntStream > means we have to either compile this project ... Sorry, meant to say compile with Java 8 in the classpath.
(In reply to comment #13) > (In reply to comment #12) > > (In reply to comment #11) > > > I should worry about this, shouldn't I? > > > > Actually, wanted to say more on this. The newly introduced type IntStream > > means we have to either compile this project ... > > Sorry, meant to say compile with Java 8 in the classpath. Just to clarify, I was trying to compile them with 1.8 just to make sure I hadn't missed out any new methods. So, the plan should be to compile the jdt.compiler.apt project with 1.6 as before. So, I will try out the following combinations to see if everything is alright. 1. With Java 8 supported eclipse, jdt.compiler.apt with JRE 1.8 in classpath and 1.8 as source level should not produce any compiler errors. 2. jdt.compiler.apt with 1.6 JRE and 1.6 source level should not produce any compiler errors. And both configurations should not fail when the two new methods are invoked by the client with 1.8 JRE. I will run the tests and report back. At the moment, it appears there is no need to worry.
In past releases where the API changed, we ended up having a separate bundle for each relevant version - first for 1.5, and then again for 1.6. For 1.5, we had to include the Sun APT interfaces within our own bundle (via legal consent) because they were not part of the JDK. For 1.6, they were part of the JDK so we just had to compile that module against 1.6. The problem here is that new APT interfaces haven't been added (so we can't just add a new bundle with the new interfaces), rather, the existing interfaces have been changed and/or the interfaces they extend or reference have been changed. This would imply having one set of APT bundles when compiling under 1.6/1.7, and a different set when compiling under 1.8; I don't see how that is possible (given that, for instance, two different projects in the same workspace could have different compliance). I suspect that the only answer is to: 1. Include the new (Java 8) interfaces within the APT bundles, so that the bundles can be compiled under Java 6. This implies redistributing parts of the JDK. 2. Implement the new Java 8 interfaces conditionally; i.e., if (compliance < 8) { throw new UnsupportedOperationException(); }
I am changing the bug dependencies. This bug need not depend on bug 413613 but it should be the other way round. Besides, I am planning to provide no-op implementation for AnnotatedConstruct.getAnnotationsByType(Class<A>), for it requires repeating annotations and take it up later. Bug 415396 has been raised to handle this.
Created attachment 234597 [details] work in progress This is a work in progress and I am in the process of writing tests. But feel free to take a look, although it just provides implementation for the newly added methods and does nothing to support type annotations. The first goal is to prove that the new API can be supported without having the new javax.lang.model APIs as part of source code.
Created attachment 234784 [details] Updated patch Patch updated with tests. All but two APT tests are passing. Both the failing tests seem to expose some issues with JDK 8. One of them appears to be a regression in the manner in which white space elemnets in Javadoc elements are processed. The other one might just mean the APT support in Oracle's JDK is not complete.
If you could split the JSR335 portions, I can review and we can release it first.
Created attachment 235039 [details] JSR 335 related API implementation I have put only the JSR 335 related APIs in the APT model along with the tests.
Created attachment 235129 [details] Separate patch for JSR 308 API + Tests
Created attachment 235342 [details] Updated patch This is a work in progress and will introduce new compilation errors. I am yet to come up with the strategy for testing the new API methods. At the moment, for the tests to pass, the classpath should have 1.8 SE and be run with 1.8 JRE. I am considering the options of using reflections for testing the new method implementations. I am parking it for the moment and will come back to this soon.
Stephan/Walter, as far as our reading of the new APIs go, there is still no way for an annotation processor to request visibility inside a method body to process annotations there. Do you know/think otherwise ? Jay did some experiments with hooking an annotation processor into javac and his experiments show that type annotations inside method bodies do not get passed on to the processor. What are the relevant past JSRs here ? As far as I can see the compiler tree APIs (http://docs.oracle.com/javase/6/docs/jdk/api/javac/tree/index.html) are completely outside the purview of both JSR199 and JSR 269 ?
(In reply to Srikanth Sankaran from comment #23) > Stephan/Walter, as far as our reading of the new APIs go, there is still no > way > for an annotation processor to request visibility inside a method body > to process annotations there. Do you know/think otherwise ? > > Jay did some experiments with hooking an annotation processor into javac > and his experiments show that type annotations inside method bodies do not > get passed on to the processor. > > What are the relevant past JSRs here ? As far as I can see the compiler tree > APIs (http://docs.oracle.com/javase/6/docs/jdk/api/javac/tree/index.html) are > completely outside the purview of both JSR199 and JSR 269 ? For the record, our review of the relevant specification showed that type use annotations that occur in type use sites are not to be discovered in the discovery process. Type use annotations at declaration sites along with type parameter declaration site are the only ones that are discovered by the annotation processing framework. With respect to annotations on constructs inside method bodies: Java 8 does not change anything here.
This bug is subsumed by https://bugs.eclipse.org/bugs/show_bug.cgi?id=413613, but we will leave it open until we have had a chance to look through all the comments to make sure that no issue went unaddressed.
(In reply to Walter Harley from comment #9) > In Comment 5 and Comment 6 it seemed like you were suggesting a path that > would let you *cast* to Element, but that might not support instanceof. I > don't know enough about the JVM to know whether that is even possible, but > if it is, it seems like a bad idea. The same rules apply here as in cast. instanceof answers true or false based on the dynamic type of the object and not the static type. So when you query if x instanceof I where I is an interface and x is an object of a non-final class, the compiler cannot determine whether the instanceof check should fail or not. So this program will compile: // -- interface I { } class X { public static void main(String[] args) { X x = getX(true); if (x instanceof I) System.out.println("x instanceof I"); else System.out.println("x !instanceof I"); x = getX(false); if (x instanceof I) System.out.println("x instanceof I"); else System.out.println("x !instanceof I"); } static X getX(boolean firstCall) { if (firstCall) return new X(); else { class Y extends X implements I {} return new Y(); } } } and print x !instanceof I x instanceof I If X were to be a final class (and the code that extends X is removed) the program will not even compile. (In reply to Walter Harley from comment #15) > I suspect that the only answer is to: > 1. Include the new (Java 8) interfaces within the APT bundles, so that the > bundles can be compiled under Java 6. This implies redistributing parts of > the JDK. This shouldn't be required since we don't have to mention the new interfaces in our code at all, just implement the methods without @Override annotation. > 2. Implement the new Java 8 interfaces conditionally; i.e., if (compliance < > 8) { throw new UnsupportedOperationException(); } Jay, can you check on this point ? If a Java6 processor uses JRE8 and incorrectly calls a Java 8 method, perhaps we should throw an error.
(In reply to Srikanth Sankaran from comment #25) > This bug is subsumed by https://bugs.eclipse.org/bugs/show_bug.cgi?id=413613, > but we will leave it open until we have had a chance to look through all the > comments to make sure that no issue went unaddressed. OK, reviewing the comment trail, I see only two possible follow ups: (1) Should the *impl classes Java 8 method throw exceptions for processors below 8 that are misconfigured to use JRE8 and call a Java 8 API iadvertantly. (2) Looking at the issue reported in comment#11, it is unclear if this is still an issue requiring action or not. Let us use the current bug to track these two items before resolving.
Comment on attachment 235342 [details] Updated patch This is subsumed by the work for bug 413613
(In reply to Srikanth Sankaran from comment #27) > OK, reviewing the comment trail, I see only two possible follow ups: > > (1) Should the *impl classes Java 8 method throw exceptions for processors > below 8 that are misconfigured to use JRE8 and call a Java 8 API > iadvertantly. I ran some quick tests for this. 1. Made the Java8ElementProcessor a 1.6 processor (via @SupportedSourceVersion at RELEASE_6) and all tests pass, including the ones with Javac. Should we bother to differ at all? 2. Ran Java8ElementsTests without the "1.8" command line parameter and obviously (since this is meant for the compiler) all tests fail because the code fails to compile. > (2) Looking at the issue reported in comment#11, it is unclear if this is > still an issue requiring action or not. Sorry, I should have mentioned earlier. These two methods are 'default' and we don't need to do anything here. I will also extra the Java8ElementProcessor out and ran it from IDE and report how it goes.
(In reply to Jayaprakash Arthanareeswaran from comment #29) > (In reply to Srikanth Sankaran from comment #27) > > OK, reviewing the comment trail, I see only two possible follow ups: > > > > (1) Should the *impl classes Java 8 method throw exceptions for processors > > below 8 that are misconfigured to use JRE8 and call a Java 8 API > > iadvertantly. > > I ran some quick tests for this. > > 1. Made the Java8ElementProcessor a 1.6 processor (via > @SupportedSourceVersion at RELEASE_6) and all tests pass, including the ones > with Javac. Should we bother to differ at all? The real test for this would be when we call getAnnotationByType(), which our tests don't yet do since https://bugs.eclipse.org/bugs/show_bug.cgi?id=418000 is still being worked on. You can always stick in a call and see what happens with both javac and ecj for the 1.6 processor with JRE8 on its build path. > I will also extra the Java8ElementProcessor out and ran it from IDE and > report how it goes. Thanks.
(In reply to Srikanth Sankaran from comment #30) > The real test for this would be when we call getAnnotationByType(), which > our tests don't yet do since > https://bugs.eclipse.org/bugs/show_bug.cgi?id=418000 is still being worked > on. You can always stick in a call and > see what happens with both javac and ecj for the 1.6 processor with JRE8 > on its build path. If you meant getAnnotationsByType(), we already do this in Java8ElementProcessor, although those tests don't test repeating annotations.
(In reply to Jayaprakash Arthanareeswaran from comment #31) > If you meant getAnnotationsByType(), we already do this in > Java8ElementProcessor, although those tests don't test repeating annotations. You are right, I missed the plural spelling. I suggest we take the ostrich approach a la javac. If the IDE testing shows any issues, let us raise follow ups and deal with it there.
