Community
Participate
Working Groups
Build Identifier: M20090917-0800 Per the specs, getEnclosedElements should return items in source order. However, it seems to return them in alphabetical order. I'm making an annotation where method parameters in a generated class are based on the source order of the annotated class. This is what would make the most sense to the user. Alphabetical is at least deterministic, but source order is preferable. http://download.java.net/jdk7/docs/api/javax/lang/model/element/TypeElement.html (Also, will debugging, I'm pretty sure I noticed getAllMembers uses hash maps internally and so will result in non-deterministic ordering. I was originally using getAllMembers and so I suspected it to be the culprit. However, removing it is still sorting orders alphabetically. I'd like to use getAllMembers if it would also respect source order--perhaps base class items first, source order, then sub class items next, also source order.) javac does return source order, which makes building cross-compiler annotation processors more complicated. I'll have to re-sort on own, most likely alphabetically since that is all I can do, so that both Eclipse and javac output is the same. Reproducible: Always Steps to Reproduce: 1. Make a source class with two fields bFoo first and aFoo second. 2. Make an annotation processor that calls getEnclosedElements and the order is aFoo then bFoo.
You cite the JDK 7 spec, but that is not (yet, anyway) what Eclipse implements, nor do I know of any JSR that superceded the Java 6 JSR 269 spec that was implemented in JDK 6 and that Eclipse is meant to implement. I don't see anything in the JDK 6 documentation about the ordering of getEnclosedElements; can you point me to it? We currently have no information about changes that may or may not be in JDK 7 when/if it emerges, since it appears to no longer be subject to the JCP. Thus there is no schedule for implementing JDK 7 changes. That said, it may be possible to get this ordering to work the way you want; I'll try to take a look.
Ack--my apologies about the jdk7 link; it was just the first thing I came across on Google. Here are the jdk6 javadocs for TypeElement with the same wording: http://java.sun.com/javase/6/docs/api/javax/lang/model/element/TypeElement.html Each method of this interface that returns a list of elements will return them in the order that is natural for the underlying source of program information. For example, if the underlying source of information is Java source code, then the elements will be returned in source code order. However, now that you mention it, I'm calling Element.getEnclosedElements, which is in the super interface of TypeElement, not TypeElement itself. So, perhaps the javadocs in TypeElement are meant to apply only to those methods declared directly in TypeElement, e.g. getInterfaces() and getTypeParameters(). Well, isn't that odd. My original search query when researching this was "getEnclosedElements order": http://www.google.com/search?q=getEnclosedElements+order Note the 1st result is the jdk7 link (no idea how it made it to the front of the list given its not even out yet). However, more interesting is note that the 2nd result is this bug: http://bugs.sun.com/view_bug.do?bug_id=6884227 The bug was only submitted in September of 2009--so maybe the javadocs have changed since you/Eclipse first implemented them.
Hmm, this might be a challenge. The Eclipse compiler internally stores the elements categorized by type (field, method, etc.) and sorted alphabetically; that is, it does not keep track of source order. To maintain source order we'll need to walk the AST and re-organize the bindings in a list of our own, separate from the compiler's internal list. Not impossible but I fear it's a substantial amount of effort. I guess if that's the direction that JDK 7 is going in, we'll need to do it at some point. Independently of that issue, I agree completely that getAllMembers should at least be deterministic. That should be a trivial change.
*** Bug 320441 has been marked as a duplicate of this bug. ***
(In reply to comment #4) > *** Bug 320441 has been marked as a duplicate of this bug. *** Apologies that I didn't find the original bug. Thanks for your time; appreciated.
Any progress on this? This is really annoying for me and prevents me from supporting ECJ for my annotation processor. My annotation processor is relying on the source order of methods. So all the methods would need another @Order annotation to feed the order to the processor. That's just not feasible for the user. It would be enough for me if you could provide a workaround. (I failed to build one myself) Thanks a lot!
Created attachment 233981 [details] Very dirty/ugly workaround for getting declaration order enclosed elements. I've created a quick and hacky workaround which works for types with available source files only. May completely fail with other eclipse versions than 4.2.2.
There's very little active development work going on in APT-land at present, unfortunately. As discussed earlier in this bug, the Oracle Java 7 behavior is something that was added after the current Eclipse implementation (which was based on Java 6). The Eclipse behavior is a valid interpretation of the Java 6 spec. In Java 7 Oracle changed the spec to match their compiler behavior. Personally I would advise against writing a processor that depends on the source order of code, because in general the Java language is not source-order dependent, except for composed methods such as static initializers. I don't think "natural order" is meaningful except in specific case where the annotations are being read from source; it is otherwise undefined. It certainly is not well defined for, e.g., PackageElement. Also as mentioned earlier, I suspect fixing this efficiently in Eclipse will be hard: it will require storing a concept of source order (of all elements in all sources) at annotation scanning time, and then referring to it later, or else reloading the source when this method is called. That's potentially a very large amount of memory or a lot of re-scanning. Now that I've said all those discouraging things: if you have cycles to spare and would like to dig in on this, your patches will be welcome, as long as the code is clean and performant, of course. I would suggest that any change in behavior should be limited to Java 7 compliance mode and above, so as not to break existing Java 6 processors that might be relying on the current behavior (which, again, is a valid interpretation of the Java 6 spec).
Thanks for your reply. Any hope there will be more resources dedicated to APT soon? I am building an embedded DSL in Java which is actually not Java anymore. So I am not restricted to usual Java semantics or specs. The DSL basically produces java code from annotated methods that represents a state machine. When interpreted for state machines the order of methods inside a class is a useful language element. As far as I've seen the fix is a matter of not sorting alphabetically since the elements usually already come in declaration order. At least for the enclosed elements in types. So for me the fix seems rather simple if its possible to change the default sorting in the internal representation. Do you think that is feasible or will that break all the tests? I will definitely try to create a patch for it at some point, but it would be more effective if one with ECJ experience looks into that ;-).
I don't have the code handy at present but my recollection was that the alphabetical sorting was happening inside the compiler (as opposed to within the APT implementation); i.e., when the APT code calls the JDT compiler code to get the elements, it gets them in alphabetical order. The sort is happening within the JDT compiler, and changing it would presumably affect other dependent code. Perhaps my recollection is wrong; if so this would be considerably easier. Re getting more resources on APT - I doubt it is likely to happen unless the volunteer community can step up. The original effort to produce APT was funded by BEA in order to support their Weblogic Workshop IDE, which depended on annotations, but after BEA got acquired by Oracle that effort got de-funded. There does not currently seem to be any corporate entity that has the interest and the funding to dedicate fulltime developers to maintaining APT. As one of the original developers I've tried to stay at least a little involved, but the demands of my current job haven't let me contribute meaningfully for some time.
*** Bug 420852 has been marked as a duplicate of this bug. ***
Sorry for duplicate. I searched, but obviously not enough. Best regards, Stefan.
(In reply to Walter Harley from comment #10) > when the APT code calls the JDT compiler code > to get the elements, it gets them in alphabetical order. The sort is > happening within the JDT compiler, and changing it would presumably affect > other dependent code. Thanks for the detailed posts. i think it is very important that we at eclipse go on quickly to solve this issue. As Walter states, the method sorting happens already in the eclipse compiler. Basically i found these locations that sort by textual name: (1) org.eclipse.jdt.internal.compiler.ast.TypeDeclaration line 415: MethodBinding[] methodBindings = sourceType.methods(); // trigger sorting (2) org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding:1167 calls org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.sortMethods(MethodBinding[], int, int) So there must be an eclipse compiler fix for this. The fact that it affects the eclipse compiler should not be a reason to wait another 3 years for a solution. If somehow possible I would like to help to fix this issue. Simple solution proposal: In order not to break the eclipse compiler, a solution must change the behaviour of the eclipse compiler BUT it can provide additional information that can be used by the jdt.apt in order to sort methods (and fields and inner types...) according to the JSR 269 (bug JDK-6884227). The most promising way is to use the two existing fields in the jdt ast tree... org.eclipse.jdt.internal.compiler.ast.ASTNode{ public int sourceStart, sourceEnd; } And adding also two additional fields on: org.eclipse.jdt.internal.compiler.lookup.Binding{ public int sourceStart, sourceEnd; } These fields should be set whenever possible (when handling source code) by the eclipse compiler. The jdt.apt can then use these fields to do correct sorting. Ideally sorting by sourceStart and an additional element to be safe in case source locations are not set. If my proposal is too straight forward or too "easy" please help making it better. People need confidence that such issues (even hard) are solved in time.
Please apologyse my typos :-) >In order not to break the eclipse compiler, a solution must change the behaviour of the eclipse compiler In order not to break the eclipse compiler, a solution must NOT change the behaviour of the eclipse compiler
Adding fields related to source binding, and sorting on those, seems like a reasonable approach. You'll need to be sure to deal with the case where a type comes in from binary rather than source, though.
> You'll need to be sure to deal with the case where a Yes, that is crucial. Therefore the sorting process in apt should be based on a composite (triplet) key of [sourcePosition,alphanumericName,sequenceNumber]
I've created a Pull-Request on GitHub with a rather simple fix: https://github.com/eclipse/eclipse.jdt.core/pull/2
I've also added the same commit to Gerrit now (with CLA): https://git.eclipse.org/r/35416
Patch looks good. I was initially tempted to move the sourceStart computation logic to individual elements, but I guess that's an overkill for this.
Some glitch with Hudson build preventing the Gerrit patch going through. Directly pushed to master: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=c326c940f9ef138395e800fef452568d0c87c6c0 Thanks for the patch Andreas!
Wow, congrats! Thank you guys!
Thanks Jayaprakash for reviewing and merging!
Verified for Eclipse Mars 4.5 M4 using build I20141209-2000
This bug is unfortunately not fixed, as it only applies when compiling from source. This means that when the Eclipse IDE does incremental compilation while editing, the elements become improperly sorted again and break code. This can be worked around by cleaning the project every time you edit a file that is affected by the bug, but this disturbs the normal workflow and takes too long on larger projects. I'm trying to find a good way to store the source start in the binary files, but I don't know if that's the best solution or not. I'd like some guidance on how to solve this properly, as I'm not familiar with ECJ, but I'm willing to write the patch.
This is a workaround I am using successfully for a while now: https://github.com/graalvm/graal/blob/master/truffle/src/com.oracle.truffle.dsl.processor/src/com/oracle/truffle/dsl/processor/java/compiler/JDTCompiler.java#L61
Forgot to mention that it also works with incremental compilation.
New Gerrit change created: https://git.eclipse.org/r/97595
(In reply to Christian humer from comment #25) > This is a workaround I am using successfully for a while now: > https://github.com/graalvm/graal/blob/master/truffle/src/com.oracle.truffle. > dsl.processor/src/com/oracle/truffle/dsl/processor/java/compiler/JDTCompiler. > java#L61 That looks like what I wrote in my patch. Check out the Gerrit change.
Guys, this bug is closed. Please open a new bug with steps that show it's not fixed and attach the Gerrit change there.
It looks like 500589 has already been filed -- I'll mention my patch there.
