Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 361378

Summary: Intermittent NullPointerException in AptCompilationParticipant
Product: [Eclipse Project] JDT Reporter: Missing name <matthias.keller>
Component: APTAssignee: Generic inbox for the JDT-APT component <jdt-apt-inbox>
Status: VERIFIED FIXED QA Contact:
Severity: major    
Priority: P3 CC: eclipse.sprigogin, Frank.Harper, jarthana, joerg.schoenfisch, manoj.palat, Rene.Brandstetter, rossj, samuelschmid75, stephen
Version: 3.7.1Flags: jarthana: review+
Target Milestone: 4.5 M4   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
editor arrangement to reproduce none

Description Missing name CLA 2011-10-19 07:27:06 EDT
We're using bindgen to generate some classes in our project. The code always compiles without problem when compiling manually.
But in eclipse we sometimes get intermittent errors, they are sometimes solved when restarting eclipse but just cleaning/recompiling usually doesn't solve it.

We get this NPE:
java.lang.NullPointerException
	at org.eclipse.jdt.apt.core.internal.AptCompilationParticipant.addJava6GeneratedFile(AptCompilationParticipant.java:272)
	at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeProcessingEnvImpl.addNewUnit(IdeProcessingEnvImpl.java:132)
	at org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeJavaSourceOutputStream.close(IdeJavaSourceOutputStream.java:72)
	at sun.nio.cs.StreamEncoder.implClose(Unknown Source)
	at sun.nio.cs.StreamEncoder.close(Unknown Source)
	at java.io.OutputStreamWriter.close(Unknown Source)
	at java.io.BufferedWriter.close(Unknown Source)
	at java.io.PrintWriter.close(Unknown Source)
	at org.bindgen.processor.generators.BindingClassGenerator.saveCode(BindingClassGenerator.java:164)
	at org.bindgen.processor.generators.BindingClassGenerator.generate(BindingClassGenerator.java:72)
	at org.bindgen.processor.GenerationQueue.processQueue(GenerationQueue.java:65)
	at org.bindgen.processor.Processor.process(Processor.java:67)
	at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.handleProcessor(RoundDispatcher.java:139)
	at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.round(RoundDispatcher.java:110)
	at org.eclipse.jdt.internal.compiler.apt.dispatch.BaseAnnotationProcessorManager.processAnnotations(BaseAnnotationProcessorManager.java:159)
	at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeAnnotationProcessorManager.processAnnotations(IdeAnnotationProcessorManager.java:134)
	at org.eclipse.jdt.internal.compiler.Compiler.processAnnotations(Compiler.java:839)
	at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:432)
	at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:364)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.compile(BatchImageBuilder.java:178)
	at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:301)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:60)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:254)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:173)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:728)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:199)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:239)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:292)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:295)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:351)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:374)
	at org.eclipse.core.internal.resources.Workspace.buildInternal(Workspace.java:513)
	at org.eclipse.core.internal.resources.Workspace.build(Workspace.java:432)
	at org.eclipse.ui.actions.BuildAction$1.runInWorkspace(BuildAction.java:305)
	at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:38)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)



-- Configuration Details --
Product: Eclipse 1.4.1.20110909-0613 (org.eclipse.epp.package.java.product)
Installed Features:
 org.eclipse.jdt 3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK
Comment 1 Ross Johnson CLA 2011-11-15 18:21:34 EST
I have also encountered the same exception. In my case on Eclipse 3.6.2 using the Hibernate JPA Metamodel Generator 1.1.1 annotation processor.

I have several projects in my workspace which use this processor. They all stopped working simultaneously (generating the same stack trace as reported above). Previously I have had difficulty getting it working again. Most recently, a restart of Eclipse was all that was needed.

I'm not sure if it contributed, but most recently the problem occurred after I had inadvertently opened the workspace with Eclipse 3.7.1 (before immediately switching to another workspace).
Comment 2 Joerg Schoenfisch CLA 2011-12-15 05:11:59 EST
The same here using the GWT RequestFactory:

java.lang.NullPointerException
	at org.eclipse.jdt.apt.core.internal.AptCompilationParticipant.addJava6GeneratedFile(AptCompilationParticipant.java:272)
	at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeProcessingEnvImpl.addNewUnit(IdeProcessingEnvImpl.java:132)
	at org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeJavaSourceOutputStream.close(IdeJavaSourceOutputStream.java:72)
	at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:320)
	at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
	at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
	at java.io.BufferedWriter.close(BufferedWriter.java:266)
	at java.io.PrintWriter.close(PrintWriter.java:339)
	at com.google.web.bindery.requestfactory.apt.DeobfuscatorBuilder.visitType(DeobfuscatorBuilder.java:146)

(Eclipse 3.7.1, JDK 1.7.0u01, Win7 x64)
Comment 3 Frank Harper CLA 2012-07-25 09:59:31 EDT
This bug just bit me while using AndroidAnnotations. Quitting Eclipse and restarting made everything work normally again.
Comment 4 Samuel Schmid CLA 2013-02-08 07:20:18 EST
I get the error on Request Factories and the workaround with restarting eclipse doesn't solve the problem.

java.lang.NullPointerException
    at org.eclipse.jdt.apt.core.internal.AptCompilationParticipant.addJava6GeneratedFile(AptCompilationParticipant.java:272)
    at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeProcessingEnvImpl.addNewUnit(IdeProcessingEnvImpl.java:132)
    at org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeJavaSourceOutputStream.close(IdeJavaSourceOutputStream.java:72)
    at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:301)
    at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:130)
    at java.io.OutputStreamWriter.close(OutputStreamWriter.java:216)
    at java.io.BufferedWriter.close(BufferedWriter.java:248)
    at java.io.PrintWriter.close(PrintWriter.java:295)
    at com.google.web.bindery.requestfactory.apt.DeobfuscatorBuilder.visitType(DeobfuscatorBuilder.java:146)
    at com.google.web.bindery.requestfactory.apt.DeobfuscatorBuilder.visitType(DeobfuscatorBuilder.java:43)
    at org.eclipse.jdt.internal.compiler.apt.model.TypeElementImpl.accept(TypeElementImpl.java:60)
    at javax.lang.model.util.ElementScanner6.scan(ElementScanner6.java:122)
    at com.google.web.bindery.requestfactory.apt.ScannerBase.scan(ScannerBase.java:63)
    at com.google.web.bindery.requestfactory.apt.State.executeJobs(State.java:248)
    at com.google.web.bindery.requestfactory.apt.RfValidator.process(RfValidator.java:86)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.handleProcessor(RoundDispatcher.java:139)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.round(RoundDispatcher.java:121)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.BaseAnnotationProcessorManager.processAnnotations(BaseAnnotationProcessorManager.java:159)
    at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeAnnotationProcessorManager.processAnnotations(IdeAnnotationProcessorManager.java:134)
    at org.eclipse.jdt.internal.compiler.Compiler.processAnnotations(Compiler.java:820)
    at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:434)
    at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:365)
    at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.compile(BatchImageBuilder.java:178)
    at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:302)
    at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:60)
    at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:254)
    at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:173)
    at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:728)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:199)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:239)
    at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:292)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:295)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:256)
    at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:394)
    at org.eclipse.core.internal.resources.Project$1.run(Project.java:618)
    at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2344)
    at org.eclipse.core.internal.resources.Project.internalBuild(Project.java:597)
    at org.eclipse.core.internal.resources.Project.build(Project.java:114)
    at org.eclipse.jdt.internal.ui.util.CoreUtility$BuildJob.run(CoreUtility.java:161)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)

I'm Usign MAC 10.7.4 and Eclipse Version: Juno Service Release 1
Comment 5 Missing name Mising name CLA 2014-11-07 05:01:44 EST
Hi all,

I've also faced this problem during developing of a custom annotation processor and I had the change to debug the problem. The reason for the NPE is a design issue of the AptCompilationParticipant class which should be used/handled as a kind of singleton. But the singleton instance is not created in singleton-pattern way as you can see in the constructor of AptCompilationParticipant. In the constructor on line 90 you see "INSTANCE = this;", which causes the singleton instance to change on every creation of an AptCompilationParticipant-object.

So beside the fact that this constructor is public and it is an "escape of the this reference" the code works fine under normal conditions. But in the case the NPE happens it doesn't because during the debugging I recognized that there where 2 instances of AptCompilationParticipant in my JVM.

So the normal steps of AptCompilationParticipant "seams to"/"should" be:
1. AptCompilationParticipant#aboutToBuild(IJavaProject project)
     this one creates all internal data structure
2. all lot of different calls during compilation and annotation processing (e.g.: addJava6GeneratedFile)
3. after the compile is done a final AptCompilationParticipant#buildComplete() is called from an internal listener
     this one removes the internal data structure

And heres the problem:

"aboutToBuild(IJavaProject project)" is called from "org.eclipse.jdt.internal.core.builder.JavaBuilder.initializeBuilder(int, boolean)" and this one treats the AptCompilationParticipant as a normal CompilationParticipant which is registered via an ExtensionPoint and the reference is kept in an array.

"addJava6GeneratedFile(IFile)" on the other side is called form the IdeProcessingEnvImpl during the close of "javax.tools.JavaFileObject#openOutputStream()" (=IdeJavaSourceOutputStream; required to compile the generated source file too). But here the IdeProcessingEnvImpl uses AptCompilationParticipant.getInstance() to call "addJava6GeneratedFile(IFile)".

So in the case of the NPE were 2 instances exist the "aboutToBuild(IJavaProject project)" is called on a cached reference by the JavaBuilder and the "addJava6GeneratedFile(IFile)" is called on the latest created AptCompilationParticipant instance by the IdeProcessingEnvImpl.  -->  The cached instance is the reason why only restarting your Eclipse can workaround this problem.

I didn't find the reason why there are sometime 2 instance, so far. But if I'm able to find this I will post my findings here too.
Comment 6 Missing name Mising name CLA 2014-11-07 05:04:40 EST
BTW, Bug 338117 seams to be the same as this one.
Comment 7 Jay Arthanareeswaran CLA 2014-11-07 05:13:27 EST
*** Bug 338117 has been marked as a duplicate of this bug. ***
Comment 8 Jay Arthanareeswaran CLA 2014-11-07 05:18:18 EST
(In reply to René Brandstetter from comment #5)

Thanks for the steps and investigation.

> I didn't find the reason why there are sometime 2 instance, so far. But if
> I'm able to find this I will post my findings here too.

The JDT team going through a resource crunch now, so, I am not sure how much time we can spare for this. I am looking forward to your finding.
Comment 9 Missing name Mising name CLA 2014-11-07 06:04:30 EST
Created attachment 248484 [details]
editor arrangement to reproduce

Hi again,

I've now also found the reason why the AptCompilationParticipant is created not only once but multiple times. The reason for this is hidden in the "org.eclipse.jdt.internal.ui.text.JavaReconciler" which is created for every "fully visible" editor on the screen.

To create multiple AptCompilationParticipant you only have to:
1. arrange multiple Java-Files next to each other (to see the content of all these files at once; see attachment)
2. close Eclipse
3. start it again and the org.eclipse.jdt.ui.text.JavaSourceViewerConfiguration will create a JavaReconciler for every editor you see

Each JavaReconciler will run in a separate thread and call the JavaModelManager, which thinks in every thread that it is running for the first time and causes the initialization of the CompilationParticipants. --> During startup multiple instances of AptCompilationParticipant will be created and in some rare race-condition situations it will happen that not the last available AptCompilationParticipant is kept by the array of CompilationParticipants.

So to temporary fix also the problem mentioned in comment #4, the workaround should be:
  "Close all your editor windows and restart your eclipse."

Regards,
  René



BTW, if you debug the creation of an AptCompilationParticipant you will see multiple threads with the following trace:

Daemon Thread [org.eclipse.jdt.internal.ui.text.JavaReconciler] (Suspended (breakpoint at line 90 in AptCompilationParticipant))
owns: Object  (id=71)
    AptCompilationParticipant.<init>() line: 90
    NativeConstructorAccessorImpl.newInstance0(Constructor<?>, Object[]) line: not available [native method]
    NativeConstructorAccessorImpl.newInstance(Object[]) line: 62
    DelegatingConstructorAccessorImpl.newInstance(Object[]) line: 45
    Constructor<T>.newInstance(Object...) line: 408
    Class<T>.newInstance() line: 433
    EquinoxRegistryStrategy(RegistryStrategyOSGI).createExecutableExtension(RegistryContributor, String, String) line: 184
    ExtensionRegistry.createExecutableExtension(RegistryContributor, String, String) line: 905
    ConfigurationElement.createExecutableExtension(String) line: 243
    ConfigurationElementHandle.createExecutableExtension(String) line: 55
    JavaModelManager$CompilationParticipants$1.run() line: 337
    SafeRunner.run(ISafeRunnable) line: 42
    JavaModelManager$CompilationParticipants.getCompilationParticipants(IJavaProject) line: 332
    ReconcileWorkingCopyOperation.notifyParticipants(CompilationUnit) line: 239
    ReconcileWorkingCopyOperation.executeOperation() line: 95
    ReconcileWorkingCopyOperation(JavaModelOperation).run(IProgressMonitor) line: 729
    ReconcileWorkingCopyOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 789
    CompilationUnit.reconcile(int, int, WorkingCopyOwner, IProgressMonitor) line: 1247
    JavaReconcilingStrategy.reconcile(ICompilationUnit, boolean) line: 126
    JavaReconcilingStrategy.access$0(JavaReconcilingStrategy, ICompilationUnit, boolean) line: 108
    JavaReconcilingStrategy$1.run() line: 89
    SafeRunner.run(ISafeRunnable) line: 42
    JavaReconcilingStrategy.reconcile(boolean) line: 87
    JavaReconcilingStrategy.initialReconcile() line: 178
    JavaCompositeReconcilingStrategy(CompositeReconcilingStrategy).initialReconcile() line: 114
    JavaCompositeReconcilingStrategy.initialReconcile() line: 136
    JavaReconciler(MonoReconciler).initialProcess() line: 105
    JavaReconciler.initialProcess() line: 406
    AbstractReconciler$BackgroundThread.run() line: 173
Comment 10 Missing name Mising name CLA 2014-11-11 03:11:37 EST
Hi all,

as you've probably seen I found the reason for those multiple instance and to provide a fix I would like to know how the AptCompilationParticipant should be treated? Should it really be a singleton or should there exist multiple different instances which should be organized by the Java-Compiler?

Because if it should be handled as a singleton I would change it to a real singleton (remove line "INSTANCE=this;" from the constructor) and I would replace the ExtensionPoint-Registration with an "IExecutableExtensionFactory" which always returns the static singleton instance.

But if it should be handled as different instances organized by the Compiler, I would have to search for all the AptCompilationParticipant.getInstance() and find a way to access the correct AptCompilationParticipant for that context.

Regards
  René
Comment 11 Missing name Mising name CLA 2014-11-24 07:33:32 EST
Hi,

I didn't receive any answers to my questions, so I decided to provide a patch for the real singleton approach.

The gerrit commit can be found at: https://git.eclipse.org/r/36913

Regards
  René
Comment 12 Jay Arthanareeswaran CLA 2014-11-24 10:35:21 EST
(In reply to René Brandstetter from comment #11)
> I didn't receive any answers to my questions, so I decided to provide a
> patch for the real singleton approach.
> 
> The gerrit commit can be found at: https://git.eclipse.org/r/36913

Sorry about it. I will take a look at the patch at the earliest possible.
Comment 13 Jay Arthanareeswaran CLA 2014-11-28 03:52:46 EST
(In reply to René Brandstetter from comment #11)
> The gerrit commit can be found at: https://git.eclipse.org/r/36913

Thanks for the patch, it looks good. Just noticed that you are working with Java 5 APT. We have plans to deprecate these (ref bug 424324) under the assumption that no one is using these. Looks like our assumption was wrong. 

Is there a reason why you wouldn't want to upgrade to Java 6 annotation processing API?
Comment 14 Jay Arthanareeswaran CLA 2014-11-28 04:47:39 EST
(In reply to Jayaprakash Arthanareeswaran from comment #13)
> Thanks for the patch, it looks good. Just noticed that you are working with
> Java 5 APT. We have plans to deprecate these (ref bug 424324) under the
> assumption that no one is using these. Looks like our assumption was wrong. 
> 
> Is there a reason why you wouldn't want to upgrade to Java 6 annotation
> processing API?

Never mind, I see this is not specific to Java 5 APT.
Comment 15 Jay Arthanareeswaran CLA 2014-11-28 05:58:32 EST
Patch looks good to me, released in master:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=67568cf7908374a12b930ce4d634e4a484a346ae
Comment 16 Manoj N Palat CLA 2014-12-10 04:31:35 EST
Verified for Eclipse Mars 4.5 M4 using build  I20141209-2000