Community
Participate
Working Groups
Trying to build JGit running mvn and thus ECJ on Java 11 but generating for Java 8 (--release 8) runs into a concurrency problem. ECJ used is ecj-I20210622-1800.jar, mvn is 3.6.3. A simple mvn clean install for JGit tells me it's using a MultiThreadedBuilder with a thread count of 4, and that runs fairly quickly into this: [DEBUG] java.nio.file.ClosedFileSystemException at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.ensureOpen(ZipFileSystem.java:1146) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.checkAccess(ZipFileSystem.java:362) at jdk.zipfs/jdk.nio.zipfs.ZipPath.checkAccess(ZipPath.java:843) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.checkAccess(ZipFileSystemProvider.java:185) at java.base/java.nio.file.Files.exists(Files.java:2440) at org.eclipse.jdt.internal.compiler.batch.ClasspathJep247.findClass(ClasspathJep247.java:84) at org.eclipse.jdt.internal.compiler.batch.ClasspathJsr199.findClass(ClasspathJsr199.java:83) at org.eclipse.jdt.internal.compiler.batch.FileSystem.internalFindClass(FileSystem.java:472) at org.eclipse.jdt.internal.compiler.batch.FileSystem.findClass(FileSystem.java:414) at org.eclipse.jdt.internal.compiler.batch.FileSystem.findType(FileSystem.java:564) at org.eclipse.jdt.internal.compiler.env.IModuleAwareNameEnvironment.findType(IModuleAwareNameEnvironment.java:101) at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:299) at org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:276) at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.getDefaultImports(CompilationUnitScope.java:687) at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.checkAndSetImports(CompilationUnitScope.java:240) at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.completeTypeBindings(LookupEnvironment.java:513) at org.eclipse.jdt.internal.compiler.Compiler.internalBeginToCompile(Compiler.java:878) at org.eclipse.jdt.internal.compiler.Compiler.beginToCompile(Compiler.java:394) at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:448) at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:426) at org.eclipse.jdt.internal.compiler.batch.Main.performCompilation(Main.java:4790) at org.eclipse.jdt.internal.compiler.tool.EclipseCompilerImpl.call(EclipseCompilerImpl.java:99) at org.eclipse.jdt.internal.compiler.tool.EclipseCompiler$1.call(EclipseCompiler.java:196) at org.codehaus.plexus.compiler.eclipse.EclipseJavaCompiler.performCompile(EclipseJavaCompiler.java:417) at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1134) at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:187) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190) at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Don't know how to put together a small example for this; in any case it appears the JSR-199 compiler is not thread-safe. A similar problem as in bug 563501, where a cached CtSym instance was closed and then re-used? But maybe the stack trace alone helps already? Running the build single-threaded (mvn -T 1) works.
Thats intersting (not intuitive): java.nio.file.Files.exists() @return false if the file does not exist *or its existence cannot be determined*. However: as far as i know jdt is not ready for parallel compile anyway (bug 531554). Work on this was stop. see https://bugs.eclipse.org/bugs/show_bug.cgi?id=531554#c50
(In reply to Jörg Kubitz from comment #1) > However: as far as i know jdt is not ready for parallel compile anyway (bug > 531554). Work on this was stop. see > https://bugs.eclipse.org/bugs/show_bug.cgi?id=531554#c50 That bug is about scheduling rules inside Eclipse for the Java Builder, not about multi-threaded maven builds. Off-topic: I was rather astonished to find that JGit uses -T 1C by default (one thread per CPU) via .mvn/maven.config. -T 1 is just as well for JGit; shouldn't be much slower. Note that with ECJ 3.25.0, the multi-threaded maven build for JGit works fine (running on Java 8 and generating for Java 8, so no --release option). The stack trace also indicates that this is a problem with some cached zip (jar) that is used only when --release is present inadvertently being shared and being reset in one thread while still being used in other threads. That's the reason why I find bug 563501 much more relevant here.
Causes https://github.com/eclipse/tycho/issues/183
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/183030
Thomas, Mickael : could you try to use your maven builds with the ecj jar linked here: https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/5295/ https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/5295/artifact/org.eclipse.jdt.core/target/org.eclipse.jdt.core-3.27.0-SNAPSHOT-batch-compiler.jar I think that this should work now.
(In reply to Andrey Loskutov from comment #5) > Thomas, Mickael : could you try to use your maven builds with the ecj jar > linked here: Seems to work for the JGit build with 4 threads with that JAR.
Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/183030 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=1b756cc2eb17dc864ba5965d33c28e97092e2796
Deployed ECJ as <groupId>org.eclipse.jdt</groupId> <artifactId>ecj</artifactId> <version>3.27.0.v20210713-2148</version> Repository <pluginRepository> <id>cbi-jdt</id> <url>https://repo.eclipse.org/content/repositories/eclipse-staging/</url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>true</enabled> </snapshots> </pluginRepository>
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.releng.aggregator/+/183055
Gerrit change https://git.eclipse.org/r/c/platform/eclipse.platform.releng.aggregator/+/183055 was merged to [master]. Commit: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=93fc5db7cdc4e80568c09e0c0f63316819988e44
(In reply to Sravan Kumar Lakkimsetti from comment #8) > Deployed ECJ as > > <groupId>org.eclipse.jdt</groupId> > <artifactId>ecj</artifactId> > <version>3.27.0.v20210713-2148</version> Yes, also works for the JGit build with 4 threads with that deployed ECJ.
As confirmed by Thomas, marking as verified.
