Bug 423310

Comment 1 Timo Kinnunen 2013-12-06 05:37:42 EST

Further investigation reveals that many of those tests take having full read and write access to workspace parent directory for granted and some even require it. Alternative fix: 

Changing the suggested default workspace location of RunJDTCoreTests launch configuration from ${workspace_loc:/../junit-workspace} to ${system_property:java.io.tmpdir}/users/user/junit-workspace

Comment 2 Timo Kinnunen 2013-12-06 08:11:00 EST

After rewinding most of the changes to tests itself, 6 failures each in 1.4, 1.5 and 1.6 compiler tests remained, 18 total. The 5 failures from tests MethodVerifyTest.test091, test092, test093, VarargsTest.test066, GenericTypeTest.test0809 were from an extra warning about a missing @Override annotation. These were caused by running the tests using a Java 8 early access VM and thus its runtime library. Overridden default methods looking like they're overridden superclass methods, I'd say.

The last failure in test BatchCompilerTest.test405225_extdirs was an NPE. This was caused by the compiler not detecting a 1.8 JDK-interface-using runtime and defaulting to the earliest known version. The way this NPE is triggered seems like a real bug:

junit.framework.AssertionFailedError: Unexpected problems [out: ][err: java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.apt.util.EclipseFileManager.concatFiles(EclipseFileManager.java:204)
	at org.eclipse.jdt.internal.compiler.apt.util.EclipseFileManager.handleOption(EclipseFileManager.java:674)
	at org.eclipse.jdt.internal.compiler.apt.dispatch.BatchProcessingEnvImpl.<init>(BatchProcessingEnvImpl.java:88)
	at org.eclipse.jdt.internal.compiler.apt.dispatch.BatchAnnotationProcessorManager.configure(BatchAnnotationProcessorManager.java:69)
	at org.eclipse.jdt.internal.compiler.batch.Main.initializeAnnotationProcessorManager(Main.java:3969)
	at org.eclipse.jdt.internal.compiler.batch.Main.performCompilation(Main.java:4087)
	at org.eclipse.jdt.internal.compiler.batch.Main.compile(Main.java:1692)
	at org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.invokeCompiler(BatchCompilerTest.java:646)
	at org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.runTest(BatchCompilerTest.java:570)
	at org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.runConformTest(BatchCompilerTest.java:435)
	at org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.test405225_extdirs(BatchCompilerTest.java:13788)

After fixing these 18 failures RunJDTCoreTests now passes. This is with file access restrictions and all required projects switched to 1.6 still in place.

Comment 3 Timo Kinnunen 2013-12-06 15:53:59 EST

Patch in Gerrit: https://git.eclipse.org/r/19462 

Resulted in a quick build failure, reason connection failure? What does that mean?

Comment 4 Stephan Herrmann 2013-12-06 16:14:45 EST

(In reply to Timo Kinnunen from comment #2)
> After rewinding most of the changes to tests itself, 6 failures each in 1.4,
> 1.5 and 1.6 compiler tests remained, 18 total. The 5 failures from tests
> MethodVerifyTest.test091, test092, test093, VarargsTest.test066,
> GenericTypeTest.test0809 were from an extra warning about a missing
> @Override annotation. These were caused by running the tests using a Java 8
> early access VM and thus its runtime library. Overridden default methods
> looking like they're overridden superclass methods, I'd say.

Timo, trouble using master on a Java 8 VM is expected (read: unavoidable). Have you checked the content of branch BETA_JAVA8 against your findings? If changes particular for 1.8 will be made, that branch is where it happens.

For work on master please use a Java 7 VM, thanks! This will also help to focus on the problems at hand :)

Comment 5 Timo Kinnunen 2013-12-06 16:57:10 EST

It is an interesting failure mode though? In that the compiler tests are not completely hermetically sealed and can pick up class files from their runtime environment. This shouldn't be possible, right!

The version detection one where a Java 8 JRE is interpreted as a 1.3 Java JRE I've seen in the IDE itself. I had to turn on project specific compliance settings one project at a time as each project unblocked the next when I started running Eclipse on Java8 EA.

Next steps, trying to run the tests under a Java 7 IDE and with maven from the command-line to see if something turns up, then.

Comment 6 Stephan Herrmann 2013-12-06 18:24:10 EST

(In reply to Timo Kinnunen from comment #5)
> It is an interesting failure mode though? In that the compiler tests are not
> completely hermetically sealed and can pick up class files from their
> runtime environment. This shouldn't be possible, right!

When we test compiling some Java programs (the test data), we need to compile these programs against some JRE. If you run on a JRE 8 the tests that challenge the Java 7- compiler do not find a compatible JRE to link against. In that sense the JRE is part of the test data. Without a JRE 7 these tests cannot succeed. This may sound surprising, because you might think that a JRE 8 is a compatible replacement for a JRE 7, but this is not the case, as you just experienced.

Other test suites work with a "JCL_MIN", a stub library replacing the JRE as part of the test data. This makes those specific test suites self-contained as you request. The compiler.regression tests are not constructed this way, because we're not only compiling against that JRE but we also execute the compiled program to verify correct behavior, which we could not do with a stub JRE. 

> The version detection one where a Java 8 JRE is interpreted as a 1.3 Java
> JRE I've seen in the IDE itself. I had to turn on project specific
> compliance settings one project at a time as each project unblocked the next
> when I started running Eclipse on Java8 EA.

The problem is: we are not allowed (legally) to let JDT (master) recognize a JRE 8. It might be possible to guess that any version greater than the greatest known version be mapped to that greatest known version, which would let JDT (master) recognize the JRE 8 as version 7. I don't recall if this has been discussed.

OTOH, by fixing corresponding issues in BETA_JAVA8 we expect to be clean in this regard after the Java 8 GA, so I don't see the 7/8 story as s.t. we'd need to fix in master (if we could).
 
> Next steps, trying to run the tests under a Java 7 IDE and with maven from
> the command-line to see if something turns up, then.

Thanks!

Comment 7 Srikanth Sankaran 2013-12-06 18:35:47 EST

(In reply to Stephan Herrmann from comment #6)
> (In reply to Timo Kinnunen from comment #5)
> > It is an interesting failure mode though? In that the compiler tests are not
> > completely hermetically sealed and can pick up class files from their
> > runtime environment. This shouldn't be possible, right!

> When we test compiling some Java programs (the test data), we need to
> compile these programs against some JRE. If you run on a JRE 8 the tests
> that challenge the Java 7- compiler do not find a compatible JRE to link
> against. In that sense the JRE is part of the test data. Without a JRE 7

Yep. In fact if the tests are sealed - it would inject a dose of artificiality
to the testing process. User code runs against a JRE and we need to test against
a JRE.


> Other test suites work with a "JCL_MIN", a stub library replacing the JRE as
> part of the test data. This makes those specific test suites self-contained
> as you request. 

My understanding is that this is done more for the reasons of having to avoid
reindexing JREs over and over against different "workspaces" and "projects" used 
by the tens of thousands of tests - less so for testing in a sealed environment.
AFAICT, that is not a factor.

Comment 8 Eclipse Genie 2020-02-08 12:15:16 EST

This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.