Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 386816

Summary: [1.7] PolymorphicSignatureTest fails with J9VM, passes with Sun VM.
Product: [Eclipse Project] JDT Reporter: Srikanth Sankaran <srikanth_sankaran>
Component: CoreAssignee: Tomasz Zarna <tomasz.zarna>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: heidinga, stephan.herrmann
Version: 3.8   
Target Milestone: 4.3 M2   
Hardware: All   
OS: All   
URL: https://git.eclipse.org/r/#/c/7269
Whiteboard:
Attachments:
Description Flags
Another piece of code failing with IBM VM
none
batch file none

Description Srikanth Sankaran CLA 2012-08-08 06:04:13 EDT
org.eclipse.jdt.core.tests.compiler.regression.PolymorphicSignatureTest.test0002()
fails with IBM VM and passes with Oracle VM.

It is likely this is due to underlying implementation differences, this
needs to be understood.
Comment 1 Srikanth Sankaran CLA 2012-08-08 08:46:43 EDT
This seems to fail only from within the junit, on the IDE it runs fine.
even with J9VM - why ???
Comment 2 Srikanth Sankaran CLA 2012-08-08 08:55:47 EDT
Just to be safe: I verified that this failures shows up even when the
recent fix for bug 386259 is rolled back (via git hard reset).

Here is the failure trace:

		""
		"java.lang.TypeNotPresentException: Type [Ljava.lang.Object; not present\n" + 
		"	at java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:300)\n" + 
		"	at java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:356)\n" + 
		"	at java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:277)\n" + 
		"	at X.main(X.java:6)\n" + 
		"	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n" + 
		"	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)\n" + 
		"	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)\n" + 
		"	at java.lang.reflect.Method.invoke(Method.java:613)\n" + 
		"	at"
org.eclipse.jdt.core.tests.compiler.regression.PolymorphicSignatureTest#test0002 - 1.7
[ERR]:java.lang.TypeNotPresentException: Type [Ljava.lang.Object; not present
	at java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:300)
	at java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:356)
	at java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:277)
	at X.main(X.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at 
X.java
Comment 3 Srikanth Sankaran CLA 2012-08-08 20:44:19 EDT
Tom, can you investigate the mystery behind this ? TIA.
Comment 4 Tomasz Zarna CLA 2012-08-09 07:28:48 EDT
There must be something about the compiled class TestVerifier [1] doesn't like. The verification [2] returns false. The compiled class seems to be just fine, I was able to execute it in the command line.

[1] org.eclipse.jdt.core.tests.util.TestVerifier
[2] 
http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core.tests.compiler/src/org/eclipse/jdt/core/tests/compiler/regression/AbstractRegressionTest.java#n2280
Comment 5 Tomasz Zarna CLA 2012-08-09 07:42:34 EDT
Weird, when run only org.eclipse.jdt.core.tests.compiler.regression.PolymorphicSignatureTest.test0002() I got a different error:

----------\n
1. ERROR in X.java (at line 1)\r\n
	import static java.lang.invoke.MethodHandles.*; \r\n
	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
Syntax error, static imports are only available if source level is 1.5 or greater\n
----------\n
2. ERROR in X.java (at line 5)\r\n
	MethodHandle mh = dropArguments(insertArguments(identity(int.class), 0, 42), 0, Object[].class);\r\n
	                                                ^^^^^^^^\n
The method identity(Class<T>) is undefined for the type X\n
----------\n
3. ERROR in X.java (at line 6)\r\n
	int value = (int)mh.invokeExact(new Object[0]);\r\n
	            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
Cannot cast from Object to int\n
----------\n

I'm on :
java.fullversion=JRE 1.7.0 IBM J9 2.6 Windows XP x86-32 20110810_88604 (JIT enabled, AOT enabled)
J9VM - R26_Java726_GA_20110810_1208_B88592
JIT  - r11_20110810_20466
GC   - R26_Java726_GA_20110810_1208_B88592
J9CL - 20110810_88604
Comment 6 Srikanth Sankaran CLA 2012-08-09 07:49:00 EDT
(In reply to comment #5)
> Weird, when run only
> org.eclipse.jdt.core.tests.compiler.regression.PolymorphicSignatureTest.
> test0002() I got a different error:

This is what I was telling you about a couple of days ago :)

The reliable way to run a single test is to create a static block
a la mode:

static {
		TESTS_NAMES = new String [] { "test0002" };
}
Comment 7 Tomasz Zarna CLA 2012-08-09 08:09:01 EDT
(In reply to comment #6)
> This is what I was telling you about a couple of days ago :)

Doh, now I got the same exception as in comment 2.
Comment 8 Tomasz Zarna CLA 2012-08-13 08:51:07 EDT
The test turns green once Object[] is replaced with Object:
...
MethodHandle mh = dropArguments(insertArguments(identity(int.class), 0, 42), 0, Object.class);
int value = (int) mh.invokeExact(new Object());
...
Comment 9 Srikanth Sankaran CLA 2012-08-13 09:35:33 EDT
(In reply to comment #8)
> The test turns green once Object[] is replaced with Object:

Ah, that rang a bell. This test is brand new, added for https://bugs.eclipse.org/bugs/show_bug.cgi?id=385404 which necessarily
requires an array of Objects.

So why would verification fail and not the real run ? We need to look
at VM arguments and such used by loadAndRun and see if command line usage
with identical arguments also fail.

Could you figure out the full VM invocation options and document them here ?
Comment 10 Tomasz Zarna CLA 2012-08-13 10:14:57 EDT
Created attachment 219808 [details]
Another piece of code failing with IBM VM

Here is another piece of code failing with IBM VM but runs smoothly on Oracle VM. With the latter you should get something similar to: 

[Ljava.lang.Object;@fbb7cb 

With the former you will get an exception:

Exception in thread "main" java.lang.IndexOutOfBoundsException
	at java.lang.invoke.MethodType.dropParameterTypes(MethodType.java:187)
	at java.lang.invoke.MethodHandles.insertArguments(MethodHandles.java:1582)
	at test0003.X.main(X.java:7)
	
The code doesn't need to be run within junit to fail.
Comment 11 Tomasz Zarna CLA 2012-08-13 10:48:02 EDT
Created attachment 219810 [details]
batch file

(In reply to comment #9)
> Could you figure out the full VM invocation options and document them here ?

Here is a batch file containing the full VM options. Running it in the command line didn't fail, I got the expected "42".
Comment 12 Tomasz Zarna CLA 2012-08-13 10:54:00 EDT
The test passes once we tell RegressionTestSetup _not_ to reuse VM [1]. The flag is checked in TestVerifier.verifyClassFiles(String, String, String, String, String[], String[], String[]) [2].

[1] http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core.tests.compiler/src/org/eclipse/jdt/core/tests/compiler/regression/RegressionTestSetup.java#n21
[2] http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core.tests.compiler/src/org/eclipse/jdt/core/tests/util/TestVerifier.java#n625
Comment 13 Srikanth Sankaran CLA 2012-08-14 02:58:39 EDT
(In reply to comment #10)
> Created attachment 219808 [details]
> Another piece of code failing with IBM VM
> 
> Here is another piece of code failing with IBM VM but runs smoothly on
> Oracle VM. With the latter you should get something similar to: 
> 
> [Ljava.lang.Object;@fbb7cb 
> 
> With the former you will get an exception:
> 
> Exception in thread "main" java.lang.IndexOutOfBoundsException
> 	at java.lang.invoke.MethodType.dropParameterTypes(MethodType.java:187)
> 	at java.lang.invoke.MethodHandles.insertArguments(MethodHandles.java:1582)
> 	at test0003.X.main(X.java:7)
> 	
> The code doesn't need to be run within junit to fail.

Daniel, is this a bug in the IBM VM ? It looks like the call to insertArguments
itself is not very realistic - it tries to insert the values which is an empty list and the internal implementation of insertArguments ends up calling dropArguments in a way that violates the API.
Comment 14 Srikanth Sankaran CLA 2012-08-14 02:59:50 EDT
(In reply to comment #12)
> The test passes once we tell RegressionTestSetup _not_ to reuse VM [1].

We need to understand the usage differences between those two paths to
determine if this is a bug in out test infrastructure or a bug in the VM.
Comment 15 Srikanth Sankaran CLA 2012-08-14 19:24:14 EDT
(In reply to comment #10)
> Created attachment 219808 [details]
> Another piece of code failing with IBM VM

We have confirmation that this is a bug in J9VM (Thanks Dan). However, this
J9 bug has no bearing on the current topic and the present mystery needs to 
be solved independently.

Tom, I'll help with if it this is still open upon my return from vacation
(21 Aug)
Comment 16 Tomasz Zarna CLA 2012-08-16 07:43:26 EDT
When the VM is not reused, the test pass since launchAndRun(...) doesn't verify the compiled code. It's executed directly within the VM. If the flag is set to true (reuseVM == true), the VM is started for VerifyTests class with a socket accepting connections. Then, the VerifyTests class is asked to execute X (by sending "X" over socket).

A little bit longer stack trace (found when debugging), sheds more light on the issue:

[java.lang.TypeNotPresentException: Type [Ljava.lang.Object; not present
	at java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:300)
	at java.lang.invoke.MethodType.parseIntoClasses(MethodType.java:356)
	at java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:277)
	at X.main(X.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:613)
	at org.eclipse.jdt.core.tests.util.VerifyTests.loadAndRun(VerifyTests.java:187)
	at org.eclipse.jdt.core.tests.util.VerifyTests$1.run(VerifyTests.java:213)
Caused by: java.lang.ClassNotFoundException
	at org.eclipse.jdt.core.tests.util.VerifyTests$VerifyClassLoader.loadClass(VerifyTests.java:128)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:658)
	at java.lang.invoke.MethodType.nonPrimitiveClassFromString(MethodType.java:298)
	... 9 more
Comment 17 Tomasz Zarna CLA 2012-08-16 08:22:46 EDT
For some reason VerifyTests.VerifyClassLoader.loadClass(...)[1] within J9 tries to load class named "[Ljava.lang.Object;" and fails. This is the class-not-found from comment 16. For the same test run under Oracle VM the complete list [2] of loaded classes is:

java.lang.Object
java.lang.String
java.lang.Throwable
java.lang.Integer
java.lang.invoke.MethodHandles
java.lang.Class
java.lang.invoke.MethodHandle
java.lang.System
java.io.PrintStream

There is no sign of array of Object.

[1] org.eclipse.jdt.core.tests.util.VerifyTests.VerifyClassLoader.loadClass(String, boolean)
[2] from a sysout planted in the first line of [1]
Comment 18 Tomasz Zarna CLA 2012-08-17 06:43:21 EDT
Fix proposal pushed to Gerrit: https://git.eclipse.org/r/#/c/7269

J9's system class loader can handle [Ljava.lang.Object; (and other java.lang.Classes representing the class of array of objects) but was not served with them since they were not recognized as excluded. This must be an implementation difference between J9 and Oracle VMs. Once J9's system class loader is asked to load [Ljava.lang.Object; everything is back to normal.
Comment 19 Tomasz Zarna CLA 2012-08-20 03:46:41 EDT
Fixed with 2eab698c606e3b17217602bbc99837242c743a5a
Comment 20 Stephan Herrmann CLA 2012-08-30 06:31:00 EDT
By chance I found http://bugs.sun.com/view_bug.do?bug_id=6434149 which mentions that Sun VM 1.5- accidentally accepts array syntax in loadClass() which has been fixed (=don't accept) for jdk1.6.

No impact on this bug, just cross-linking what I just learned.
Comment 21 Srikanth Sankaran CLA 2012-09-17 10:39:36 EDT
Verified for 4.3 M2 - This test suite issue does not show up anymore
with IBM SDK.