Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 563442 - Regression: Post 4.16M2 byte code is incompatible with pre 4.16M2
Summary: Regression: Post 4.16M2 byte code is incompatible with pre 4.16M2
Status: CLOSED NOT_ECLIPSE
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.16   Edit
Hardware: PC Windows 10
: P3 normal (vote)
Target Milestone: 4.16 RC1   Edit
Assignee: Stephan Herrmann CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 563068
  Show dependency tree
 
Reported: 2020-05-21 14:08 EDT by Ed Willink CLA
Modified: 2020-05-22 14:32 EDT (History)
1 user (show)

See Also:


Attachments
Disassembly (1.41 KB, text/plain)
2020-05-21 14:23 EDT, Ed Willink CLA
no flags Details
Correct disassembly (2.58 MB, text/plain)
2020-05-21 15:49 EDT, Ed Willink CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2020-05-21 14:08:02 EDT
Original bug 563068

The following call

         setWithLastConsumed(current, "isTypeof", true, "typeof");

to 

	protected void setWithLastConsumed(EObject _this, String feature, Object value, String lexerRule) {
		set(_this, feature, value, lexerRule, lastConsumedNode);
	}


has 4 arguments of which the third requires boxing.

Prior to 14.6M2 the generated code boxed and called 4 arguments.

If compiled post 4.16M2 there is instead a 3 argument call omitting the boxed argument that results in a NoSuchMethodError.

Given that the 3 argument seems to be self consistent the problem seems to arise when a user mixes calling code in a plugin built post 4.16M2 with an earlier plugin, e.g. the 2020-03 release built pre-4.16M2 where the missing boxed argument optimization was not supported.
Comment 1 Ed Willink CLA 2020-05-21 14:23:36 EDT
Created attachment 282981 [details]
Disassembly

Attached disassembly shows that some calls to 

setWithLastConsumed:(Lorg/eclipse/emf/ecore/EObject;Ljava/lang/String;ZLjava/lang/String;)V

have 3 arguments giving the NoSuchMethodError while others have 4

setWithLastConsumed:(Lorg/eclipse/emf/ecore/EObject;Ljava/lang/String;Ljava/lang/Object;Ljava/lang/String;)V
Comment 2 Stephan Herrmann CLA 2020-05-21 15:06:58 EDT
I tried to reproduce like this:
//---
public class Test {
    void m(Object _this, String feature, Object value, String rule) {
        System.out.println(value);
    }
    public static void main(String... args) {
        Test t = new Test();
        t.m(t, "isTypeof", true, "typeof");
    }
}
//---

No problem here, compiles and prints:
true

(In reply to Ed Willink from comment #1)
> Created attachment 282981 [details]
> Disassembly
> 
> Attached disassembly shows that some calls to 

This doesn't look like the file you intended to attach :)

> setWithLastConsumed:(Lorg/eclipse/emf/ecore/EObject;Ljava/lang/String;ZLjava/
> lang/String;)V

wait, somewhere you must have a method with a boolean arg3, see the 'Z' parameter!

Please provide a complete example. Thanks.
Comment 3 Stephan Herrmann CLA 2020-05-21 15:12:54 EDT
Variant:
//---
public class Test {
    void m(Object _this, String feature, Object value, String rule) {
        System.out.println("Object:"+value);
    }
    void m(Object _this, String feature, boolean value, String rule) {
        System.out.println("boolean:"+value);
    }
    public static void main(String... args) {
        Test t = new Test();
        t.m(t, "isTypeof", true, "typeof");
        t.m(t, "isTypeof", (Object)true, "typeof");
    }
}
//---

$ ecj -1.8 Test.java
$ java Test
boolean:true
Object:true


bytecode for the two invocations (showing signatures like comment 1):

         8: aload_1
         9: aload_1
        10: ldc           #48                 // String isTypeof
        12: iconst_1
        13: ldc           #50                 // String typeof
        15: invokevirtual #52                 // Method m:(Ljava/lang/Object;Ljava/lang/String;ZLjava/lang/String;)V


        18: aload_1
        19: aload_1
        20: ldc           #48                 // String isTypeof
        22: iconst_1
        23: invokestatic  #54                 // Method java/lang/Boolean.valueOf:(Z)Ljava/lang/Boolean;
        26: ldc           #50                 // String typeof
        28: invokevirtual #60                 // Method m:(Ljava/lang/Object;Ljava/lang/String;Ljava/lang/Object;Ljava/lang/String;)V

identical for ecj 4.15 and current HEAD.
Comment 4 Ed Willink CLA 2020-05-21 15:49:39 EDT
Created attachment 282984 [details]
Correct disassembly
Comment 5 Ed Willink CLA 2020-05-21 15:50:16 EDT
Comment on attachment 282981 [details]
Disassembly

Wrong file
Comment 6 Ed Willink CLA 2020-05-21 15:59:08 EDT
(In reply to Stephan Herrmann from comment #2)
> wait, somewhere you must have a method with a boolean arg3, see the 'Z'
> parameter!
> 
> Please provide a complete example. Thanks.

The calling code is

https://git.eclipse.org/r/plugins/gitiles/ocl/org.eclipse.ocl/+/master/plugins/org.eclipse.ocl.xtext.oclstdlib/src-gen/org/eclipse/ocl/xtext/oclstdlib/parser/antlr/internal/InternalOCLstdlibParser.java

The called code which does not have a 'Z' variant is Xtext.

Is this possibly a Java 14ism that has accidentally leaked back to break Java 8 users?
Comment 7 Stephan Herrmann CLA 2020-05-21 17:51:49 EDT
(In reply to Ed Willink from comment #4)
> Created attachment 282984 [details]
> Correct disassembly

This contains a happy mix of both signatures, but how am I supposed to analyse? It seems that xtext provides overloaded methods of that name? 

Seeing the calling code without the declarations of methods being called tells me nothing. In what you show me there's no hint of ecj being at error.

Pretty please isolate a standalone example.
Comment 8 Ed Willink CLA 2020-05-22 00:52:16 EDT
(In reply to Stephan Herrmann from comment #7)
> Pretty please isolate a standalone example.

That's not the way to go.

The clear observation is that there are now two signatures.

When I single step normally on my machine, I step through a boxing cll before entering the routine.

When I single step on the same machine but using Tycho to provide a 2020-03 platform, I step straight to NSME.

I have leapt to the conclusion that new ECJ has changed. Another possibility is that a new Xtext, only installed on the new build platform has this second signature, so a new build uses it without crashing but an execution of the new build on an old Xtext crashes. Need to dig into the myriad Xtext GITs.
Comment 9 Stephan Herrmann CLA 2020-05-22 05:53:15 EDT
(In reply to Ed Willink from comment #8)

It's your lucky day: I happen to have a dev workspace for Xtext.

From there I can see that the issue is caused by the following commit in the xtext-core repo:

----------------------------------------------------------------------------
commit 1647521e6354732bbe01c0ca5a3fff08beec3d34
Sebastian Zarnekow <Sebastian.Zarnekow@xxx.yy> on 2020-05-04 13:47:16
[#1462] Syntax errors will lead to unexpected calls to value converters (#1463)

Don't call a value converter if nothing was matched. Also don't assign naively true for boolean values without checking if there was a token consumed.

closes #1462
----------------------------------------------------------------------------

This commit adds the following method:
	protected void setWithLastConsumed(EObject _this, String feature, boolean value, String lexerRule) {

Hence, where a boolean is provided, it is correct to invoke the new method, rather then apply boxing and invoke the old method.

Apparently, you were compiling against a new version but running against an older version of Xtext.
Comment 10 Ed Willink CLA 2020-05-22 07:32:31 EDT
Seems I neglected to save this. It is ECLIPSE since Xtext is ECLIPSE.

(In reply to Ed Willink from comment #8)
> Need to dig into the myriad Xtext GITs.

https://github.com/eclipse/xtext-core/blob/master/org.eclipse.xtext/src/org/eclipse/xtext/parser/antlr/AbstractInternalAntlrParser.java

shows:


	protected void setWithLastConsumed(EObject _this, String feature, Object value, String lexerRule) {
		if (value != null) {
			set(_this, feature, value, lexerRule, lastConsumedNode);
		}
	}
	
	protected void setWithLastConsumed(EObject _this, String feature, boolean value, String lexerRule) {
		if (value) {
			set(_this, feature, value, lexerRule, lastConsumedNode);
		}
	}

Problem explained.

---

It's a strange change that appears to permitted API-wise.
Comment 11 Ed Willink CLA 2020-05-22 07:56:26 EDT
(In reply to Ed Willink from comment #10)
> It's a strange change that appears to permitted API-wise.

Bug 563476 raised to at least document the Xtext 2.22 compatibility problem and to provide three workarounds that Xtext could pursue, and three workarounds that users can pursue.
Comment 12 Stephan Herrmann CLA 2020-05-22 12:37:38 EDT
(In reply to Ed Willink from comment #11)
> (In reply to Ed Willink from comment #10)
> > It's a strange change that appears to permitted API-wise.
> 
> Bug 563476 raised to at least document the Xtext 2.22 compatibility problem
> and to provide three workarounds that Xtext could pursue, and three
> workarounds that users can pursue.

Isn't this one of the situations where overloading breaks compatibility as soon as your sources are re-compiled?

Did I mention how much "fun" overloading is? Still people think it's cool. <shrug>
Comment 13 Ed Willink CLA 2020-05-22 14:32:53 EDT
Yes. But at least injecting an inheritance between user code and Xtext code allows both overloads to always be present. Problem solved. But hopefully Xtext will clean up the disgraceful change.