Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 331378

Summary: Serializing an expression DSL with parentheses and different operator priorities looses the parentheses
Product: [Modeling] TMF Reporter: Cedric Vidal <c.vidal>
Component: XtextAssignee: Project Inbox <tmf.xtext-inbox>
Status: CLOSED FIXED QA Contact:
Severity: major    
Priority: P3 CC: moritz.eysholdt
Version: 1.0.0Flags: moritz.eysholdt: helios+
moritz.eysholdt: indigo+
Target Milestone: ---   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Attachments:
Description Flags
operator_as_crossref_throws_exception Testcase
none
operator_name_as_rule_fails_to_serialize_parentheses Testcase
none
operator_name_as_terminal_serializes_fine Testcase none

Description Cedric Vidal CLA 2010-11-29 15:39:38 EST
Build Identifier: I20100608-0911

The following grammar serializes correctly:

Expression : Addition;

Addition returns Expression :
    Multiplication ({Operation.left=current} op=OPERATOR_ADD right=Multiplication) *
    ;

Multiplication returns Expression :
    Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
    ;

Literal returns Expression:
    '(' Expression ')'
    | {IntLiteral} value=INT
    ;

terminal OPERATOR_MUL : '*'|'/';
terminal OPERATOR_ADD : '+'|'-';

When using rules instead of terminals, serialization fails to include parentheses:

Expression : Addition;

Addition returns Expression :
    Multiplication ({Operation.left=current} op=OPERATOR_ADD right=Multiplication) *
    ;

Multiplication returns Expression :
    Literal ({Operation.left=current} op=OPERATOR_MUL right=Literal) *
    ;

Literal returns Expression:
    '(' Expression ')'
    | {IntLiteral} value=INT
    ;

OPERATOR_MUL : '*'|'/'; // <-- using rules instead of terminals
OPERATOR_ADD : '+'|'-';

--> parentheses are not serialized

When operators are specific meta classes which are referenced, it also fails. The following DSL illustrates that. It adds the possibility to define operators and reference them.

Evaluation:
    (ops+=Operator ';') *
    'exp' exp=Expression
    ;

Expression : Addition;

Addition returns Expression :
    Multiplication ({Operation.left=current} op=[Operator|OPERATOR_ADD] right=Multiplication) *
    ;

Multiplication returns Expression :
    Literal ({Operation.left=current} op=[Operator|OPERATOR_MUL] right=Literal) *
    ;

Literal returns Expression:
    '(' Expression ')'
    | {IntLiteral} value=INT
    ;

terminal OPERATOR_MUL : '*'|'/';
terminal OPERATOR_ADD : '+'|'-';

Operator :
    'op' name=(OPERATOR_ADD|OPERATOR_MUL)
    ;

--> fails with "org.eclipse.xtext.conversion.ValueConverterException: The value '*' is an invalid OPERATOR_ADD"


Reproducible: Always
Comment 1 Cedric Vidal CLA 2010-11-29 15:47:56 EST
Created attachment 184078 [details]
operator_as_crossref_throws_exception Testcase
Comment 2 Cedric Vidal CLA 2010-11-29 15:48:49 EST
Created attachment 184079 [details]
operator_name_as_rule_fails_to_serialize_parentheses Testcase
Comment 3 Cedric Vidal CLA 2010-11-29 15:49:21 EST
Created attachment 184080 [details]
operator_name_as_terminal_serializes_fine Testcase
Comment 4 Cedric Vidal CLA 2010-11-29 15:50:09 EST
Attached archives contain full projects with a testcase which demonstrate the issue.
Comment 5 Moritz Eysholdt CLA 2010-12-03 08:51:30 EST
Fixed in 'master' (1) and 'Helios_Maintenance' (2).

The problem was, that the serializer's backtracking algorithm did not take into account whether a cross reference's token complies with the cross reference's
terminal- or data type rule.

Cedric, thanks for your support.

(1):
http://git.eclipse.org/c/tmf/org.eclipse.xtext.git/commit/?id=132a8d5d818a97c6b74b24613663fa76e3345f71

(2):
http://git.eclipse.org/c/tmf/org.eclipse.xtext.git/commit/?h=Helios_Maintenance&id=298e85916079809a29096049ef302a9ed6a18f51