Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 315559

Summary: [formatter/matcher] Unordered groups are not handled correctly
Product: [Modeling] TMF Reporter: Johan Wannheden <johan.wannheden>
Component: XtextAssignee: Project Inbox <tmf.xtext-inbox>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: asheiduk, benjamin.schwertfeger, btickets, carl.wannheden, horst, moritz.eysholdt, philipp.salvisberg, sebastian.zarnekow, stephane, sven.efftinge
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
Early Tests, Debug output none

Description Johan Wannheden CLA 2010-06-03 07:11:27 EDT 
Build Identifier: 20100527-0614

It seems that sometimes the formatter does not insert new lines where expected. It seems to happen after unassigned keywords.

Steps to reproduce:

1. Create the following grammar:

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals

generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"

Model :
  'model' name=ID
  (elements+=Element)*
;

Element :
  'element' id=INT
  (
    ('statement' (statements+=Statement)*)? &
    ('name' name=ID)
  )
;

Statement :
  '(' name=ID ')'
;

2. Implement the following formatter rules:

org.xtext.example.mydsl.services.MyDslGrammarAccess f = (org.xtext.example.mydsl.services.MyDslGrammarAccess) getGrammarAccess();

c.setLinewrap(0, 1, 2).before(f.getSL_COMMENTRule());
c.setLinewrap(0, 1, 2).before(f.getML_COMMENTRule());
c.setLinewrap(0, 1, 1).after(f.getML_COMMENTRule());

ModelElements modelAccess = f.getModelAccess();
c.setLinewrap(2).after(modelAccess.getNameAssignment_1());

ElementElements elementAccess = f.getElementAccess();
c.setLinewrap(2).before(elementAccess.getElementKeyword_0());
c.setLinewrap().before(elementAccess.getNameKeyword_2_1_0());
c.setLinewrap().before(elementAccess.getStatementKeyword_2_0_0());

StatementElements statementAccess = f.getStatementAccess();
c.setNoSpace().around(statementAccess.getLeftParenthesisKeyword_0());
c.setNoSpace().around(statementAccess.getRightParenthesisKeyword_2());

3. And test with the following model instance:

model test

element 1
statement (b)(c)
name apple

element 2
name banana
statement (x)(y)(z)

which should already be more or less formatted according to the rules. Unfortunately, formatting yields the following result:

model test

element 1
statement (b)(c)name apple

element 2
name banana
statement (x)(y)(z)

That is, there are not new lines before "name" (after the unassigned keywords). Note that for the "banana case" it worked fine.

Reproducible: Always
Comment 1 Carl Wannheden CLA 2010-06-28 08:20:51 EDT 
When changing

c.setNoSpace().around(statementAccess.getRightParenthesisKeyword_2());

to

c.setNoSpace().before(statementAccess.getRightParenthesisKeyword_2());

the formatter works as expected / intended.
Comment 2 Moritz Eysholdt CLA 2010-08-23 05:43:21 EDT 
Created attachment 177200 [details]
Early Tests, Debug output

I started working on this, but the impact turns out to be to huge for SR1. The patch contains test cases which produce debug output that shows what went wrong.
Comment 3 Benjamin Schwertfeger CLA 2012-08-24 04:18:05 EDT 
I have the same problem with xtext 2.3.0. My grammar contains the unordered group

(
& ('pages' pages+=PageRef (',' pages+=PageRef)*)?
& ('preconditions' (preConditions+=Precondition)+)?
)

And the formatter contains

c.setIndentationIncrement().before(ga.getPreconditionRule());
c.setIndentationDecrement().after(ga.getPreconditionRule());

This formats correctly, if the model contains the keywords in order of the definitions in the (un)ordered group. If I change the order The formatting for the direct containing Modelelements seems to be ok, but the siblings are indented.
Comment 4 Benjamin Schwertfeger CLA 2012-08-24 07:04:41 EDT 
Additional observation:
I tried to create a simple sample, like the one above, and had to add the decrement after the rule which contains the unordered group. Additionally the unordered group contains two rule calls. The indent decrement is only called, if the unordered group has the "right" order:

Models :
	m += Model*
;

Model:
	'model' name=ID
	(
		('greet' greetings+=Greeting+)?
		& ('test' test+=Content+)?
	);

Content:
	name=STRING
;

Greeting:
	'Hello' name=ID '!';



public class MyDslFormatter extends AbstractDeclarativeFormatter {

	@Override
	protected void configureFormatting(FormattingConfig c) {
		MyDslGrammarAccess ga = (MyDslGrammarAccess) getGrammarAccess();
		c.setLinewrap().before(ga.getModelAccess().getModelKeyword_0());
		c.setLinewrap().before(ga.getModelAccess().getGreetKeyword_2_0_0());
		c.setLinewrap().before(ga.getModelAccess().getTestKeyword_2_1_0());
		c.setLinewrap().before(ga.getContentAccess().getNameAssignment());

		c.setLinewrap().before(ga.getGreetingAccess().getHelloKeyword_0());

		c.setIndentationIncrement().before(ga.getGreetingRule());
		c.setIndentationDecrement().after(ga.getGreetingRule());

		c.setIndentationIncrement().before(ga.getContentRule());
		c.setIndentationDecrement().after(ga.getContentRule());

		c.setIndentationIncrement().after(
				ga.getModelAccess().getNameAssignment_1());
		// This one is not called correctly
		c.setIndentationDecrement().after(ga.getModelRule());
	}
}


And the model.mydsl:

model one
	greet
		Hello a
		Hello b
		Hello c
	test
model one
	greet
		Hello a
		Hello b
		Hello c
	test
Comment 5 Andreas Horst CLA 2014-05-23 07:27:33 EDT 
I can confirm this behavior for Xtext SDK 2.5.4.v201404100756: unordered groups only get correctly formatted if the concrete syntax is in fact "ordered" according to the definition of the unordered group in the respective rule.

Are there any known workarounds?
Comment 6 Andreas Heiduk CLA 2014-12-15 10:56:39 EST 
Xtext 2.7.3 still has this annoying behaviour. *sigh*
Comment 7 Moritz Eysholdt CLA 2014-12-15 11:48:06 EST 
It's actually fixed with the new formatting infrastructure (see bug 436454). 
The new formatting infrastructure is already part of Xtext's nightly builds and will be released as internal API with Xtext 2.8.0. It will become public API later.

I'm leaving this bug open to make sure we'll implement a test case for this scenario.
Comment 8 Sebastian Zarnekow CLA 2014-12-15 11:49:58 EST 
Scheduled for v2.8 - see comment #7
Comment 9 Moritz Eysholdt CLA 2016-07-21 04:26:58 EDT 
won't fix because it affects the old formatter infrastructure