Community
Participate
Working Groups
Formatting where a linewrap is wanted between "statements" in the attached example has no effect. The attached grammar and associated classes mimics the real grammar and reveals the problem, although it seems like the problem occurs also without the special handling (see below). Instructions to reproduce: Unzip the attachment (contains src and Manifest.mf) Run the JUnit MyDslTest One of the tests fail. Note about the special processing. - A ValueSerializer is needed to enable serialization of the unassigned OWS, and terminals for operators - The formatter introduces a special token stream. This stream handles the "transformation" of the semantic OWS into hidden OWS. If this is not done, the formatter will add spaces also for OWS. - A HiddenTokenHelper is used to define that OWS is the whitespace rule to use. If not defined, the formatter will not know where to insert spaces. (And since OWS is not a terminal it can not simply be returned from getWSRule as it requires a terminal to be returned). - backracking is turned on to surpress output about "multiple alternatives" The MyDSLTest shows that some rules work as they should, but not the linewrap between statements in: Model returns Manifest hidden(): {Manifest} (statements += Expression)* ; It is probably possible to further reduce the grammar, but I wanted it to include the principle I am using to deal with a complex grammar that can not use terminals for whitespace. (The real grammar also handles single and multiple line comments and strings the same way).
Created attachment 186692 [details] src including JUnit test
Please note that hidden() effectively tells the parser (and the formatter!) that there are no hidden tokens allowed within this parser rule. This suppression of hidden tokens applies recursively within all parser rules that are being called. Consequently, the formatter doesn't insert line-wraps, white-space, etc.
(In reply to comment #2) > Please note that hidden() effectively tells the parser (and the formatter!) > that there are no hidden tokens allowed within this parser rule. This > suppression of hidden tokens applies recursively within all parser rules that > are being called. > > Consequently, the formatter doesn't insert line-wraps, white-space, etc. afaikt it does not seem to use that as the only mechanism to insert white space and linewraps. I *do* get insertion of both for other formatting rules in the attached sample. The mechanism that seems to tell the formatter that it is ok to insert formatting is getWhitespaceRuleFor in the HiddenTokenHelper - a modified helper is included in the sample project to deal with this. The problem has to do (I think) with matching - since using a different rule like 'range' on delimiters as in '{' statemets += Expression '}' I can get the formatter to insert newlines after each token - which makes me think that the detection of where to insert the whitespace is probably working.
Same problem on Xtext version 2.0
Found that this has nothing to do with my complex handling of OWS. To see the error easily. Generate a standard "MyDsl" and change it to: grammar org.xtext.example.mydsl.MyDsl2 with org.eclipse.xtext.common.Terminals generate myDsl2 "http://www.xtext.org/example/mydsl/MyDsl2" Model: greetings+=Greeting*; Greeting : SimpleGreeting | GroupGreeting; SimpleGreeting returns Greeting: 'Hello' name=ID '!' ; GroupGreeting returns Greeting: {GroupGreeting} '{' greetings += Greeting* '}' ; Then modify the formatter to contain: c.setLinewrap().after(ga.getModelAccess().getGreetingsAssignment_1()); c.setLinewrap().after(ga.getGroupGreetingAccess().getGreetingsAssignment_2()); for(Keyword k : ga.findKeywords("!")) c.setNoSpace().before(k); i.e. set linewraps between the assignments When run, the assignments to the Model.greetings does not have linewraps, but the GroupGreeting.greetings has. It does not matter if the first rule is changed from 'Greeting*' to 'Greeting+'. Or, if a {Model} is added (it is always created anyway, but just for testing...). But if a keyword is added like this: Model: '=' greetings+=Greeting*; Then, the linewrap rule has effect. Also tested with a nested model object - i.e. Model : greetings = GreetingBlock; GreetingBlock : greetings += Greeting* ; This to see if it is an issue with formatting in the root object. But it has the same problem. (Again, if something like a keyword is added, formatting works.
The same problem exists in Xtext 2.0 (which I used for testing). Also found that if the rule is specified like this: Model: '='? greetings+=Greeting*; Then formatting is applied when the '=' is present in the text, but not otherwise !
Problem can also be observed by selecting text and applying formatting. with the input text: { Hello fred! Hello mary! } If selection includes the braces, the formatting inserts linewraps. But not, if braces are not selected.
Thanks for your observations, Hendrik. This seems to boil down to the issue when a list-assignment that invokes a parser rule is not matched correctly. *** This bug has been marked as a duplicate of bug 318883 ***
