Bug 340034

Description Ralph Mayr 2011-03-15 10:43:23 EDT

Build Identifier: M20100909-0800

When a terminal is defined to start with a non-alphanumeric character (e.g. '<' or '1'), whitespace following the terminal are ignored. When terminals start with alphanumeric characters, at least one following whitespace is mandatory.
This behaviour sounds inconsistent to me, as the start character of a token should not directly influence the handling of token/whitespace after the token.

Reproducible: Always

Steps to Reproduce:
1. Start with a new Xtext project, define a simple grammar as follows:

Model:
  (elements += ModelElement ";")* ;
ModelElement: NormalElement | SpecialCharacterElement | SpecialCharacterElement2;
NormalElement: "normal" "name" "=" name=STRING;
SpecialCharacterElement: "<special" "name" "=" name=STRING;
SpecialCharacterElement2: "2special" "name" "=" name=STRING;

2. Generate the editor

3. The rule NormalElement can only be parsed if the "normal"-terminal is followed by at least one whitespace (this seems correct to me):
  normal name = "normal"; // No parse error
  normalname = "normal"; // Parse error
  
4. The rules SpecialCharacterElement and SpecialCharacterElement2 can be parsed regardless of whitespace following the "<special" or "2special" terminals:

  <special name="special"; // No parse error
  <specialname="special"; // No parse error
  2special name="2special"; // No parse error
  2specialname="2special"; // No parse error

In my opinion, the text "<specialname="special";" and "2specialname="2special";" should give a parse error in the same way "normalname = "normal";" does.