Community
Participate
Working Groups
First, the feature request: --------------------------- Consider a simplified language's grammer rule: Foo: ('public'|'private') name=ID; I propose that the generated editor should be equipped with validation rules that check the "name" feature for invalid keywords that can be derived from the grammar, e.g. "public" and "private" in the above example. Second, this might be a bug due to which I'd like to have the above feature: ---------------------------------------------------------------------------- Currently, if one fails to add these error checks for id rule validations manually, very odd things can happen when serializing a model that was build in memory first. For instance, using Ecore API to create a "Foo" element and set its name to "internal.public" will actually serialize the model but ommit "public" when writing the name of "Foo". Cheers Rob
Not sure I got it, but the error will provide a corresponding annotation if you write """ private public """ given your grammar rule. Is this not the case?
(In reply to comment #1) > Not sure I got it, but the error will provide a corresponding annotation if you I meant .... the editor will provide.... of course. That is the parser will create a syntax error.
Hi Robert, the expected behavior of Xtext during serialization is, that if an identifier collides with a keyword, the is escaped. For example, Foo { name="public" } should be serialized to "public ^public". Xtext's value converter are responsible for escaping. Please note that "internal.public" is not a valid ID since it contains a ".".
(In reply to comment #1) > given your grammar rule. Is this not the case? Yes, indeed that is the case for rules as the one in my original bug description. Sorry for that, but see below in this reply, for a rule that won't work. (In reply to comment #3) > the expected behavior of Xtext during serialization is, that if an identifier > collides with a keyword, the is escaped. Yes, I just realized that. However, the escape mechanism fails on these rules in my grammar: Package: 'package' name=FQN; FQN returns ecore::EString: ID ('.' ID)*; That rule is intended to collect a Java like package name, e.g. "foo.bar.buz". But it will fail, if someone creates a package "foo.bar.documentation" (using Ecore API) if "documentation" is a keyword. When using an Xtext editor I saw that I do get an error marker. But when creating that model using ecore API, there's nothing wrong with a "foo.bar.documentation" package name until the model is serialized. Then the file will contain "foo.bar." as package name only and I got no warning that a segment was dropped. That's where I'd like to see support on Xtext side to detect such ID-keywords to be escaped automatically. As Xtext knows the grammar rule behind "name=FQN" I believe it should escape any ID occuring in the serialized model file.
> Package: > 'package' name=FQN; > FQN returns ecore::EString: > ID ('.' ID)*; > > That rule is intended to collect a Java like package name, e.g. "foo.bar.buz". > But it will fail, if someone creates a package "foo.bar.documentation" (using > Ecore API) if "documentation" is a keyword. When using an Xtext editor I saw > that I do get an error marker. But when creating that model using ecore API, > there's nothing wrong with a "foo.bar.documentation" package name until the > model is serialized. Then the file will contain "foo.bar." as package name only > and I got no warning that a segment was dropped. That's where I'd like to see > support on Xtext side to detect such ID-keywords to be escaped automatically. > As Xtext knows the grammar rule behind "name=FQN" I believe it should escape > any ID occuring in the serialized model file. It doesn't do this automatically. Escaping the names is a responsibility of your VaueConverter for FQN. I'm not sure if there is a way for Xtext to solve this generically since data type rules can have any kind of structure. Furthermore, if there would be a generic solution it's likely to be slow and since this functionality is used very often during serialization I don't think this is a place to be sloppy with performance.
(In reply to comment #5) > It doesn't do this automatically. Escaping the names is a responsibility of > your VaueConverter for FQN. I think you're wrong on this one because it does escape automatically for simple ID rules such as: Class: 'class' name=ID; If such a class is created using Ecore API and given a name "documentation" then serialization will automatically escape it to "^documentation". I did not provide a value converter for that feature, so it's got to be part of Xtext already. And my FQN rule only makes use of the ID rule, so I'd expect it to "inherit" that feature. > > I'm not sure if there is a way for Xtext to solve this generically since data > type rules can have any kind of structure. Furthermore, if there would be a > generic solution it's likely to be slow and since this functionality is used > very often during serialization I don't think this is a place to be sloppy with > performance. Well, you do it already for simple ID rules. Using a Map can give an O(1) complexity. Hence, I don't think this would be such a big performance issue.
(In reply to comment #6) > (In reply to comment #5) > > It doesn't do this automatically. Escaping the names is a responsibility of > > your VaueConverter for FQN. > I think you're wrong on this one because it does escape automatically for > simple ID rules such as: > Class: 'class' name=ID; > If such a class is created using Ecore API and given a name "documentation" > then serialization will automatically escape it to "^documentation". I did not > provide a value converter for that feature, so it's got to be part of Xtext > already. And my FQN rule only makes use of the ID rule, so I'd expect it to > "inherit" that feature. True, rule ID ships with Xtext: /org.eclipse.xtext/src/org/eclipse/xtext/common/Terminals.xtext This is why Xtext has a predefined value converter for it: org.eclipse.xtext.conversion.impl.IDValueConverter It's not trivial for the rule FQN to inherit the behavior of rule ID, as we are discussing it below. > > I'm not sure if there is a way for Xtext to solve this generically since data > > type rules can have any kind of structure. Furthermore, if there would be a > > generic solution it's likely to be slow and since this functionality is used > > very often during serialization I don't think this is a place to be sloppy with > > performance. > Well, you do it already for simple ID rules. Using a Map can give an O(1) > complexity. Hence, I don't think this would be such a big performance issue. Sure, but first you have to split up a FQN into the IDs it consists of. If you do this in your value converter you can to it very fast by fqnName.split("\."). How would you want to do this generically? Think of Model: value=STRING unit=('Hex'|'Oct'|'Dec') 'in' date=Date; Date returns ecore::EDate: INT ('st'|'rd'|'th') ID INT; Examples: "32434" Oct in 3rd ^Oct 2010 "1234" Dec in 10th ^Dec 1853 "BAD" Hex in 29th Apr 1883 Only the value converter knows how to convert the values from the semantic model into the textual representation in the document. In this example, it's converting values of ecore::EDate to "3rd ^Oct 2010" or "10th ^Dec 1853". Not even the parser can help with this. One can think of making an exception for data type rules which return ecore::EString (this is true for your FQN rule). However, i don't think that the general assumption that the string value in the semantic model (not in the document!) can always be parsed its data type rule holds true. p.s. A lookup in an array has constant complexity, map implementations have logarithmic complexity for the most of the time.
(In reply to comment #7) > Sure, but first you have to split up a FQN into the IDs it consists of. If you > do this in your value converter you can to it very fast by fqnName.split("\."). Very true, though. Would it be possible to provide a Set of keywords of a grammar to support language developers in writing such value converters? Xtext should easily be able to extract all keywords from the grammar when it generates the editor. Maybe there is already an API call that I'm not aware of to get a list or set of all keywords of a language?
(In reply to comment #8) > (In reply to comment #7) > > Sure, but first you have to split up a FQN into the IDs it consists of. If you > > do this in your value converter you can to it very fast by fqnName.split("\."). > Very true, though. Would it be possible to provide a Set of keywords of a > grammar to support language developers in writing such value converters? Xtext > should easily be able to extract all keywords from the grammar when it > generates the editor. Maybe there is already an API call that I'm not aware of > to get a list or set of all keywords of a language? yes, there is: GrammarUtil.getAllKeywords(myGrammar) Some time ago I proposed (bug 306031) to include the FQN rule into common terminals.xtext and ship a value converter with it. This seems reasonable to me, since cross referencing is based on FQNs for the very most languages. I don't think there is anything better Xtext can do in this scenario... so closing this bug.
