Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 354101

Summary: Too many constants despite fieldsPerClass = "200"
Product: [Modeling] TMF Reporter: Peter Feiler <phf>
Component: XtextAssignee: Project Inbox <tmf.xtext-inbox>
Status: CLOSED FIXED QA Contact:
Severity: major    
Priority: P3 CC: phf, sebastian.zarnekow
Version: unspecifiedFlags: sebastian.zarnekow: indigo+
Target Milestone: SR2   
Hardware: PC   
OS: Windows XP   
See Also: https://bugs.eclipse.org/bugs/show_bug.cgi?id=349992
Whiteboard:
Attachments:
Description Flags
zip file with four projects to demo the bug
none
revised AADL grammar none

Description Peter Feiler CLA 2011-08-08 00:45:16 EDT
Build Identifier: Build id: 20110615-0604  Xtext 2.0.1.v201108041112

I have a large grammar for the SAE AADL against an already existing Ecore model. The grammar requires backtrack =true. I have set classSplitting=true and fieldsPerClass="200" (I first tried "500" per instructions from Bug report 349992). Running the generator produces the Too many constants message upon compilation.

I will upload an attachment that contains four projects, one with the Ecore model and generated code, and the three Xtext projects. The first contains the grammar and mwe2 file.

Reproducible: Always

Steps to Reproduce:
1. Unzip the attachment into an empty workspace folder and import as existing project in Eclipse. 
2. Generate the editor via mwe2.
3. observe the error message "Too many constants, the constant pool for InternalAadl2Parser would exceed 65536 entries"
Comment 1 Peter Feiler CLA 2011-08-08 01:03:05 EDT
Created attachment 201053 [details]
zip file with four projects to demo the bug

The zip file contains four projects.
* a project with the Ecore model 
* a project containing the xtext grammar and mwe2 file
* the two other Xtext generated projects with empty structures.
Unzip & import into a workspace folder.
Then generate the Xtext editor.
Comment 2 Peter Feiler CLA 2011-08-08 01:19:49 EDT
You may need to increase your heap space setting when running mwe2.
I am running with a setting of -Xmx1300m.
Comment 3 Sebastian Zarnekow CLA 2011-08-12 10:10:03 EDT
Your grammar contains some surprising, repeated patterns, e.g. you seem to use syntactic predicates for basically any decision point and enable backtracking too. That's not necessary. I bet you can safely remove every explicit predicate without modifying the behavior of the generated parser. Furthermore you seem to use a duplicated expression tree at the very same place in the language which does not look right to me. Please revisit your boolean logic there. Eliminating the redundancy will reduce the file size too. Last but not least I'd recommend to explore the DebugAntlrGeneratorFragment in combination with AntlrWorks to tackle the ambiguities. It would help if you could disable backtracking.
Comment 4 Peter Feiler CLA 2011-08-14 18:30:23 EDT
Hi Sebastian,

at first I tried to avoid using back tracking, thus, the predicates  and I had not gotten around to removing them after turning on backtracking.

I have removed them all.

The boolean expressions syntax has an ambiguity in the the use of parentheses. They orignally addressed it by not allowing parentheses at the outmost level of boolean expressions - thus the strange rules. They did process models correctly under back tracking. 
I am working with the AADL standard committee to disambiguate those rules differently.
I took the liberty to make chagnes that allow me to turn off back tracking.

As you noticed some rules are quite repetitive, e.g., for component types and component implementations. Different specific component types use different subsets of elements, which are enforced by the Ecore model. 
I have a variant of the rules where the common backend for the different rules is defined as part of the component type rule - I have this variant included in the newly attached grammar.
However, those rules resulted in another problem, which I reported under bug 353969. 

I have attached the revised grammar. As stated above it does not use backtracking and a simplified boolean grammar, as well as less repetition for component types by pulling the common backend up into a parent rule.

I still get the "too many constants" issue. 

We do have a hand-crafted ANTLR grammar which the person used syteactic and semantic predicates to resolve the issue. Unfortunately I am working with a langauge standard that requries committee approval for changes.
Comment 5 Peter Feiler CLA 2011-08-14 18:32:05 EDT
Created attachment 201481 [details]
revised AADL grammar

I revised the grammar as per suggestions.
Comment 6 Peter Feiler CLA 2011-08-15 10:40:47 EDT
I have a variant of the grammar where the property expression/association portion is in a separate grammar that is mixed in to the core language. It will also be reused in language extensions to the core language, such as fault modeling associated with core architecture models.

Splitting of the grammar did not help with respect to the "too many constants" issue.
Comment 7 Sebastian Zarnekow CLA 2011-10-11 19:22:06 EDT
Pushed to master.

Introduced a new option methodsPerClass in AntlrOptions which allows to force a super-type extraction. Default is 5000 methods per class. The AADL ui parser (generated from the 'revised AADL grammar') has approx 7800 methods.

The only necessary option to generate parsers for AADL is now:

options = {
	classSplitting = true
}
Comment 8 Karsten Thoms CLA 2017-09-19 17:10:31 EDT
Closing all bugs that were set to RESOLVED before Neon.0
Comment 9 Karsten Thoms CLA 2017-09-19 17:22:10 EDT
Closing all bugs that were set to RESOLVED before Neon.0