Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 327080 - Make it easy to use BinaryResource to persist the grammar model
Summary: Make it easy to use BinaryResource to persist the grammar model
Status: CLOSED FIXED
Alias: None
Product: TMF
Classification: Modeling
Component: Xtext (show other bugs)
Version: 1.0.1   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: M7   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 406684 (view as bug list)
Depends on: 406459
Blocks:
  Show dependency tree
 
Reported: 2010-10-06 06:01 EDT by Mark Christiaens CLA
Modified: 2013-04-30 11:33 EDT (History)
5 users (show)

See Also:
sebastian.zarnekow: kepler+


Attachments
XtendGrammarAccess with XMIResource (411.14 KB, image/png)
2013-04-30 10:11 EDT, Moritz Eysholdt CLA
no flags Details
XtendGrammarAccess with BinaryResource (449.26 KB, image/png)
2013-04-30 10:12 EDT, Moritz Eysholdt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Christiaens CLA 2010-10-06 06:01:25 EDT
Build Identifier:  I20100608-0911

I have a terminal rule:

terminal WS:
	(' ' | '\t' | '\f')+; 


When running the workflow to generate the XText editor I get a: 

...
449  [main] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /home/mark.christiaens/workspace_sigasi_vhdl/com.sigasi.xtext/../com.sigasi.xtext.ui/src-gen
574  [main] INFO  ipse.xtext.generator.LanguageConfig  - generating infrastructure for com.sigasi.xtext.VHDLLang with fragments : ImplicitRuntimeFragment, ImplicitUiFragment, GrammarAccessFragment, EcoreGeneratorFragment, ParseTreeConstructorFragment, ResourceFactoryFragment, XtextAntlrGeneratorFragment, JavaValidatorFragment, ImportNamespacesScopingFragment, QualifiedNamesFragment, BuilderIntegrationFragment, FormatterFragment, LabelProviderFragment, TransformerFragment, OutlineNodeAdapterFactoryFragment, QuickOutlineFragment, QuickfixProviderFragment, JavaBasedContentAssistFragment, XtextAntlrUiGeneratorFragment
2494 [main] ERROR enerator.CompositeGeneratorFragment  - An invalid XML character (Unicode: 0xc) was found in the element content:
java.lang.RuntimeException: An invalid XML character (Unicode: 0xc) was found in the element content:
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl$Escape.convert(XMLSaveImpl.java:3298)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.getDatatypeValue(XMLSaveImpl.java:3082)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.saveDataTypeSingle(XMLSaveImpl.java:1678)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.saveFeatures(XMLSaveImpl.java:1265)
	at 
...
org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:52)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:74)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:35)
4348 [main] INFO  or.validation.JavaValidatorFragment  - executing generate for org.eclipse.xtext.generator.validation.JavaValidatorFragment
5201 [main] INFO  .emf.mwe2.runtime.workflow.Workflow  - Done.


The workflow does seem to execute to the end succesfully. 
The cause is the form feed character '\f' in the rule. Any suggestions how to avoid this problem?

Reproducible: Always
Comment 1 Sven Efftinge CLA 2010-10-06 07:02:12 EDT
Workaround: 
Use XML 1.1 by setting the xmlVersion like this:

fragment = grammarAccess.GrammarAccessFragment {
	xmlVersion = "1.1"
}
Comment 2 Sven Efftinge CLA 2010-10-06 07:05:24 EDT
We should provide a switch in grammarAccess.GrammarAccessFragment to use BinaryResource instead of what is registered with *.xmi file extension. Also a corresponding GrammarProvider should be available in that case.
Comment 3 Mark Christiaens CLA 2010-10-12 09:33:09 EDT
FYI, 

When I enable XML 1.1 I also ran into the issue described here: http://www.eclipse.org/forums/index.php?t=tree&th=173813&S=35669656837b43154c1790271f5a797a

I've reverted back to not supporting form-feeds in my lexer for the time being and using XML 1.0.
Comment 4 Mark Christiaens CLA 2011-01-11 10:28:12 EST
I just tried enabling the form feed character on Xtext 2.0 M4 and got

...
8448 ERROR CompositeGeneratorFragment - An invalid XML character (Unicode: 0xc) was found in the element content:
java.lang.RuntimeException: An invalid XML character (Unicode: 0xc) was found in the element content:
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl$Escape.convert(XMLSaveImpl.java:3298)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.getDatatypeValue(XMLSaveImpl.java:3082)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.saveDataTypeSingle(XMLSaveImpl.java:1678)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.saveFeatures(XMLSaveImpl.java:1265)
	at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.saveElementID(XMLSaveImpl.java:2685)
...

So I guess this issue is still present.
Comment 5 Sebastian Zarnekow CLA 2013-04-19 05:53:32 EDT
Ed is working on an optimized persistence format for the index structure.
Comment 6 Sebastian Zarnekow CLA 2013-04-29 08:40:44 EDT
(In reply to comment #5)
> Ed is working on an optimized persistence format for the index structure.

This was obviously the wrong ticket for this comment :-)

Anyway, we'll look into binary storage of the grammar model to improve the startup time.
Comment 7 Moritz Eysholdt CLA 2013-04-29 09:47:37 EDT
*** Bug 406684 has been marked as a duplicate of this bug. ***
Comment 8 Moritz Eysholdt CLA 2013-04-29 10:21:52 EDT
we discussed internally to introduce binary resource for the persisted XtextGrammar:

The GrammarProvider should:
- check if there is a binary resource, and if so, load it.
- otherwise load the XMI.

The GrammarAccessFragment should:
- save a XMI, if GrammarAccessFragment.setXmlVersion(String) has been set.
- save a binary resource otherwise.
- log a warning message, if XMI is being saved.


proposed file extension for binary Xtext grammars: *.xtextbin
Comment 9 Moritz Eysholdt CLA 2013-04-30 10:11:34 EDT
Created attachment 230300 [details]
XtendGrammarAccess with XMIResource
Comment 10 Moritz Eysholdt CLA 2013-04-30 10:12:02 EDT
Created attachment 230301 [details]
XtendGrammarAccess with BinaryResource
Comment 11 Moritz Eysholdt CLA 2013-04-30 10:15:48 EDT
I pushed the patch to Gerrit: 
https://git.eclipse.org/r/#/c/12385/

Loading time from XtendGrammrAccess decreases from 171 ms to 52 ms. However, that's just a small drop in the ocean compared to the total injector initialization time.
Comment 12 Moritz Eysholdt CLA 2013-04-30 11:33:00 EDT
fixed in 'master'.

please note that you'll need to regenerate your language to benefit from this improvement.