Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 321047

Summary: Problems with UTF-8 encoded templates
Product: [Modeling] M2T Reporter: Karsten Thoms <karsten.thoms>
Component: XpandAssignee: Darius Jockel <darius.jockel>
Status: CLOSED FIXED QA Contact:
Severity: trivial    
Priority: P3 CC: darius.jockel, jan, m2t.xpand-inbox, sebastian.zarnekow
Version: 1.0.0Flags: sebastian.zarnekow: helios+
Target Milestone: SR1   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
Stacktrace
none
documentation none

Description Karsten Thoms CLA 2010-07-27 12:42:29 EDT
Created attachment 175338 [details]
Stacktrace

The generator fails reading UTF-8 encoded templates, although the Generator component has explicitly been configured with UTF-8 fileEncoding value. Only if the templates are ISO-8859-1 encoded the generator works.

Exception raised:
349  [main] ERROR mf.mwe2.launch.runtime.Mwe2Launcher  - Problems running workflow workflow.EntityDSLGenerator: no viable alternative at input 'Â' on line 1
java.lang.RuntimeException: Problems running workflow workflow.EntityDSLGenerator: no viable alternative at input 'Â' on line 1
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:82)
Comment 1 Karsten Thoms CLA 2010-07-27 12:48:10 EDT
Project set that reproduces the bug:
svn checkout http://svn.codespot.com/a/eclipselabs.org/lwc11-xtext/branches/bug321047/ lwc11-xtext-read-only

The workflow sets encoding to UTF-8
--------------
EntityDSLGenerator.mwe2
var fileEncoding = "UTF-8"
...
	component = org.eclipse.xpand2.Generator {
		expand = "templates::Main::Start FOREACH entityModel"
		outlet = {
			path = targetDir
			postprocessor = org.eclipse.xpand2.output.JavaBeautifier {}
		}
		fileEncoding = fileEncoding
	}
--------------


The templates are UTF-8 encoded.
Comment 2 Sebastian Zarnekow CLA 2010-07-27 14:41:24 EDT
As far as I understand the docs, fileEncoding is the encoding of the output file.

Did you try to configure the fileEncoding on the resource manager?
Comment 3 Jan Koehnlein CLA 2010-08-02 05:56:40 EDT
Stumbled across the same issue in a customer's project. Maybe we should add an additional property for template/extension encoding to the component and pass it on to the ResourceManager. Also holds for the XtendComponent.
Comment 4 Karsten Thoms CLA 2010-08-02 10:31:20 EDT
Yes, at the moment it is not configurable through ordinary workflow. I came upon this, since this behaved definetely otherwise before. It was not good that the same encoding and property was used for output encoding and for loading resources. I did not check yet if it behaves correct, but the output encoding must be configurable on level of an Outlet. 
Anyway, this bug entry is to enable (and document) configuration of encoding of resources.
Comment 5 Sebastian Zarnekow CLA 2010-08-02 10:34:13 EDT
Why does:

component = org.eclipse.xpand2.Generator {
	metaModel = org.eclipse.xtend.typesystem.emf.EmfRegistryMetaModel {}
	expand = "templates::Template::main FOREACH model"
	outlet = {
		fileEncoding = "someEncoding"
		path = targetDir
	}
	resourceManager = org.eclipse.xtend.expression.ResourceManagerDefaultImpl {
		fileEncoding = "someOtherEncoding"
	}
	fileEncoding = fileEncoding
}

not work for you? What whould be "enabled"?
Comment 6 Karsten Thoms CLA 2010-08-03 00:35:47 EDT
This works, thanks for the hint. However, this behavior was different before (workspace encoding was used by default) and it has to be documented how a different template encoding has to be configured. 

Leaving this open as documentation bug.
Comment 7 Sebastian Zarnekow CLA 2010-08-11 11:56:42 EDT
MWE has the very same behavior as MWE2.
The Xpand documentation should clarify how the various encoding options work.
Comment 8 Darius Jockel CLA 2010-08-12 09:09:25 EDT
Created attachment 176462 [details]
documentation

Updated documentation
Comment 9 Darius Jockel CLA 2010-08-12 09:10:01 EDT
Documentation updated