Community
Participate
Working Groups
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060200 SUSE/3.0.11-0.1.1 Firefox/3.0.11 Build Identifier: When used to generate even a small number of files, it takes far too long for the templates to produce all the output. For example, generating 35 small HTML files (< 200 lines each) takes approximately 10 seconds. The reason appears to be that the templates are found on the classpath, parsed and invoked at runtime. This is great for quick prototyping but becomes tedious when anything is done on a larger scale. There may also be some other bottlenecks - see for example some of the code in org.openarchitectureware.xpand2.output.OutputImpl which does some strange manipulation of the StringBuffer (could this be a StringBuilder?) in the case where TextStatement#isDeleteLine() returns true. I note that the M2T team intends to make Xpand an Xtext-based language. Since ANTLR can be used underneath, could something like StringTemplate be leveraged? It seems like the templates and extensions should be compiled down to Java classes instead of being interpreted. Reproducible: Always Steps to Reproduce: N/A
Yes, this sounds slow. Could you provide your example? Any templates are parsed only once, btw.
Hi Andrew, I am sorry for the bad experience you had with Xpand so far. Unfortunately, I cannot confirm the numbers you stated in this bug. Performance issues are often a direct result of inefficient xtend functions or complex defines. Is this the case? Could you please provide a sample project that executes that slowly on your site? I will look into this.
I'm afraid I can't provide my examples. However, a colleague wrote some straightforward templates, containing a minimal number of Xtend functions, to generate C++ files. Generation of 34 files (source and headers) took about 9 seconds on his machine, which is the latest MacBook Pro. These are not atypical numbers by any means - I routinely see such characteristics. Sven, what do you mean by the templates only being parsed once? Once per workflow execution? I am almost certain this is the case, which is likely to play a big role in these performance issues. While developing an RCP application which calls some templates, I can run the application in an Eclipse runtime workspace and make changes to the templates in the "base" workspace which are immediately reflected in the output when I next run the workflow. This suggests that there is no caching of the template contents or underlying AST. Compare to the "hot code replacement failed" error message when you make structural changes to the "signature" of a Java class (not all modifications to the AST can be reflected in the other workspace). Perhaps you have some sample templates we can look at to determine some of the bottlenecks? It is difficult to compare Xpand's performance to that of Velocity or StringTemplate, for example, as it would be an apples vs pears comparison given the different feature sets (e.g., the dispatch semantics and the fact that Xpand is domain specific). However, I would guess that the performance problems stem from the fact that Xpand templates are interpreted and not compiled like most other modern template languages.
Do you execute the code generator in an OSGi environment? Maybe this one is interesting for you: https://bugs.eclipse.org/bugs/show_bug.cgi?id=278753
(In reply to comment #4) > Do you execute the code generator in an OSGi environment? > Maybe this one is interesting for you: > https://bugs.eclipse.org/bugs/show_bug.cgi?id=278753 Yes I do, Sebastian. However, in the templates I only use one Java helper - much of the complexity resides in Xtend functions. Therefore the fix adopted for the bug you referenced will only make a minor difference in my case. I still think there are other more fundamental factors at play here.
Please try to use the CachingResourceLoaderImpl as it will greatly improve the performance. The java helper thing was misleasing and not the actual reason for the performance degration. Do you use the JavaBeansMetamodel or the EMFMetamodel?
(In reply to comment #6) > Please try to use the CachingResourceLoaderImpl as it will greatly improve the > performance. > The java helper thing was misleasing and not the actual reason for the > performance degration. > > Do you use the JavaBeansMetamodel or the EMFMetamodel? OK, I will try using that and let you know. I use the EMFMetamodel. Thanks.
Created attachment 146768 [details] place model with 58 entities into my.generator.project/src
Andrew, please verify which part of your workflow takes that much time. You can do so by looking at the output of your workflow runner. 0 INFO WorkflowRunner - -------------------------------------------------------------------------------------- 3 INFO WorkflowRunner - EMF Modeling Workflow Engine 0.8.0, Build v200909071914 3 INFO WorkflowRunner - (c) 2005-2009 openarchitectureware.org and contributors 3 INFO WorkflowRunner - -------------------------------------------------------------------------------------- 4 INFO WorkflowRunner - running workflow: /Users/behrens/Documents/projects/xtext/workspaces/runtime-xpandbug/my.generator.project/src/workflow/generator.mwe 4 INFO WorkflowRunner - 381 INFO StandaloneSetup - Registering platform uri '/Users/behrens/Documents/projects/xtext/workspaces/runtime-xpandbug' 458 INFO CompositeComponent - Reader: Loading model from platform:/resource/my.generator.project/src/Model.xmi 609 INFO CompositeComponent - CheckComponent: slot model check file(s): metamodel::Checks 1118 INFO CompositeComponent - Generator: generating 'template::Template::main FOR model' => src-gen 1966 INFO Generator - Written 58 files to outlet [default](src-gen) 1967 INFO WorkflowRunner - workflow completed in 1510ms! In my case, the model-to-text transormation only took 849ms (=1967ms-1118ms) for 58 entities. You can reproduce this setup by creating the Xpand sample project (New->Other...->Xpand Project) and replacing the model.xmi with the version I have attached.
(In reply to comment #8) > Created an attachment (id=146768) [details] > place model with 58 entities into my.generator.project/src Hi Heiko The workflow in the standard Xpand sample project (New->Other...->Xpand Project) does not run out-of-the-box due to missing dependencies in the manifest. This should be fixed. In the meantime, do you have a working version you could attach? I must say that I don't believe we will be able to do meaningful profiling with such a trivial template. The number and complexity of templates and Xtend functions in a real-world project and the number of meta model elements is far greater than what this project demonstrates. Do you have some non-trivial sample projects which generate, say, a 10,000 LOC application? Even with this example, 849ms to write 58 small Java classes is excessive. The problem here is most certainly not I/O-bound: "The fastest “enterprise” HDDs spin at 10,000 or 15,000 rpm, and can achieve sequential media transfer speeds above 1.6 Gbit/s.[18] and a sustained transfer rate up to 125 MBytes/second.[18]" - from http://en.wikipedia.org/wiki/Hard_disk_drive Obviously it takes a lot longer to write 58 small files than a smaller number of files which occupy the same space. Still, I would guess that these files are no more than a few KB in total, which is tiny compared to the ~100 MB you might expect a modern drive to be able to write in that time. Thanks.
(In reply to comment #6) > Please try to use the CachingResourceLoaderImpl as it will greatly improve the > performance. I tried this (had to make a local copy of it and change the package as I am using the old openArchitectureWare 4.3.1 version, not the MWE version) but it made no measurable difference. I suspect this is because, unlike those who commented on that bug, I am not using the JavaBeansMetamodel.
Andrew, indeed there's a bug with the Wizard I fixed yesterday. You have to manually add the org.apache.commons.logging dependency in the manifest.mf of your sample project. Your performance issues might be caused by different reasons. On of them is IO-related (e.g. indexing content on write, refreshing IDE in write, etc.). Another option can be beautifying, where the generation iteself is fast but the post-processor consumes time before writing. You can check this by deactivating the post-processors. Yet another option is the complexity of your transformation. I have just committed the documentation of the new Xpand/Xtend profiler you can find in the head and will be part of the M2-Release that will be available on 29th. Meanwhile, we can at least measure if the IO causes your problems. Could you please verify the times with sample project? I am afraid that the more complex scenarios I am aware cannot be published either.
(In reply to comment #12) > Andrew, > > indeed there's a bug with the Wizard I fixed yesterday. You have to manually > add the org.apache.commons.logging dependency in the manifest.mf of your sample > project. I had to add a number of oAW dependencies but still get several errors. My MANIFEST.MF is below. Please can you paste yours. Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: my.generator.project Bundle-SymbolicName: my.generator.project; singleton:=true Bundle-Version: 1.0.0 Require-Bundle: org.eclipse.xpand, org.eclipse.emf.ecore, org.eclipse.xtend, org.eclipse.xtend.typesystem.emf Bundle-RequiredExecutionEnvironment: J2SE-1.5 Import-Package: org.eclipse.mwe.emf, org.openarchitectureware.workflow, org.openarchitectureware.type, org.eclipse.emf.ecore.xmi.impl, org.eclipse.mwe.emf
Hey Andrew, the dependencies to org.openarchitectureware.* seem to be erroneous This is my manifest: Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: my.generator.project Bundle-SymbolicName: my.generator.project; singleton:=true Bundle-Version: 1.0.0 Require-Bundle: org.apache.log4j;resolution:=optional, org.eclipse.xtend.profiler;resolution:=optional, org.eclipse.jdt.core;bundle-version="3.5.0", org.apache.commons.logging, com.ibm.icu;bundle-version="4.0.1", org.antlr.runtime;bundle-version="3.0.0", org.eclipse.core.runtime;bundle-version="3.5.0", org.eclipse.emf.mwe.utils;bundle-version="0.7.0", org.eclipse.emf.ecore.xmi;bundle-version="2.5.0", org.eclipse.jface.text;bundle-version="3.5.0", org.eclipse.xpand;bundle-version="0.7.0", org.eclipse.xtend;bundle-version="0.7.0", org.eclipse.xtend.typesystem.emf;bundle-version="0.7.0" Bundle-RequiredExecutionEnvironment: J2SE-1.5 Can you specifiy which error appears when running the workflow?
(In reply to comment #14) > Can you specifiy which error appears when running the workflow? I get the following output after changing my manifest to the one you specified: 0 INFO WorkflowRunner - -------------------------------------------------------------------------------------- 78 INFO WorkflowRunner - EMF Modeling Workflow Engine 0.7.1, Build v200907170432 79 INFO WorkflowRunner - (c) 2005-2009 openarchitectureware.org and contributors 80 INFO WorkflowRunner - -------------------------------------------------------------------------------------- 83 INFO WorkflowRunner - running workflow: /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe 85 INFO WorkflowRunner - 666 ERROR WorkflowRunner - [ERROR]: Class not found: 'org.eclipse.mwe.emf.StandaloneSetup'(Element: bean bean class='org.eclipse.mwe.emf.StandaloneSetup' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:7; Reported by: -UNKNOWN-) 667 ERROR WorkflowRunner - [ERROR]: No getter or adder method for property 'platformUri' in clazz 'java.lang.Object' found.(Element: platformUri='..' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:8; Reported by: -UNKNOWN-) 692 ERROR WorkflowRunner - [ERROR]: Class not found: 'org.eclipse.mwe.emf.Reader'(Element: bean component class='org.eclipse.mwe.emf.Reader' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:12; Reported by: -UNKNOWN-) 693 ERROR WorkflowRunner - [ERROR]: No getter or adder method for property 'uri' in clazz 'org.eclipse.emf.mwe.core.WorkflowComponent' found.(Element: uri='platform:/resource/my.generator.project/src/Model.xmi' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:13; Reported by: -UNKNOWN-) 695 ERROR WorkflowRunner - [ERROR]: No getter or adder method for property 'modelSlot' in clazz 'org.eclipse.emf.mwe.core.WorkflowComponent' found.(Element: modelSlot='model' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:14; Reported by: -UNKNOWN-) 696 ERROR WorkflowRunner - [ERROR]: Class not found: 'org.eclipse.m2t.type.emf.EmfRegistryMetaModel'(Element: bean metaModel class='org.eclipse.m2t.type.emf.EmfRegistryMetaModel' id='mm' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:20; Reported by: -UNKNOWN-) 698 ERROR WorkflowRunner - [ERROR]: Class not found: 'org.eclipse.xtend.xpand2.Generator'(Element: bean component class='org.eclipse.xtend.xpand2.Generator' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:26; Reported by: -UNKNOWN-) 699 ERROR WorkflowRunner - [ERROR]: No setter or adder method for property 'metaModel' in clazz 'org.eclipse.emf.mwe.core.WorkflowComponent' found(Element: <metaModel idRef='mm'/> in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:27; Reported by: -UNKNOWN-) 706 ERROR WorkflowRunner - [ERROR]: No getter or adder method for property 'expand' in clazz 'org.eclipse.emf.mwe.core.WorkflowComponent' found.(Element: expand='template::Template::main FOR model' in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:29; Reported by: -UNKNOWN-) 707 ERROR WorkflowRunner - [ERROR]: No getter or adder method for property 'outlet' in clazz 'org.eclipse.emf.mwe.core.WorkflowComponent' found. Forgot to customize?(Element: bean outlet in /home/Andrew/runtime-eclipse/my.generator.project/src/workflow/generator.mwe:30; Reported by: -UNKNOWN-) 708 ERROR WorkflowRunner - Workflow interrupted because of configuration errors.
(In reply to comment #14) > Hey Andrew, > > the dependencies to org.openarchitectureware.* seem to be erroneous > Heiko, I guess Andrew is still on oAW - at least for MWE. See commment #11
(In reply to comment #16) > I guess Andrew is still on oAW - at least for MWE. See commment #11 To clarify, I am running the latest itemis Eclipse distribution with Xtext 0.7.2. I have also "overlaid" the installation with oAW 4.3.1 plugins and features, with the idea that we'll eventually migrate fully to MWE. Therefore I should be able to use MWE features... unless there is some configuration I missed.
For now, Adrew, can you stick to MWE and run the Xpand sample with the blown xmi I provided above? Please be sure to use the right wizard. The manifest of this project should look as shown in comment 14, sample workflow looks like this on my side <?xml version="1.0"?> <workflow> <property name="model" value="my.generator.project/src/Model.xmi" /> <property name="src-gen" value="src-gen" /> <!-- set up EMF for standalone execution --> <bean class="org.eclipse.emf.mwe.utils.StandaloneSetup" > <platformUri value=".."/> </bean> <!-- instantiate metamodel --> <bean id="mm_emf" class="org.eclipse.xtend.typesystem.emf.EmfRegistryMetaModel"/> <!-- load model and store it in slot 'model' --> <component class="org.eclipse.emf.mwe.utils.Reader"> <uri value="platform:/resource/${model}" /> <modelSlot value="model" /> </component> <!-- check model --> <component class="org.eclipse.xtend.check.CheckComponent"> <metaModel idRef="mm_emf"/> <checkFile value="metamodel::Checks" /> <emfAllChildrenSlot value="model" /> </component> <!-- generate code --> <component class="org.eclipse.xpand2.Generator"> <metaModel idRef="mm_emf"/> <expand value="template::Template::main FOR model" /> <outlet path="${src-gen}" > <postprocessor class="org.eclipse.xpand2.output.JavaBeautifier" /> </outlet> </component> </workflow>
Hello Heiko Apologies for the late response. I imagine with the new incremental generation support in MWE, the overall generation speed should improve dramatically across repeated runs of the same generator: http://blog.efftinge.de/2009/10/helios-m2-of-xtext-xpand-and-mwe.html My question is, will this work with programmatically-defined workflows? What exactly is used to find the changes to model instances across generation runs? Eventually when I have some time I'd like to use the new profiler to find the bottlenecks in my existing project. However, I'd like to keep this issue open and potentially assemble some more realistic test cases which could be used to investigate performance improvements. I believe fast generation is absolutely critical for larger projects. Thanks Andrew (In reply to comment #18) > For now, Adrew, can you stick to MWE and run the Xpand sample with the blown > xmi I provided above? Please be sure to use the right wizard. The manifest of > this project should look as shown in comment 14, sample workflow looks like > this on my side > > <?xml version="1.0"?> > <workflow> > <property name="model" value="my.generator.project/src/Model.xmi" /> > <property name="src-gen" value="src-gen" /> > > <!-- set up EMF for standalone execution --> > <bean class="org.eclipse.emf.mwe.utils.StandaloneSetup" > > <platformUri value=".."/> > </bean> > > <!-- instantiate metamodel --> > <bean id="mm_emf" > class="org.eclipse.xtend.typesystem.emf.EmfRegistryMetaModel"/> > > <!-- load model and store it in slot 'model' --> > <component class="org.eclipse.emf.mwe.utils.Reader"> > <uri value="platform:/resource/${model}" /> > <modelSlot value="model" /> > </component> > > <!-- check model --> > <component class="org.eclipse.xtend.check.CheckComponent"> > <metaModel idRef="mm_emf"/> > <checkFile value="metamodel::Checks" /> > <emfAllChildrenSlot value="model" /> > </component> > > <!-- generate code --> > <component class="org.eclipse.xpand2.Generator"> > <metaModel idRef="mm_emf"/> > <expand > value="template::Template::main FOR model" /> > <outlet path="${src-gen}" > > <postprocessor class="org.eclipse.xpand2.output.JavaBeautifier" /> > </outlet> > </component> > </workflow>
There is a known severe performance bug in 0.8.0-M3 (Bug#297053). Since it is mentioned here that 0.7.2 is used this bug might not be correlated.
Closing this one due to inactivity and lack of reproducabilty. Feel free to reopen if there is a concrete, reproducable issue with Xpand's performance.
Requested via bug 522520. -M.
