Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 335764 - EMTL file resolves URI unresolvably
Summary: EMTL file resolves URI unresolvably
Status: CLOSED WONTFIX
Alias: None
Product: Acceleo
Classification: Modeling
Component: Core (show other bugs)
Version: 3.0.0   Edit
Hardware: PC Windows Vista
: P3 major
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-29 08:20 EST by Ed Willink CLA
Modified: 2016-03-31 05:02 EDT (History)
4 users (show)

See Also:


Attachments
Workaround (2.65 KB, application/octet-stream)
2011-02-18 05:16 EST, Ed Willink CLA
no flags Details
test projects (127.72 KB, application/x-zip-compressed)
2011-02-25 06:51 EST, Nicolas Bros CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2011-01-29 08:20:32 EST
In Acceleo 3.0

[module generateOclCommon('http://www.eclipse.org/ocl/3.1.0/Pivot','http://www.eclipse.org/emf/2002/Ecore')/]
 ...

generated EMTL such as

        <eType xsi:type="ecore:EClass" href="http://www.eclipse.org/ocl/3.1.0/Pivot#//NamedElement"/>

In 3.1.0M4a it has changed to

        <eType xsi:type="ecore:EClass" href="/org.eclipse.ocl.examples.pivot/model/pivot.ecore#//NamedElement"/>

and so results in proxy resolution failures when the EMTL file is run.
Comment 1 Ed Willink CLA 2011-01-29 11:55:26 EST
On further investigation, this may be nothing to do with M4b, but a longstanding unresolved releng problem for me. I had not appreciated that the migration of emtl files from src to bin had enabled me to put bin files in CVS leaking good files to confuse me.

I think the good files arise when I allow the builder to run in a run-time workspace so that all models are installed as plugins.

The bad files arise if the emtl files are missing.

Usually I have got away with it, because with good emtl files in bin, the Acceleo builder failures in a non-runtime workspace could be ignored.

So the problem is, why does the EMTL file have workspace URIs that the executor cannot resolve after the builder runs in a non-runtime workspace without meta-models installed as plugins?
Comment 2 Laurent Goubet CLA 2011-02-07 09:13:27 EST
Ed,

The Acceleo compiler is not in charge of determining what URIs to use ... and that causes enough trouble for us to evaluate the cost of creating our own URI scheme. This is the same problem as simply editing an ecore model in the sample ecore editor which sometimes saves URIs toward the metamodel as http://.../2002/Ecore, and sometimes as platform:/plugin.../Ecore.ecore; sometimes even going back and forth from one to the other as you save the file, close the editor, reopen the editor, change one the elements' property, resave the file.

I've more or less given up on trying to understand why the URIs are saved one way or the other (let's not talk about URIs pointing to a model that is not under the workspace root (symlink)); we (Acceleo) simply resorted to using an URI converter that tries its best to find the bundle back whatever the URI might be, or wherever we might be (resolving platform:/plugin URIs in standalone or absolute URIs in a jarred environment is quite the challenge).

About this particular bug ... did you find a workaround or another cause for the issue? Or could you elaborate on the actual problem you're facing?
Comment 3 Ed Willink CLA 2011-02-08 01:04:19 EST
I think the root problem was actually Bug 336119, perhaps aggravated by 336109.

The real issue is that unresolved proxies are silently diagnosed by EMF at load time and not overtly diagnosed by Acceleo allowing downstream activities to fall over strangely.

I have discovered that after loading a resource, it is not sufficient to just check that no silent problems await exposure from Resource.eErrors, but also from all referenced resources. In practice, in a controlled context, it is a good idea to scan the entire ResourceSet for any resource with any error and let the user know.
Comment 4 Laurent Goubet CLA 2011-02-08 02:47:38 EST
Lazy loading is at the core of EMF, it has its pros (memory usage, performance...) and its cons... Among which is the fact that unresolved proxies are more often than not detected too late in the execution flow.

True, walking the entire resource set contents an resolving every single proxy right off the bat would allow us to detect "unresolved proxy" errors before doing anything else... But we've often observed that "breaking" the lazy loading that way is too high of a performance issue for something that is useless in 90% of the use cases, as it _does_ walk the entire content tree of the resource set, be there proxies or not, even though it will only be useful if 1) there are proxies, 2) one of them cannot be resolved and 3) one of these unresolved proxies would actually be used by the program.

That is something we've observed in our projects so far, and unless someone manages to prove to me that the performance cost is low or this would be useful in more than half of the use cases, there will never be a "resolveAll" in the core of Acceleo. Users for which this would serve a purpose can always override the "doGenerate" method of the java class we generate to launch Acceleo generation in order to resolve all proxies before resuming the "normal" workflow through "super.doGenerate".

That being said, I close this bug as it seems you did resolve this particular issue and it would serve as a duplicate for other.
Comment 5 Ed Willink CLA 2011-02-08 03:11:58 EST
(In reply to comment #4)
> True, walking the entire resource set contents an resolving every single proxy
> right off the bat would allow us to detect "unresolved proxy" errors before
> doing anything else... But we've often observed that "breaking" the lazy

This isn't what I'm suggesting. 

After a load, and other major activity, it is necessary to look at 

ResourceSet.resources.eErrors

to see what deep silent errors have been discovered.

When accessing potential unresolved proxies, it is a good idea to check for a proxy. The most evident unchecked place to me is at the root of AcceleoServ ice.doGenerate where

final EClassifier argumentType = template.getParameter().get(0).getType();

is unchecked and the unresolved proxy results in a very quick execution and no output for no matches.
Comment 6 Laurent Goubet CLA 2011-02-08 03:27:13 EST
(In reply to comment #5)
> ResourceSet.resources.eErrors
> 
> to see what deep silent errors have been discovered.

That's another matter yes ... but haven't you already raised a bug for this? I recall this was in my todo list already.

> 
> When accessing potential unresolved proxies, it is a good idea to check for a
> proxy. The most evident unchecked place to me is at the root of AcceleoServ
> ice.doGenerate where
> 
> final EClassifier argumentType = template.getParameter().get(0).getType();
> 
> is unchecked and the unresolved proxy results in a very quick execution and no
> output for no matches.

We can't check for proxies everywhere ... but yes, this particular access should be. Could you raise a separate bug for this though? This one doesn't really correspond.
Comment 7 Nicolas Bros CLA 2011-02-18 04:28:53 EST
I'm having the same problem with Acceleo 3.1M5 in Indigo M5. The generator stopped working : it doesn't generate anything anymore. And there isn't any error message.

I have a project that contains a metamodel and an Acceleo generator.
The metamodel is in a file named ModiscoWorkflow.ecore, and it has the URI "http://www.eclipse.org/gmt/modisco/workflow".

With Acceleo 3.0, the compiled emtl files contained this reference:
<takesTypesFrom href="http://www.eclipse.org/gmt/modisco/workflow#/"/>
But with Acceleo 3.1 M5, they now contain this reference:
<takesTypesFrom href="/org.eclipse.gmt.modisco.workflow/model/ModiscoWorkflow.ecore#/"/>


As a result, in AcceleoService#doGenerate (line 376), the following test always returns false, thus preventing the generation from starting:
  if (argumentType.isInstance(model))

I have found this is because the model and the argumentType reside in two different metamodels. In the debugger, I see:

argumentType.eResource() : 
org.eclipse.emf.ecore.xmi.impl.EcoreResourceFactoryImpl$1@1bcfb6b
uri='/org.eclipse.gmt.modisco.workflow/model/ModiscoWorkflow.ecore'

and 

model.eClass().eResource() :
org.eclipse.emf.ecore.xmi.impl.EcoreResourceFactoryImpl$1@234594
uri='http://www.eclipse.org/gmt/modisco/workflow'


We tested with two different configurations: with the plug-in containing our metamodel installed and not installed. And in both cases, we encounter the same problem. This is currently preventing me from continuing to work on this generator.

Do you have a workaround for this problem?
Will it be fixed in a future version of Acceleo?


Eclipse SDK
Version: 3.7.0
Build id: I20110127-2034
Acceleo SDK 3.1.0.v20110128-1020
Comment 8 Ed Willink CLA 2011-02-18 05:16:35 EST
Created attachment 189265 [details]
Workaround

Attached is my workaround for Bug 336109 and 336119 as a result of which Acceleo works well for me in MWE scripts.

Just use MyGeneratePivotVisitors in place of e.g. GeneratePivotVisitors and manually resolve all URIs before Acceleo starts.

The diagnostic loops are intended to bomb helpfully, but didn't seem to.
Comment 9 Nicolas Bros CLA 2011-02-18 10:42:21 EST
(In reply to comment #8)
Thank you for your workaround Edward.
I adapted it for my generator, and added:

uriResourceMap.put(URI.createURI("/org.eclipse.gmt.modisco.workflow/model/ModiscoWorkflow.ecore"), ModiscoworkflowPackage.eINSTANCE.eResource());

Although as a user I don't believe I should have to do this.

Anyway, now it starts generating, but unfortunately the generator now skips all templates that have arguments with types from my metamodel.

Also, each time I modify the main template, Acceleo adds error markers in the editor like:
"illegal operation signature: (generateActionClass(Workflow))"

If I modify generateActionClass.mtl, it recompiles it and the error disappears in the referencing template.

But then it comes back when I modify the main template again.
So, I think the Acceleo compiler is getting confused with these multiple URIs for the same metamodel.
Comment 10 Ed Willink CLA 2011-02-18 10:53:32 EST
(In reply to comment #9)
> uriResourceMap.put(URI.createURI("/org.eclipse.gmt.modisco.workflow/model/ModiscoWorkflow.ecore"),
> ModiscoworkflowPackage.eINSTANCE.eResource());
> 
> Although as a user I don't believe I should have to do this.

+1. Hopefully with both of us wingeing a solution will be found. REOPENing since clearly an issue.

> Anyway, now it starts generating, but unfortunately the generator now skips all
> templates that have arguments with types from my metamodel.
> 
> Also, each time I modify the main template, Acceleo adds error markers in the
> editor like:
> "illegal operation signature: (generateActionClass(Workflow))"
> 
> If I modify generateActionClass.mtl, it recompiles it and the error disappears
> in the referencing template.
> 
> But then it comes back when I modify the main template again.
> So, I think the Acceleo compiler is getting confused with these multiple URIs
> for the same metamodel.

That would be Bug 335761.
Comment 11 Ed Willink CLA 2011-02-18 10:54:02 EST
Reopening as commented above.
Comment 12 Nicolas Bros CLA 2011-02-25 06:51:23 EST
Created attachment 189785 [details]
test projects

I'm attaching test projects demonstrating the problem.

I found that as soon as I have both the metamodel and the generator in the same workspace, Acceleo generates the incorrect emtl files (with a reference to the workspace ecore file).
Comment 13 Ed Willink CLA 2011-03-23 17:26:30 EDT
M6a: It is no longer necessary to create a derived Generate. I find the following to be a sufficient workaround.

Rename registerPackages as registerPackagesGen then add

@Override
public void registerPackages(ResourceSet resourceSet) {
    registerPackagesGen(resourceSet);
    if (!isInWorkspace(org.eclipse.emf.ecore.impl.EPackageImpl.class)) {
        	resourceSet.getPackageRegistry().put("/org.eclipse.ocl.examples.pivot/model/pivot.ecore", org.eclipse.ocl.examples.pivot.PivotPackage.eINSTANCE);
    }
}
Comment 14 Stephane Begaudeau CLA 2011-05-13 04:14:25 EDT
A fix has been contributed on HEAD, it will be available in Acceleo 3.1.0 RC1.

With this new fix, whether the metamodel is in the workspace or in a plugin, the uri that will be saved in XML resources and the uri that should be registered in the Java launcher class with XML resources will be the "logical" URI (http://myLogicalURI) instead of a "physical" uri (myProject/model/myMetamodel.ecore).

For those who want to test this new change, the build 178 on Hudson integrates those new modifications. Of course, it is a nightly build and it is not recommended on a production environment.

Build 178: https://hudson.eclipse.org/hudson/view/Modeling/job/m2t-acceleo-3.1/178/

Several configurations will need to be tested with XML resources and Binary resources in stand alone, in eclipse and when deployed as a plugin. The comments in the generated Java class (in registerPackage(...)) will also need to be changed if the fix works for everyone.
Comment 15 Ed Willink CLA 2011-05-13 05:44:56 EDT
(In reply to comment #14)
> Build 178:
> https://hudson.eclipse.org/hudson/view/Modeling/job/m2t-acceleo-3.1/178/

This build is unuseable. See Bug 345708.
Comment 16 Ed Willink CLA 2011-05-14 05:50:18 EDT
Build N201105140106 is much better. But the problem seems only half resolved.

If I am using binary EMTL, I get two messages and stack traces reporting that the template parameter is a proxy; the message is a big improvement - it would be really nice to include the proxy URI in the message.

If I am using text EMTL it works, I no longer need my override of registerPackages to install the unresolved proxy.
Comment 17 Ed Willink CLA 2011-05-18 07:03:01 EDT
(In reply to comment #16)
> If I am using binary EMTL, I get two messages and stack traces reporting that
> the template parameter is a proxy; the message is a big improvement - it would
> be really nice to include the proxy URI in the message.
> 
> If I am using text EMTL it works, I no longer need my override of
> registerPackages to install the unresolved proxy.

This is probably related to a difference in proxy resolution.

In the textual EMTL, a reference to http://www.eclipse.org/ocl/3.1.0/Pivot is consistently serialised as http://www.eclipse.org/ocl/3.1.0/Pivot#//....

In the binary EMTL there seems to be some almost uncontrolled model search so that the equivalent serialisation is /org.oml.ocl/model/Clause8Dev/pivot.ecore#//.... which is indeed a model that has the http://www.eclipse.org/ocl/3.1.0/Pivot URI, but it was not on anything resembling a classpath so it was not eligible to be found.

Therefore:
- binary and textual EMTL are not equivalent
- the URI is incorrectly being resolved for binary EMTL
- resources are wasted searching for files in inappropriate places.
Comment 18 Stephane Begaudeau CLA 2011-05-18 11:38:28 EDT
- binary and textual EMTL are not equivalent
Just like several other options, the URI handler only exists for XML resources (XMLResource.OPTION_URI_HANDLER and XMLResource.URIHandler) so it won't be possible to affect the binary serialization in the same way as the XML serialization.

Unless someone has a solution, the binary serialization will always resolves uri with the logical uri (http://...) if the metamodel is deployed as a plugin and with the physical uri (myproject/model/mymetamodel.ecore) if the metamodel is in the workspace.

- the URI is incorrectly being resolved for binary EMTL
I have fixed a problem that had appear with binary resources but since EMF doesn't provide an API for URI handler with binary resources, this serialization will still create some problems in some very specific configurations:

1- the same bundle containing the metamodel is installed as a plugin and it has been imported in the workspace *AND* the user wants to manually register the package.

The Acceleo builder will register the package from the workspace and the user should have a "!isInWorkspace" to prevent the registration of the package from the workspace but since the package also exists in the plugin, the call to "!isInWorkspace" will not prevent the user from overriding the already registered package from the workspace by the package from the plugin. In that case, the uri of the metamodel registered in the binary files will not be resolved anymore since the metamodel from the workspace registered in our proxy package registry would have been overridden by the package from the plugin. A message indicating that the first parameter of the main template is a proxy (uri: myProject/model/myMetamodel.ecore) would appear. If the metamodel is installed as a plugin and if the user imports it in the workspace then the package registration must be removed.

2- creating emtl files with binary resources serialization with the metamodel in the workspace will create emtl files with the uri "myProject/model/myMetamodel.ecore" if those emtl files are not compiled once again with the metamodel deployed as a plugin then if the Acceleo generator is deployed as a plugin, it will look for a metamodel registered with the uri "myProject/model/myMetamodel.ecore" and not "http://...". Registering the metamodel with both uris may solve the problem.


I'll run some new tests tomorrow and I'll create a build with the fix.
Comment 19 Ed Willink CLA 2011-05-18 11:52:15 EDT
(In reply to comment #18)
> - binary and textual EMTL are not equivalent
This seems to be the problem. I couldn't follow the rest but it all seems a complex and dubious workaround for this problem. You risk creating new problem to hide old problems and so cause more confusion. If binary and textual EMTL are not equivalent, there seems to be ample opportunity for a series of user surprises.

If the URI handling is different, perhaps you need to normalize the URIs in the files yourself so that once it hits EMF there is no distinction.

Perhaps you need to serialize to text always, and if binary is required reload and then reserialize so that the first to text serialisation normalizes.
Comment 20 Laurent Goubet CLA 2016-03-31 05:02:14 EDT
This only happens with binary serialization of the compiled emtl file, and only if the target metamodel is in the workspace when developing the generation module. Since there is no means for us to influence the uri handlers during binary serialization, we do not plan on trying to fix this last case.