Bug 360926

Comment 1 Ed Willink 2011-10-14 05:42:37 EDT

The following code in my Acceleo Java successfully fights the strange API to allow standalone cross-plugin cross-GIT execution. The addition is all conditionalised on EMFPlugin.IS_ECLIPSE_RUNNING to avoid interaction with inherited plugin initialisations. The two paths should usefully be combined in the caller.

- why is a URL used at all? it sole purpose seems to be to make it really hard by throwing an exception on platform: protocols

- why is the URI doubly constructed?

- PROJECT_NAME would be a useful generated constant

    public static final String PROJECT_NAME = "org.eclipse.qvtd.build";

    private static ProjectMap projectMap = new ProjectMap();
    
    @Override
    public void registerResourceFactories(ResourceSet resourceSet) {
        super.registerResourceFactories(resourceSet);
        if (!EMFPlugin.IS_ECLIPSE_RUNNING) {
            projectMap.initializeResourceSet(resourceSet);
        }
    }

    @Override
    protected URL findModuleURL(String moduleName) {
        if (!EMFPlugin.IS_ECLIPSE_RUNNING) {
            URL url;
            try {
                String moduleURL = getClass().getResource(moduleName).toString();
                int index = moduleURL.lastIndexOf("/" + PROJECT_NAME + "/");
                if (index >= 0) {
                    url = new URL("file:" + moduleURL.substring(index));    // Bogus URL to pass value to createTemplateURI
                    return url;
                }
            } catch (MalformedURLException e) {        // Never happens
            }
        }
        return super.findModuleURL(moduleName);
    }

    @Override
    protected URI createTemplateURI(String entry) {
        if (!EMFPlugin.IS_ECLIPSE_RUNNING) {
            return URI.createPlatformResourceURI(entry.substring(5), true);
        }
        return super.createTemplateURI(entry);
    }

The whole construction of the template URI could be much more elegant if the original platform:/resource URI was emitted to the Java rather than being reconstructed from the class resource.

Comment 2 Ed Willink 2011-10-15 12:16:38 EDT

(In reply to comment #0)
> I plan to contribute
> /org.eclipse.ocl.examples.domain/src/org/eclipse/ocl/examples/domain/utilities/ProjectMap.java
> in the bug/349962 branch to EMF. It ensures that a ResourceSet is initialized
> to satisfy the above in both standalone and plugin modes.

See Bug 361063.

Comment 3 Ed Willink 2011-10-26 04:32:43 EDT

A related problem, not worth a separate Bugzilla, is that 

AcceleoResourceFactoryRegistry.computeFactory

cannot open a URI such as

platform:/resource/org.eclipse.ocl.examples.codegen/bin/org/eclipse/ocl/examples/codegen/tables/model2tables.emtl

for

ContentHandler.UNSPECIFIED_CONTENT_TYPE.

(It returns null).

The 3.2 functionality seems to rely on a non-standard AcceleoResourceFactoryRegistry in order to provide a non-standard solution to a standard problem. The standard functionality is very powerful, difficult to understand and so easily ignored and broken. I think you will find that your non-standard solution introduces more problems than it solves, which is why URIs are proving so troublesome.

Philosophically content-types are more powerful and elegant than extensions, but they require a content analysis and you have fixed extensions anyway, so why not use them? *.emtl therefore opens an EMTL file, and the EMTLResourceFactory performs the content analysis to create the BinaryEMTLResource or XMIEMTLResource (if distinct Resource classes rather than Load/Save options are actually needed).

Comment 4 Stephane Begaudeau 2011-10-26 05:16:52 EDT

(In reply to comment #3)
> A related problem, not worth a separate Bugzilla, is that 
> 
> AcceleoResourceFactoryRegistry.computeFactory
> 
> cannot open a URI such as
> 
> platform:/resource/org.eclipse.ocl.examples.codegen/bin/org/eclipse/ocl/examples/codegen/tables/model2tables.emtl
> 
> for
> 
> ContentHandler.UNSPECIFIED_CONTENT_TYPE.
> 
> (It returns null).
> 
> The 3.2 functionality seems to rely on a non-standard
> AcceleoResourceFactoryRegistry in order to provide a non-standard solution to
> a standard problem.

We have to use a custom solution to handle both XMI and Binary resources for the same extension. The standard solution with the content type declared in the extension point cannot work for us in all our cases (stand alone for example). (Yet we are also using this extension point in org.eclipse.acceleo.model)

> The standard functionality is very powerful, difficult to understand and so 
> easily ignored and broken.

> Philosophically content-types are more powerful and elegant than extensions,
> but they require a content analysis and you have fixed extensions anyway, so
> why not use them?

We do in... AcceleoResourceFactoryRegistry.computeFactory from line 206 to line 222 with the EMtlBinaryResourceContentDecriber.

The content type of the file at the given URI is used but in order to use this, we need a valid URI and from a look at the code we are supporting in this method two types of URIs: the standard "file:" ones and the standard "jar:file:" ones.

This uri "platform:/resource/org.eclipse.ocl.examples.codegen/bin/org/eclipse/ocl/examples/codegen/tables/model2tables.emtl" should be resolved prior the call the the resource factory registry.

Comment 5 Ed Willink 2011-10-26 12:23:24 EDT

(In reply to comment #4)

> This uri
> "platform:/resource/org.eclipse.ocl.examples.codegen/bin/org/eclipse/ocl/examples/codegen/tables/model2tables.emtl"
> should be resolved prior the call the the resource factory registry.

Which is why Acceleo is so buggy. The URIConverter resolves it.

EMF ONLY works when files are opened using the consistent URI and the only consistent URI is platform:/... anything involving file: is subject to inconsistent ../../.. proxy resolution between Standalone and Plugin. Hence my development of Projectmap to ensure that the full platform:/... access capability can be reconstructed Synadalone. Since using ProjectMap many of my Standalone problems have vanished, and I look forward to eliminating a lot more clunky URI code as I review.

Comment 6 Laurent Goubet 2011-10-27 03:02:27 EDT

(In reply to comment #5)
> (In reply to comment #4)
> 
> > This uri
> > "platform:/resource/org.eclipse.ocl.examples.codegen/bin/org/eclipse/ocl/examples/codegen/tables/model2tables.emtl"
> > should be resolved prior the call the the resource factory registry.
> 
> Which is why Acceleo is so buggy. The URIConverter resolves it.

The nominal case works, standalone works, but indeed we have trouble when the very metamodels we depend on, OCL and Ecore, are somewhere in the workspace.

> 
> EMF ONLY works when files are opened using the consistent URI and the only
> consistent URI is platform:/... anything involving file: is subject to
> inconsistent ../../.. proxy resolution between Standalone and Plugin. Hence my
> development of Projectmap to ensure that the full platform:/... access
> capability can be reconstructed Synadalone. Since using ProjectMap many of my
> Standalone problems have vanished, and I look forward to eliminating a lot more
> clunky URI code as I review.

Did you take into account the fact that a model might very well be neither in plugins, nor in workspace? That can happen even within Eclipse. "platform:" URIs are not limited to "plugin" and "resource". models located in plugin fragments will be referred to as platform:/fragment/..., there also exists platform:/base, platform:/meta, and I seem to remember there was a sixth one, but can't find it back. The models could even be located outside of the workspace (symlink, a "jar" added to one of the project's classpath as a library...)

Digressions apart, EMF does not "only" work with platform. As a matter of fact, the "platform" protocol is totally useless in standalone (true standalone, no RCP, no headless, no OSGi instance running in the background ... only plain old jars and "org.eclipse.core.*", "org.eclipse.equinox", "org.eclipse.osgi",... jars nowhere to be found in the classpath; that's what is "standalone").

In standalone, what we have is a filesystem, and jars. The URI protocols that are useful in that setup are "file" and "jar:file". Even the EMF "virtual" http-protocol URIs need to be manually registered then. Trying to resolve a "platform" protocol, unknown to Java, in such a context will simply fail.

When your projectmap project has come to fruition and when you are able to assure us that it works in standalone, rcp, headless and Eclipse contexts, that it works with models located anywhere (plain jars, plugins, fragments, workspace (plugin projects and java projects alike), out of the workspace, etc), we might consider switching our solution to yours.

i.e : we have a solution that works in most cases even though some "metamodel schyzophrenia" cases remain. We cannot be certain that we work in all cases, but are pretty confident about the mechanism nonetheless. We won't switch to another mechanism if it is only to resolve the corner case of 'having OCL and Ecore in the workspace' and see some of the cases we handle 'now' fail. It needs to be a more consequent fix, and we do not have much bugs opened against URI resolution yet.

Comment 7 Ed Willink 2011-10-27 13:35:05 EDT

(In reply to comment #6)
> The nominal case works, standalone works, but indeed we have trouble when the
> very metamodels we depend on, OCL and Ecore, are somewhere in the workspace.

When I step through the code, it feels as if I'm wading through workaround layered upon workaround that sort of works for limited cases. It's not surprising you don't want to change.

ProjectMap was developed specifically because of the problems I encountered when running Acceleo as a GenModelAdapter; I wanted a solution that also worked for JUnit tests, MWE workflows and Acceleo launches.

I've not heard of platform:/base so that may require further consideration.

RCP etc, I think, fall into one of two categories;

ECLIPSE_IS_RUNNING in which case plugin registrations have occurred, and EMF provides useful but not always complete platform:/resource and platform:/plugin mapping

!ECLIPSE_IS_RUNNING in which case EMF is very naked and platform:/resource and platform:/plugin are dysfunctional.

---

The basic problem is how does A.ecore reliably reference B.ecore, when the location of either may change for different users? may be project file; may be in a plugin jar, may be in a simple workspace where all projects are siblings, or may be in a partitioned workspace where different projects are in different GITs.

For a single environment, the problem is relatively simple; just make sure that URIs work once; this motivates use of file: as a simple fix for a standalone environment.

For multiple environments it is much harder, and Acceleo is about as challenging as it can get; editor, editor-builder, nature-builder, plugin-executor, standalone-executor, debuggers.

Back to the problem: how does A.ecore reliably reference B.ecore when absolute paths are unstable and relative paths are unstable? Well Done EMF; the problem is solvable. If all workspace resources are always accessed as platform:/... then there is a stable absolute URI for all workspace resources, and a corresponding stable relative URI for all workspace resources. (Non-workspace resources can just use absolute file:/whatever URIS; they are not part of this portability discussion.)

Since platform:/... provides a stable relative URI, XMLSave can safely deresolve URIs with respect to the URI of the referencing resource thereby reducing the length of persisted URIs. This unfortunately gives the impression that they can be reresolved with respect to file: URIs. For simple workspace layouts they can; in general they cannot. It is essential that both creator and user of resources use compatible platform:/... schemes. This is easy in a plugin environment where it's almost all setup nicely. It's really hard in a standalone environment where nothing works out of the box. But EMF provides the tools to make it work.

The URIConverter.uriMap is used early and regularly during URI normalisation. It can map the appropriate platform:/resource/x to platform:/plugin/x when x is a 'plugin', and platform:/plugin/y to platform:/resource/y when y is a 'project'.

The EcorePlugin.getPlatformResourceMap is used very late by ExtensibleURIConverter.openStream. It can therefore map platform:/plugin/x to some JAR file, and platform:/resource/y to some file: location.

MWE partially showed how an Eclipse environment could be recreated for standalone users; scan the class path for:
JAR files with manifests => PlatformResourceMap plugin entries
folders with .project files =>  PlatformResourceMap resource entries

However MWE did not fully support the platform:/resource <> platform:/plugin duality.

ProjectMap goes the extra step further of initialising not only the PlatformResourceMap from the class path, but also the URIMap, so that thereafter platform:/resource and platform:/plugin have the same semantics standalone and in-Eclipse. So, if all workspace resources are opened with platform: URIs, the difference between standalone and Eclipse for URI resolution is removed. Magic.

[The class path scan could be taken further to find genmodel and package registrations in plugin.xml, but for niow URI resolution is the main problem.]

[This discussion is all about portable URI resolution, but it may mitigate metamodel schizophrenia since if http: URIs are mapped to platform:/ URIs it is possible for EMF to detect that the same path is in use. If http: is mapped to file: then EMF tends not to notice that http: and platform:/... are the same resource.]

Comment 8 Laurent Goubet 2011-10-28 03:01:44 EDT

(In reply to comment #7)
> (In reply to comment #6)
> > The nominal case works, standalone works, but indeed we have trouble when the
> > very metamodels we depend on, OCL and Ecore, are somewhere in the workspace.
> 
> When I step through the code, it feels as if I'm wading through workaround
> layered upon workaround that sort of works for limited cases. It's not
> surprising you don't want to change.

That's the gist of it.

URI Resolution is no more a pile of kludgy workarounds than a critical part of the Acceleo core. We won't break such a part because "it is buggy" (point taken, though in a single identified case), but because "it does not work for a number of use cases, when the models/modules are structured such or such way". We won't change it simply on account of a theory, but when we have been shown that our workarounds are worse than solution X.

If the ProjectMap works for the use case that fails for you _and_ handle the cases that we already handle through our workarounds, allowing Acceleo to work in standalone, in Eclipse, when metamodels are deployed in the workspace (yes, for others than OCL and Ecore, you can have your metamodel in the workspace), to compile and evaluate through ant and maven... Then we will consider changing our solution to the ProjectMap. It is too much of a critical issue to change that much (and risk breaking) on account of a single bug as long as we are not certain not to introduce a number of regressions.

We will check our workflow with projects from different gits and located in different locations of the file system. If that does fail, we will check how we can handle the use case. The ProjectMap might be one of the solutions, that we'll only know when we tackle this bug.

Comment 9 Laurent Goubet 2016-04-06 05:06:49 EDT

We no longer have relative URIs when referencing other files from the xmi serialization. There are still problems when importing modules from another project in standalone mode (since they're referred to as platform:/resource and that won't stand in standalone).

Comment 10 Laurent Goubet 2016-04-06 09:08:17 EDT

*** Bug 360934 has been marked as a duplicate of this bug. ***