Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 364419 - Provide Standalone init
Summary: Provide Standalone init
Status: VERIFIED FIXED
Alias: None
Product: MDT.UML2
Classification: Modeling
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows Vista
: P3 enhancement (vote)
Target Milestone: M7   Edit
Assignee: Kenn Hussey CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 365487 369252 (view as bug list)
Depends on:
Blocks: 377262
  Show dependency tree
 
Reported: 2011-11-21 16:53 EST by Ed Willink CLA
Modified: 2012-04-20 14:51 EDT (History)
1 user (show)

See Also:
Kenn.Hussey: juno+


Attachments
Standalone init (8.78 KB, text/plain)
2011-12-02 09:12 EST, Ed Willink CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2011-11-21 16:53:03 EST
I'm trying to open Infrastructure.xmi standalone. Hard. Looking at how UMLEditor works, it seems that UMLEditor.createModel() contains useful code that could be refactored for re-use as a standalone ResourceSet init; null ResourceSet for global registries.
Comment 1 Ed Willink CLA 2011-11-22 02:26:23 EST
Cloning the UMLEditor.createModel() code is not enough. It doesn't contain the package registrations and it relies on a running Eclipse platform to perform content analysis.

Is there a way to open Infrastructure.xmi standalone (without prejudging the content)?

When content types were first introduced, I found that the usage of the platform content analysis subverted the input stream cache and so made content types needlessly expensive. Is it therefore desirable for EMF to provide direct content type analysis and so avoid the platform dependence and inefficiency?

---

For the benefit of Bugzilla searchers, this problem ultimately leads to

Thread [main] (Suspended (exception UnsupportedOperationException))	
	UnionEObjectEList<E>(BasicEList$UnmodifiableEList<E>).grow(int) line: 1035	
	UnionEObjectEList<E>(BasicEList<E>).addUnique(E) line: 422	
	XMIHelperImpl(XMLHelperImpl).setValue(EObject, EStructuralFeature, Object, int) line: 1179	

because the default SAXXMIHandler, rather than the required XMI2UMLHandler, is used as a consequence of a XMIResourceImpl, rather than a XMI2UMLResourceImpl, being created as a result on content analysis failure.

The above may go unnoticed in which case

Caused by: org.eclipse.emf.ecore.xmi.ClassNotFoundException: Class 'Type' is not found or is abstract. (platform:/resource/org.eclipse.uml2/model/Infrastructure.xmi, 36, 95)
	at org.eclipse.emf.ecore.xmi.impl.XMLHandler.validateCreateObjectFromFactory(XMLHandler.java:2244)

appears in the ResourceSet errors, because the new XMI handling is needed to resolve XMI references to the non-abstract target type.
Comment 2 Ed Willink CLA 2011-12-02 09:12:11 EST
Created attachment 207844 [details]
Standalone init

The attached which is /org.eclipse.ocl.examples.pivot/src/org/eclipse/ocl/examples/pivot/uml/UMLUtils.java in the OCL examples area of GIT has evolved so that

use of ProjectMap.initializeResourceSet(null) (from Bug 361063 and /org.eclipse.ocl.examples.domain/src/org/eclipse/ocl/examples/domain/utilities/ProjectMap.java) supports init of EMF global registries for equivalent content in either standalone or plugin mode, in particular ensuring platform: is mapped.

then use of UMLUtils.initializeContents() installs the many subtle UML registrations necessary to allow UML models to be opened standalone.

It would be helpful for a UML committer to complete/correct the registrations and make the initialization capability a fully supported facility.
Comment 3 Kenn Hussey CLA 2011-12-05 13:02:21 EST
*** Bug 365487 has been marked as a duplicate of this bug. ***
Comment 4 Kenn Hussey CLA 2012-01-21 08:33:01 EST
*** Bug 369252 has been marked as a duplicate of this bug. ***
Comment 5 Kenn Hussey CLA 2012-03-13 21:29:50 EDT
OK, I've committed/pushed changes which extract the redundant initializations out of UMLEditor and UMLImporter into a utility method in UMLUtil. I'm not sure how much more I'll be willing/able to do for Juno. What's there now should be sufficient for use in Eclipse-based applications...
Comment 6 Ed Willink CLA 2012-03-14 02:22:36 EDT
That is certainly helpful, but in comparison to /org.eclipse.ocl.examples.pivot/src/org/eclipse/ocl/examples/pivot/uml/UMLUtils.java

(trivially: I think using explicit versions throughout is more maintainable e.g. UML_2_4_1_CONTENT_TYPE_IDENTIFIER)

The pathmaps are missing:

uriMap.put(URI.createURI(UMLResource.PROFILES_PATHMAP), URI.createPlatformPluginURI("/org.eclipse.uml2.uml.resources/profiles/", true)); //$NON-NLS-1$
uriMap.put(URI.createURI(UMLResource.METAMODELS_PATHMAP), URI.createPlatformPluginURI("/org.eclipse.uml2.uml.resources/metamodels/", true)); //$NON-NLS-1$
uriMap.put(URI.createURI(UMLResource.LIBRARIES_PATHMAP), URI.createPlatformPluginURI("/org.eclipse.uml2.uml.resources/libraries/", true)); //$NON-NLS-1$

The local package registrations are missing (perhaps they're not needed since a global registration is unavoidable) e.g.:

packageRegistry.put(XMI2UMLResource.UML_PRIMITIVE_TYPES_LIBRARY_2_4_URI, TypesPackage.eINSTANCE);

The content handlers are missing: (too much code to cut and paste)

The Ecore2XML registration is missing:

Ecore2XMLPackage.eINSTANCE.eClass();		


-------

In one of the duplicates, I suggested that this init could be called automatically from UMLPackage.init() if EMFPlugin.IS_ECLIPSE_RUNNING is false avoiding all the complexities for users.
Comment 7 Kenn Hussey CLA 2012-04-16 15:36:45 EDT
One problem I ran into (while considering adding the code to ensure packages are initialized) is that a utility in org.eclipse.uml2.uml can't initialize things for org.eclipse.uml2.uml.profile.l2 or org.eclipse.uml2.uml.profile.l3 without introducing circular dependencies. If you are looking for a one-stop utility to initialize everything needed to use UML2 in a stand-alone environment, I think it should be added to a new package/class in org.eclipse.uml2.uml.resources (which also happens to define the needed pathmap registrations). Unfortunately, since we're past API freeze, I think this may have to wait for Keppler. :(
Comment 8 Ed Willink CLA 2012-04-16 15:49:47 EDT
This is new API so it cannot break anybody and it's so important that I feel it should be done for Juno.

UML 2.4/2.2 co-existence is really hard.
Comment 9 Kenn Hussey CLA 2012-04-16 17:32:54 EDT
OK, I'll try to get it in.
Comment 10 Kenn Hussey CLA 2012-04-17 12:25:22 EDT
The stand-alone utility, org.eclipse.uml2.uml.resources.util.ResourcesUtil#init(ResourceSet), has been committed and pushed to git. Please confirm that it meets your requirements.
Comment 11 Ed Willink CLA 2012-04-18 02:42:13 EDT
The new code replaces almost all of my code but I still need to specify the following default content handlers for extension only matches _before_ content handlers for explicit nsURIs. 

private static final String CMOF_CONTENT_TYPE_IDENTIFIER = UMLUtils.class.getName() + ".cmof";
private static final String UML_CONTENT_TYPE_IDENTIFIER = UMLUtils.class.getName() + ".uml";
private static final String XMI_CONTENT_TYPE_IDENTIFIER = UMLUtils.class.getName() + ".xmi";

private static final RootXMLContentHandlerImpl CMOF_DEFAULT = new RootXMLContentHandlerImpl(CMOF_CONTENT_TYPE_IDENTIFIER, new String[]{ "cmof" },
		RootXMLContentHandlerImpl.XMI_KIND, "", null);
private static final RootXMLContentHandlerImpl UML_DEFAULT = new RootXMLContentHandlerImpl(UML_CONTENT_TYPE_IDENTIFIER, new String[]{ "uml" },
		RootXMLContentHandlerImpl.XMI_KIND, "", null);
private static final RootXMLContentHandlerImpl XMI_DEFAULT = new RootXMLContentHandlerImpl(XMI_CONTENT_TYPE_IDENTIFIER, new String[]{ "xmi", "xml" },
		RootXMLContentHandlerImpl.XMI_KIND, "", null);

Otherwise when Xtext opens its GrammarFile.xmi it tries to use XMI212UMLResourceFactoryImpl with catastrophic results.

[Shouldn't the uml.resources have dependency ranges?]

[uml.resources could re-export its UML2 dependencies saving downstream projects from UML structural knowledge.]
Comment 12 Kenn Hussey CLA 2012-04-18 09:07:09 EDT
(In reply to comment #11)
> The new code replaces almost all of my code but I still need to specify the
> following default content handlers for extension only matches _before_ content
> handlers for explicit nsURIs.

I'm not sure I understand the need for this. UML2 doesn't make such registrations when running in Eclipse, why should it do so when running stand-alone?

> Otherwise when Xtext opens its GrammarFile.xmi it tries to use
> XMI212UMLResourceFactoryImpl with catastrophic results.

If anything, I suppose a general handler for .xmi files (as declared in EMF) is needed, then?

> [Shouldn't the uml.resources have dependency ranges?]

No, these are added by the build.

> [uml.resources could re-export its UML2 dependencies saving downstream projects
> from UML structural knowledge.]

Added.
Comment 13 Ed Willink CLA 2012-04-18 09:57:16 EDT
(In reply to comment #12)
> I'm not sure I understand the need for this. UML2 doesn't make such
> registrations when running in Eclipse, why should it do so when running
> stand-alone?
> 
> > Otherwise when Xtext opens its GrammarFile.xmi it tries to use
> > XMI212UMLResourceFactoryImpl with catastrophic results.
> 
> If anything, I suppose a general handler for .xmi files (as declared in EMF) is
> needed, then?

The introduction of content-type registration at lower rather than higher priority was a safe compatibility decision but it means that there can be a variety of adverse interactions between the mechanisms.

The 'Xtext' problem arises entirely in the global registries where there are extensionMap entries for {*,essentialocl,ecore} and UML's content types, so an attempt to open a non-UML XMI file proceeds globally as:
- protocol match - fail
- explicit extension match - fail
- exact content type match - fail
  content type for extension match - pass; use UML 2.1 for non-UML *.xmi
- default extension match - occluded

It is very wrong for a content type for a known namespace to provide a default match for a completely inappropriate namespace. Hence my entries giving defaults ensure that the default extension match is not occluded.

Standalone is a nightmare; every project has its own measures to workaround the lack of a coherent EMF standalone registration solution. And this means that projects gradually acquire counter-measures to workaround the bad solutions of known co-existing projects. This in turn requires counter-counter-measures ...

> > [Shouldn't the uml.resources have dependency ranges?]
> 
> No, these are added by the build.

Yes, but the defaults are very restrictive. No problem today since 4.0.0 is a solid limit, but on 4.1 4.0 will be excluded.
Comment 14 Kenn Hussey CLA 2012-04-18 10:32:00 EDT
(In reply to comment #13)

Is it not enough, then, just to do this before the namespace-specific handlers are added to the list?

		contentHandlers.add(new XMLContentHandlerImpl.XMI());
Comment 15 Ed Willink CLA 2012-04-18 10:40:31 EDT
Yes. That works for me, but highlights another concern; multiple init() calls.

Registries don't mind multiple puts, but ContentHandler lists grow ridiculously large. My code therefore had static instances so that it could detect already present. Your code creates more and more new instances.
Comment 16 Kenn Hussey CLA 2012-04-18 13:17:33 EDT
OK, I've committed and pushed a change which adds the default XMI handler and avoid duplicate additions. If this looks OK, I'll run an integration build.
Comment 17 Ed Willink CLA 2012-04-18 15:19:31 EDT
Looks good. Many thanks. This will really help when UML 2.5 comes along.

Probably worth a m2m-dev, mdt-dev, m2t-dev posting to highlight the opportunity for UML tools to become UML-version blind.
Comment 18 Kenn Hussey CLA 2012-04-18 16:14:14 EDT
An integration build containing the changes is now available.

I'll give you the honor of spreading the news, seeing as it was your idea. ;)
Comment 19 Ed Willink CLA 2012-04-19 01:08:03 EDT
A last minute query:

Is ResourceUtil a good name? The usage may often be in a batch of initialisations and so it is not clear, unless package qualification is used, that this is UML rather than EMF functionality. UMLResource(s)Util would seem friendlier.

For new OCL and QVT code, I've been adopting the Xtext spelling convention of XXXStandaloneSetup.doSetup(). So perhaps UMLStandaloneSetup.doSetup() just calls UMLStandaloneSetup.init(ResourceSet) with a null argument.
Comment 20 Ed Willink CLA 2012-04-19 08:12:00 EDT
Thought I should try my build scripts as well as my JUnit tests before committing the OCL simplification to GIT. Three problems.

1) new XMLContentHandlerImpl.XMI() is not a substutute for the three defaults from comment #6. XMLContentHandlerImpl.XMI() identifies a http://www.omg.org/spec/UML/20090901 *.cmof file as XMI rather than CMOF.

2) OMG UML namespaces are registered for just *.xmi. Why not *.uml as well?

3) Ecore2XMLPackage needs to be registered. Arguably this is an EMF bug/feature since Ecore2XMLResource.Factory.INSTANCE could register Ecore2XMLPackage automatically. But without it *.cmof reading fails miserably.
Comment 21 Kenn Hussey CLA 2012-04-19 09:28:52 EDT
(In reply to comment #19)
> Is ResourceUtil a good name? The usage may often be in a batch of
> initialisations and so it is not clear, unless package qualification is used,
> that this is UML rather than EMF functionality. UMLResource(s)Util would seem
> friendlier.

Renamed.

(In reply to comment #20)
> 1) new XMLContentHandlerImpl.XMI() is not a substutute for the three defaults
> from comment #6. XMLContentHandlerImpl.XMI() identifies a
> http://www.omg.org/spec/UML/20090901 *.cmof file as XMI rather than CMOF.

Hmm, but that sounds correct. That's the UML namespace in a *.cmof file, which is an unexpected file extension for UML (as opposed to CMOF) resources. What am I missing?

> 2) OMG UML namespaces are registered for just *.xmi. Why not *.uml as well?

Because this is how it has long been done in plugin.xml and here we're just trying to support the same thing in standalone mode.

> 3) Ecore2XMLPackage needs to be registered. Arguably this is an EMF bug/feature
> since Ecore2XMLResource.Factory.INSTANCE could register Ecore2XMLPackage
> automatically. But without it *.cmof reading fails miserably.

Done.
Comment 22 Ed Willink CLA 2012-04-19 11:42:45 EDT
(In reply to comment #21)
> (In reply to comment #20)
> > 1) new XMLContentHandlerImpl.XMI() is not a substutute for the three defaults
> > from comment #6. XMLContentHandlerImpl.XMI() identifies a
> > http://www.omg.org/spec/UML/20090901 *.cmof file as XMI rather than CMOF.
> 
> Hmm, but that sounds correct. That's the UML namespace in a *.cmof file, which
> is an unexpected file extension for UML (as opposed to CMOF) resources. What am
> I missing?

The relevant 'cmof' file is a multi-package merge controller that I created in Oct 2010 when I had less idea what I was doing. It is possible that I adjusted an nsURI to workaround a perceived problem. So I can easily adjust this and we could ignore it. The file has been edited many times by the UML Model Editor, so the usage is a persistent output.

The philosophical question is: If a *.cmof file has an unknown nsURI should it be opened as a best endeavours latest CMOF or as XMI? I favour latest CMOF since when a new or prototype nsURI comes along it will probably be 99% correct as CMOF rather than 100% wrong as XMI.
Comment 23 Kenn Hussey CLA 2012-04-19 15:02:32 EDT
UML2 doesn't generally register resource factories by file extension, which would be how you'd do "best effort based on latest version". UML2 has always been specific about which file extension and namespace combinations it supports, and I'd rather keep it that way. So, if you could revisit the content of that file, that would be cool. Are the other changes (including the ignore files for bin folders, for bug 377122) OK? If so, I'll publish an(other) integration build.
Comment 24 Ed Willink CLA 2012-04-20 02:28:17 EDT
UMLResourcesUtil name - good.

Ecore2XMLPackage registration - good.

(An eight-fold cut and paste might merit a helper routine).

XMI content handler - bad.

Registering XMI as the first normal priority content handler ensures that it occludes everything else. Putting it last would be better. Making it LOW_PRIORITY or VERY_LOW_PRIORITY is much better.

*.uml OMG content - bad.

I misdiagnosed the problem: The "http://www.omg.org/spec/UML/20090901 *.cmof" was for the "http://www.omg.org/spec/UML/20090901" UML package; the definition was "http://schema.omg.org/spec/MOF/2.0/cmof.xml". After instruimenting the content handler code and fixing the XMI priority the residual problem is that the result of the UML package merge is a *.uml file for "http://www.omg.org/spec/UML/20110701", which is not a registered content. Changing the merged file by hand to "http://www.eclipse.org/uml2/4.0.0/UML" and regenerating once again gives "http://www.omg.org/spec/UML/20110701", so it would seem that either

- package merge should give "http://www.eclipse.org/uml2/4.0.0/UML"
- or *.uml should be acceptable for "http://www.omg.org/spec/UML/20110701"
Comment 25 Ed Willink CLA 2012-04-20 02:39:04 EDT
(In reply to comment #24)
> - package merge should give "http://www.eclipse.org/uml2/4.0.0/UML"
> - or *.uml should be acceptable for "http://www.omg.org/spec/UML/20110701"
No: my code should specify "org.eclipse.uml2.uml_4_0_0" content type.

Just the XMI priority needed:

diff --git a/plugins/org.eclipse.uml2.uml.resources/src/org/eclipse/uml2/uml/resources/util/UMLResourcesUtil.java b/plugins/org.eclipse.uml2.uml.resources/src/org/eclipse/uml2/uml/resources/util/UMLResourcesUtil.java
index af43264..7d3a0d5 100644
--- a/plugins/org.eclipse.uml2.uml.resources/src/org/eclipse/uml2/uml/resources/util/UMLResourcesUtil.java
+++ b/plugins/org.eclipse.uml2.uml.resources/src/org/eclipse/uml2/uml/resources/util/UMLResourcesUtil.java
@@ -187,6 +187,24 @@
 
 		if (resourceSet == null) {
 			contentHandlers = ContentHandler.Registry.INSTANCE
+				.get(ContentHandler.Registry.LOW_PRIORITY);
+
+			if (contentHandlers == null) {
+				ContentHandler.Registry.INSTANCE.put(
+					ContentHandler.Registry.LOW_PRIORITY,
+					contentHandlers = new ArrayList<ContentHandler>());
+			}
+		} else {
+			contentHandlers = resourceSet.getURIConverter()
+				.getContentHandlers();
+		}
+
+		if (!contentHandlers.contains(XMI_CONTENT_HANDLER)) {
+			contentHandlers.add(XMI_CONTENT_HANDLER);
+		}
+
+		if (resourceSet == null) {
+			contentHandlers = ContentHandler.Registry.INSTANCE
 				.get(ContentHandler.Registry.NORMAL_PRIORITY);
 
 			if (contentHandlers == null) {
@@ -197,10 +215,6 @@
 		} else {
 			contentHandlers = resourceSet.getURIConverter()
 				.getContentHandlers();
-		}
-
-		if (!contentHandlers.contains(XMI_CONTENT_HANDLER)) {
-			contentHandlers.add(XMI_CONTENT_HANDLER);
 		}
 
 		if (!contentHandlers.contains(UML2_1_0_0_CONTENT_HANDLER)) {
Comment 26 Ed Willink CLA 2012-04-20 05:38:47 EDT
Exploiting the new code in o.e.ocl.uml.OCL.initialize allows a slight simplification.

resourceFactoryRegistry.getExtensionToFactoryMap().put(
	UMLResource.FILE_EXTENSION, UMLResource.Factory.INSTANCE);

is not (yet) provided by UMLResourcesUtil.init

and the final

uriMap.put(URI.createPlatformPluginURI("/org.eclipse.uml2.uml.resources/", true), resourcesLocationURI);

must be done manually until such time as a ProjectMap-style classpath scan is able to determine the file: or jar: resourceLocationURI of the uml.resources plugin.

The extension(s) could be added; the required uriMap entry could be Javadoc'd.
Comment 27 Kenn Hussey CLA 2012-04-20 09:23:24 EDT
(In reply to comment #24)
> Registering XMI as the first normal priority content handler ensures that it
> occludes everything else. Putting it last would be better. Making it
> LOW_PRIORITY or VERY_LOW_PRIORITY is much better.

OK, but how does registering it at a lower priority help the case where a resource set is specified (where there are no priorities)? It seems that a better approach would be to just add it last.

(In reply to comment #26)
> resourceFactoryRegistry.getExtensionToFactoryMap().put(
> UMLResource.FILE_EXTENSION, UMLResource.Factory.INSTANCE);
> 
> is not (yet) provided by UMLResourcesUtil.init

Added it to UMLUtil.init(ResourceSet).
Comment 28 Ed Willink CLA 2012-04-20 09:36:07 EDT
(In reply to comment #27)
> (In reply to comment #24)
> > Registering XMI as the first normal priority content handler ensures that it
> > occludes everything else. Putting it last would be better. Making it
> > LOW_PRIORITY or VERY_LOW_PRIORITY is much better.
> 
> OK, but how does registering it at a lower priority help the case where a
> resource set is specified (where there are no priorities)? It seems that a
> better approach would be to just add it last.

Perhaps it's a design flaw then that ResourceSets do not have priorities.

Lower priority is much better since it continues to work for additional user content handlers. So low priority for global. Last or my original per-extension default handlers for local.

I'm not convinced that local registries are very relevant here. Local is useful to stop Eclipse application A disturbing Eclipse application B. Here we are doing remedial standalone installation so only global really makes sense. 
 
> (In reply to comment #26)
> > resourceFactoryRegistry.getExtensionToFactoryMap().put(
> > UMLResource.FILE_EXTENSION, UMLResource.Factory.INSTANCE);
> > 
> > is not (yet) provided by UMLResourcesUtil.init
> 
> Added it to UMLUtil.init(ResourceSet).

?? any others e.g. cmof ...
Comment 29 Kenn Hussey CLA 2012-04-20 11:23:12 EDT
(In reply to comment #28)
> Lower priority is much better since it continues to work for additional user
> content handlers. So low priority for global. Last or my original per-extension
> default handlers for local.

OK, did low for no resource set, last for resource set.

> ?? any others e.g. cmof ...

No, plugin.xml only registers extension handler for .uml so that's all I added. The thinking is that UML2 doesn't "own" extensions other than .uml.
Comment 30 Ed Willink CLA 2012-04-20 13:41:27 EDT
+1 for all my use cases

OCL UML binding standalone init for JUnit tests
OCL Pivot binding standalone init for JUnit tests 
OCL MWE2 auto-generation scripts init
Comment 31 Kenn Hussey CLA 2012-04-20 14:51:17 EDT
The latest changes are now available in an integration build.