Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 344373 - [xtext][builder] UnsupportedOperationException with newly materialized domainmodel
Summary: [xtext][builder] UnsupportedOperationException with newly materialized domain...
Status: REOPENED
Alias: None
Product: TMF
Classification: Modeling
Component: Xtext (show other bugs)
Version: 2.1.0   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: SR2   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 355362 (view as bug list)
Depends on: 350419
Blocks:
  Show dependency tree
 
Reported: 2011-04-30 08:35 EDT by Sebastian Zarnekow CLA
Modified: 2020-06-23 09:47 EDT (History)
7 users (show)

See Also:
sebastian.zarnekow: indigo+


Attachments
Test Implementation (1.84 MB, application/x-zip-compressed)
2020-06-23 05:45 EDT, Martin Steinbichler CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Zarnekow CLA 2011-04-30 08:35:59 EDT
java.lang.UnsupportedOperationException: getReferenceDescriptions()
at org.eclipse.xtext.builder.clustering.CopiedResourceDescription.getReferenceDescriptions(CopiedResourceDescription.java:70)
at org.eclipse.xtext.resource.DescriptionUtils.collectOutgoingReferences(DescriptionUtils.java:29)
at org.eclipse.xtext.resource.impl.DefaultResourceDescriptionManager.isAffected(DefaultResourceDescriptionManager.java:108)
at org.eclipse.xtext.builder.clustering.ClusteringBuilderState.queueAffectedResources(ClusteringBuilderState.java:315)
at org.eclipse.xtext.builder.clustering.ClusteringBuilderState.doUpdate(ClusteringBuilderState.java:130)
at org.eclipse.xtext.builder.builderState.AbstractBuilderState.update(AbstractBuilderState.java:107)
at org.eclipse.xtext.builder.impl.ProjectOpenedOrClosedListener$2$1.execute(ProjectOpenedOrClosedListener.java:148)
at org.eclipse.ui.actions.WorkspaceModifyOperation$1.run(WorkspaceModifyOperation.java:106)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2310)
at org.eclipse.ui.actions.WorkspaceModifyOperation.run(WorkspaceModifyOperation.java:118)
at org.eclipse.xtext.builder.impl.ProjectOpenedOrClosedListener$2.run(ProjectOpenedOrClosedListener.java:157)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Comment 1 Sebastian Zarnekow CLA 2011-05-02 09:18:20 EDT
I applied a workaround for M7 but we should investigate where the CopiedDescriptions in the index come from.
Comment 2 Sebastian Zarnekow CLA 2011-07-12 17:17:34 EDT
Since I expect this one to be resolved due to bug 350419 I removed the workaround and left a comment in the sources.

Please reopen if the problem persists.
Comment 3 Sven Efftinge CLA 2011-08-22 14:41:54 EDT
got that exception with the latest nightly
Comment 4 Sven Efftinge CLA 2011-08-23 15:41:32 EDT
pushed to master
Comment 5 Sven Efftinge CLA 2011-08-23 15:42:43 EDT
*** Bug 355362 has been marked as a duplicate of this bug. ***
Comment 6 Karsten Thoms CLA 2017-09-19 16:59:44 EDT
Closing all bugs that were set to RESOLVED before Neon.0
Comment 7 Karsten Thoms CLA 2017-09-19 17:11:10 EDT
Closing all bugs that were set to RESOLVED before Neon.0
Comment 8 Titouan Vervack CLA 2017-09-29 09:01:52 EDT
This no longer throws an UnsupportedOperationException but now it throws an IllegalStateException and a TODO to this bug report still exists.
Comment 9 Christian Dietrich CLA 2017-09-29 09:15:35 EDT
how to reproduce?
Comment 10 Titouan Vervack CLA 2017-09-29 10:40:14 EDT
Sadly I only have a stacktrace myself so I don't know where the CopiedDescriptions came from, nor can I reproduce it.
Comment 11 Titouan Vervack CLA 2019-04-11 06:11:29 EDT
This can be reproduced by creating a new file which triggers `org.eclipse.xtext.builder.clustering.ClusteringBuilderState.doUpdate(BuildData, ResourceDescriptionsData, IProgressMonitor)`. If the build is then cancelled at line 141 the index contains a CopiedResourceDescription.

If another build for the same resource triggers after that and you catch a WrappedException, you'll retrieve the CopiedResourceDescription from the index.
Comment 12 Karsten Thoms CLA 2019-04-11 06:13:47 EDT
@Titouan Could you work on a fix? You may reopen the bug and assign it  to you.
Comment 13 Titouan Vervack CLA 2019-04-12 08:06:06 EDT
@Karsten Not at the moment, quite busy. I have however noticed that it's much easier to trigger the bug, all you need is the exception.

The first part (writeNewResourceDescriptions(buildData, this, newState, progress.split(20));) makes sure to create a CopiedResourceDescription if you then get an exception while trying to load the resource you'll be stuck with an invalid index.
Comment 14 Elie Richa CLA 2019-07-19 07:15:48 EDT
Hello,

I've encountered this bug in my development and I'd like to ask if there's a workaround. The scenario is the following.

My validator has a check that needs to look at the IResourceDescription instances contained in the index. This is similar to the typical use case of checking global uniqueness of some attributes. Usually all works well if we look at the IEObjectDescriptions exported by resources.

The error occurs because I try to access the reference descriptions of the resource description. The CopiedResourceDescription doesn't seem to copy the reference descriptions. Instead it logs an IllegalStateException without throwing it, which clutters the Error Log.

Any idea how I can work around this problem?

Why doesn't the CopiedResourceDescription class also copy reference descriptions?

Should I try to implement MyCopiedResourceDescription that copies references and provide MyClusteringBuilderState that instantiates MyCopiedResourceDescription instead of CopiedResourceDescription?

Any help would be appreciated :) Thanks!

Here is the logged error:

 java.lang.IllegalStateException: getReferenceDescriptions platform:/resource/QGen/3_osss/3.1_scfr/dataflow.bls
	at org.eclipse.xtext.builder.clustering.CopiedResourceDescription.getReferenceDescriptions(CopiedResourceDescription.java:84)
	at org.irit.xtext.utils.VTArtifactUtils$AbsoluteLabelComputer.internalGetAbsoluteLabel(VTArtifactUtils.java:106)
	at org.irit.xtext.utils.VTArtifactUtils$AbsoluteLabelComputer.lambda$0(VTArtifactUtils.java:64)
	at org.eclipse.xtext.util.SimpleCache.get(SimpleCache.java:75)
	at org.irit.xtext.utils.VTArtifactUtils$AbsoluteLabelComputer.getAbsoluteLabel(VTArtifactUtils.java:86)
	at org.irit.xtext.validation.BlocklibraryValidator.lambda$148(BlocklibraryValidator.java:2669)
	at org.eclipse.xtext.xbase.lib.IteratorExtensions.groupBy(IteratorExtensions.java:814)
	at org.eclipse.xtext.xbase.lib.IterableExtensions.groupBy(IterableExtensions.java:711)
	at org.irit.xtext.validation.BlocklibraryValidator.uniqueUid(BlocklibraryValidator.java:2671)
	at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.xtext.validation.AbstractDeclarativeValidator$MethodWrapper.invoke(AbstractDeclarativeValidator.java:120)
	at org.eclipse.xtext.validation.AbstractDeclarativeValidator.internalValidate(AbstractDeclarativeValidator.java:314)
	at org.eclipse.xtext.validation.AbstractInjectableValidator.validate(AbstractInjectableValidator.java:71)
	at org.eclipse.xtext.validation.CompositeEValidator.validate(CompositeEValidator.java:151)
	at org.eclipse.emf.ecore.util.Diagnostician.doValidate(Diagnostician.java:257)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:244)
	at org.eclipse.xtext.validation.CancelableDiagnostician.validate(CancelableDiagnostician.java:40)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:201)
	at org.eclipse.emf.ecore.util.Diagnostician.validate(Diagnostician.java:143)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:146)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:124)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:90)
	at org.eclipse.xtext.ui.validation.DefaultResourceUIValidatorExtension.addMarkers(DefaultResourceUIValidatorExtension.java:60)
	at org.eclipse.xtext.ui.validation.DefaultResourceUIValidatorExtension.updateValidationMarkers(DefaultResourceUIValidatorExtension.java:46)
	at org.eclipse.xtext.builder.builderState.MarkerUpdaterImpl.processDelta(MarkerUpdaterImpl.java:93)
	at org.eclipse.xtext.builder.builderState.MarkerUpdaterImpl.updateMarkers(MarkerUpdaterImpl.java:63)
	at org.eclipse.xtext.builder.builderState.AbstractBuilderState.updateMarkers(AbstractBuilderState.java:82)
	at org.eclipse.xtext.builder.clustering.ClusteringBuilderState.doUpdate(ClusteringBuilderState.java:284)
	at org.eclipse.xtext.builder.builderState.AbstractBuilderState.update(AbstractBuilderState.java:116)
	at org.eclipse.xtext.builder.impl.XtextBuilder.doBuild(XtextBuilder.java:357)
	at org.eclipse.xtext.builder.impl.XtextBuilder.fullBuild(XtextBuilder.java:392)
	at org.eclipse.xtext.builder.impl.XtextBuilder.build(XtextBuilder.java:225)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:833)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:220)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:263)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:316)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:319)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:371)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:392)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Comment 15 Christian Dietrich CLA 2019-08-17 10:22:13 EDT
reopening
Comment 16 Christian Dietrich CLA 2019-08-17 11:12:34 EDT
@Titouan Vervack
I tried to reproduce this using your description but i could not.
Comment 17 Martin Steinbichler CLA 2020-06-23 05:45:10 EDT
Created attachment 283385 [details]
Test Implementation
Comment 18 Martin Steinbichler CLA 2020-06-23 05:48:28 EDT
Dear Xtext community!

I'm also facing the problem described in this bug.
Therefore I tried and managed to reproduce it with a minimum implementation based on my actual problem. You can find it attached to this post (Implementation and Runtime workspace without .metadata directories because of their sizes)

The model consists of three classes:
- Document (root doc in the grammar, container for Specifications and SubSpecifications)
- Specification (may reference a SubSpecification)
- SubSpecification


I implemented a validation that checks, that a SubSpecification may only be referenced once by a Specification. 
In this validation I am using a method "findReferencesTo" which uses the index to find all ResourceDescriptions and there ReferenceDescritpions to find any references to the given SubSpecification.

In a continuous build this Validation works. However if I trigger a full build using Project - Clean - Clean all Projects the error occurs.
In this case the ResourceDescriptions are of instance CopiedResourceDescription and therefore the call of getReferenceDescriptions throws the error.

I hope this helps and would be greatful for any advice on this issue.
Currently this means for me, that I cannot use ReferenceDescriptions in Validations at all.

Many Thanks and Regards,
Martin
Comment 19 Christian Dietrich CLA 2020-06-23 06:09:44 EDT
the reference desciptions are updated one after another it depends on which resource descriptions you call getReferenceDescriptions on

correct @szarnekow?
Comment 20 Sebastian Zarnekow CLA 2020-06-23 06:12:38 EDT
Indeed. Generally speaking, it is not possible to access reference descriptions from a validation and each attempt is prone to fail in one or the other situation.
Comment 21 Elie Richa CLA 2020-06-23 06:45:52 EDT
(In reply to Sebastian Zarnekow from comment #20)
> Indeed. Generally speaking, it is not possible to access reference
> descriptions from a validation and each attempt is prone to fail in one or
> the other situation.

Almost a year after I initially encountered this issue, I think I finally understand that there's no ultimate fix to this :)

Does it mean that validation should never rely on reference descriptions?

If yes, could this be documented somewhere since we are several people to attempt the same thing? I'd be happy to help if pointed to a good place to mention this.

And finally, CopiedResourceDescription.getReferenceDescriptions() currently doesn't throw an exception. Instead it simply logs the error and returns an empty list. So in my case the error went unnoticed for a good while before I finally noticed the errors in the Error Log.

Why not throw an exception to let callers decide on a course of action?
Comment 22 Martin Steinbichler CLA 2020-06-23 07:00:59 EDT
Thank you for the quick responses!

While debugging this issue I stumbled upon ClusteringBuilderState.doUpdate(..). What surprises me, there is a section commented with "// Resolve links here" and only later updateMarkers(..) is called to execute the validations.

So I guess my assumption, that within the "Resolve links" section the ReferenceDescriptions are instantiated, is wrong?

Do you have any ideas how to implement validations that require to find references?
Comment 23 Christian Dietrich CLA 2020-06-23 07:02:17 EDT
the question is: which resource description do you call the method on
Comment 24 Sebastian Zarnekow CLA 2020-06-23 07:06:48 EDT
The comments in the source are correct. But we are still in the same loop iteration. When you start the validation of the first resource, all the other resources are not yet build and their up-to-date reference descriptions are not yet available.
Comment 25 Martin Steinbichler CLA 2020-06-23 07:13:13 EDT
Is there a reason why this has to be done that way?
Shouldn't it be possible to first fully create the index including ReferenceDescriptions and only after that call user implemented validations?
Comment 26 Sebastian Zarnekow CLA 2020-06-23 09:47:27 EDT
Yes, there is a reason why it was done like that:

Larger projects may (and regularly do) run into the situation where not all resources fit into memory. Therefore the ClusteringBuilder observes the available memory and may decide to clear the resource set. In other words: There is no guarantee, that all resources fit into memory.

Now if this happens, there is already a lot of pressure on the memory and also on the performance. Iff we'd first link all resources and subsequently validate all resources, the linking would effectively happen multiple times. In order to avoid this, the loop was implemented as is.

Yes, it is true that the generator will subsequently run into the same issue for large projects. Therefore I often see the generation step being incorporated into the validation step of the build to avoid unnecessary additional resource loading.