Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 318343

Summary: ClassCastException in Xtext validation and XtextReconcilerJob when removing a datatype
Product: [Modeling] TMF Reporter: Lothar Wendehals <lothar.wendehals>
Component: XtextAssignee: Project Inbox <tmf.xtext-inbox>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: chanskw, floplateau, sebastian.zarnekow, sven.efftinge
Version: unspecifiedFlags: sebastian.zarnekow: helios+
Target Milestone: SR1   
Hardware: PC   
OS: Windows Vista   
Whiteboard: v2.9
Attachments:
Description Flags
Patch to solve the problem sven.efftinge: iplog+

Description Lothar Wendehals CLA 2010-06-29 11:30:04 EDT
Build Identifier: I20100608-0911

When removing the first two datatypes of the given model, a ClassCastException is thrown twice by the Xtext validation and by the XtextReconcilerJob.

Reproducible: Always

Steps to Reproduce:
1. Use this grammar:
grammar xtext.workshop.basic.Entities with org.eclipse.xtext.common.Terminals

generate entities "http://www.workshop.xtext/basic/Entities"

Model:
	(entities+=Type)*;

Type:
	Datatype|Entity;

Datatype:
	'datatype' name=ID;

Entity:
	'entity' name=ID ('extends' superType=[Entity])? '{'
		(properties+=Property)*
	'}';

Property:
	SimpleProperty | ReferenceProperty;

SimpleProperty:
	(visibility=[Visibility])? 'property' name=ID ':' type=[Datatype];

ReferenceProperty:
	name=ID 'refs' (many?='many')? type=[Entity];
	
enum Visibility:
	PUBLIC = 'public' | PROTECTED = 'protected' | PRIVATE = 'private';

2. Use this model:
datatype String
datatype Bool

entity Conference {
		property Name : String
		Attendees refs many Person
		Speakers refs many Speaker
	}
entity Person {
		property Name : String
	}
entity Speaker extends Person {
		Sessions refs many Session
	}
entity Session {
		property Title: String
		property IsTutorial : Bool
	}
3. Cut the first two lines.
Comment 1 Lothar Wendehals CLA 2010-06-29 11:40:58 EDT
Stacktrace:
java.lang.ClassCastException: xtext.workshop.basic.entities.impl.EntityImpl cannot be cast to xtext.workshop.basic.entities.Datatype
	at xtext.workshop.basic.entities.impl.SimplePropertyImpl.getType(SimplePropertyImpl.java:121)
	at xtext.workshop.basic.entities.impl.SimplePropertyImpl.eGet(SimplePropertyImpl.java:167)
	at org.eclipse.emf.ecore.impl.BasicEObjectImpl.eGet(BasicEObjectImpl.java:1021)
	at org.eclipse.emf.ecore.impl.BasicEObjectImpl.eGet(BasicEObjectImpl.java:1013)
	at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.hasNext(EContentsEList.java:409)
	at org.eclipse.xtext.EcoreUtil2.resolveCrossReferences(EcoreUtil2.java:485)
	at org.eclipse.xtext.EcoreUtil2.resolveAll(EcoreUtil2.java:469)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.resolveProxies(ResourceValidatorImpl.java:122)
	at org.eclipse.xtext.validation.ResourceValidatorImpl.validate(ResourceValidatorImpl.java:60)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob$1.exec(ValidationJob.java:76)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob$1.exec(ValidationJob.java:1)
	at org.eclipse.xtext.util.concurrent.IStateAccess$AbstractImpl.readOnly(IStateAccess.java:40)
	at org.eclipse.xtext.ui.editor.model.XtextDocument.readOnly(XtextDocument.java:58)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob.createIssues(ValidationJob.java:74)
	at org.eclipse.xtext.ui.editor.validation.ValidationJob.run(ValidationJob.java:63)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Comment 2 Holger Schill CLA 2010-07-30 10:35:20 EDT
Created attachment 175581 [details]
Patch to solve the problem

I created a patch to solve the problem. I introduced a org.eclipse.xtext.resource.containers.FilterUriContainer that wraps an IContainer to filter out the IResourceDescriptions that have the same URI as the Resource of an given contextobject. This FilterUriContainer is used in the DefaultGlobalScopeProvider. I also implemented a test for this scenario. 
With this patch some other validation issues caused by wrong resolution of proxies are solved.
Comment 3 Sebastian Zarnekow CLA 2010-07-30 10:50:51 EDT
I removed some whitespace-diffs from the patch and applied it. Thanks, Holger.
Comment 4 Samantha Chan CLA 2010-09-30 23:26:48 EDT
I experienced the same problem in my editor.  I have my own global scope provider which extends from the DefaultGlobalScopeProvider.  I can resolve the problem, by making sure that I call the super.getScope() method to take advantage of this fox.  However, I'd like to understand what the issue is... and what this fix is about.  Can you briefly explain how we get wrong resolution to proxies and why this filtering mechanism will work?
Comment 5 Sebastian Zarnekow CLA 2010-10-03 12:24:20 EDT
Xtext keeps an index for any eobject with the current URI of the object. The default URI mechanism relies on list indices for multi value features. If a list may hold data types and entities you may get an entity even if the Xtext index contains a data type for that reference. Therefore we'll have to ignore the index data for the current resource because the resource information is correct while the index information is outdated. 

Does this help?
Comment 6 Samantha Chan CLA 2010-10-05 17:32:18 EDT
Yes, does.  

I made sure that my global scope provider calls super.getScope() and made sure that FilterUriContainer is created.

However, I am still running into this CassCastException.

My problem relates to multiple resources:

Resource 1 references to an element from Resources 2.
I opened the editor for Resource 1.. and then for Resource 2.
From resource 2, I changed the content of the file, invalidating the URI of the element referenced by Resource 1.

And I got into the same exception.  Do you expect that that this scenario will work with this fix here?
Comment 7 Flo Plateau CLA 2015-05-15 11:21:02 EDT
Dear Xtext team,

this ClassCastException also occurs really often in my DSL editor (and also  "java.lang.IllegalArgumentException: The feature 'foo' is not a valid feature"), as Samantha Chan in presence of several resources. 

I open side by side an editor with a resource1 defining some elements, for instance `Entitie`s, and a resource2 referencing those elements. Then I paste a new element in resource1 for instance a `Datatype`, and I save. The exception occurs when I respect a certain timing between the paste and the save (not to slow, not too fast). It is raised sometimes during the ValidationJob, sometimes by MarkOccurrence. What version of the index is used by concurrent jobs as ValidatorJob between the moment where I save (and thus the editor is no more dirty) and the moment where the build has computed the new index ?
Comment 8 Sebastian Zarnekow CLA 2015-05-15 12:32:25 EDT
This is probably a dup of bug 464483
Comment 9 Flo Plateau CLA 2015-05-15 14:43:40 EDT
Thank you for your quick answer. 

Indeed, after posting my comment, I had managed to reproduce the problem  on Xtext "Simple Arithmetic Example", and my reproduction procedure doesn't trigger the exception on an Xtext 2.8.2 on which I have applied the patch of the bug you mention. Good news :-).
Comment 10 Sebastian Zarnekow CLA 2015-05-16 07:57:24 EDT
Thanks for verifying this.
Comment 11 Flo Plateau CLA 2015-05-21 05:00:24 EDT
I have migrated my DSL plugin to Xtext 2.8.2 on which I applied the patch of the bug you mention (that fixed the problem of the comment 7, on the Xtext "Simple Arithmetic Example"). Bad news, I still manage to get some ClassCastExceptions and "java.lang.IllegalArgumentException: The feature 'foo' is not a valid feature".

Here is the new minimal reproduction procedure, on a unique editor this time:

I have one editor opened, containing a list of `Entitie`s. I paste a 
`Datatype` between two `Entitie`s. The exception occurs when I respect a certain timing between the paste and the save (not to slow, not too fast). It is raised by a validator (in a ValidatorJob) that uses an injected ResourceDescriptionsProvider to get the IResourceDescription of the validated `resource`. I observed the execution step by step, when the bug occurs:
- the call to ResourceDescriptionsProvider.getResourceDescriptions(resource).getResourceDescription(resource.getURI()) uses
DirtyStateAwareResourceDescriptions.getResourceDescription(URI), which uses the global index because DirtyStateManager.getDirtyResource(URI) returns null.
But the global index is not yet up to date. Thus the resource description is not coherent with the resource content, which leads to the mentioned exceptions when we use the uris contained in the IResourceDescription to load EObjects of the resource.
Comment 12 Sebastian Zarnekow CLA 2015-05-21 05:21:01 EDT
Let's see what we can do about this.
Comment 13 Flo Plateau CLA 2015-05-21 08:42:25 EDT
Thanks in advance.
Comment 14 Sven Efftinge CLA 2015-05-26 02:54:34 EDT
Hi Flo,
I am not able to reproduce it. Could you please provide more detailed steps?
a) What language?
b) What is the contents of the two files?
c) What is the exact modification?
Comment 15 Sven Efftinge CLA 2015-05-27 04:49:01 EDT
Ok, I see from your description how and why that happens. It's is not urgent enough to be done in 2.8.x (due Monday) as it is a very rare scenario. 
We should consider whether we could keep all open editors in the dirty state, no matter they are synched or not. Another solution would be to use a ShadowingResourceDescriptions that asks the ResourceDesciption.Manager for all resources contained in the resourceset.
Comment 16 Flo Plateau CLA 2015-05-29 06:08:29 EDT
Thank you very much for having tracked the problem, I'm currently working on another project and haven't enough time to build a reproduction procedure on an example DSL. 

I understand well the first solution you propose, not the second one because I don't find the ShadowedResourceDescriptions class.

Note that the scenario is not rare in my context, the exception is raised several times a day, thus I'm looking forward to this bug fix :-).
Comment 17 Flo Plateau CLA 2015-05-29 06:25:04 EDT
Still, a question concerning the first solution: what if I close the editor just after saving it ? The problem will still occur between the moment where I close the editor and the moment where the build has finished the update of the global index  ?
Comment 18 Sven Efftinge CLA 2015-05-29 07:21:41 EDT
(In reply to Flo Plateau from comment #17)
> Still, a question concerning the first solution: what if I close the editor
> just after saving it ? The problem will still occur between the moment where
> I close the editor and the moment where the build has finished the update of
> the global index  ?

Yes, good point. So we probably should go with a resourcedescritpions implementation that uses the current resource's ResourceDescription that shadows the one from the index. We do it in a similar way within scoping. 
'ShadowedResourceDescriptions' doesn't exist yet :-)
Comment 19 Flo Plateau CLA 2015-07-20 11:43:45 EDT
Hi, 

I come again with bad news : not only the patch of https://github.com/eclipse/xtext/commit/3dd02f0059e9bc96ade35450dadc16884664efa4 doesn't solve the problem as I explained above, but it introduces (or reveals) another problem. Here is a reproduction procedure on the DSL "Simple arithmetic example":

1. Open in two editors side by side the following modules :

defi.calc
-------------
module defi

def pi: 3.14;
-------------

ref.calc (contains on purpose a linking error)
-------------
module ref

import defi.*

pi2 * 4;
-------------

2. in defi.calc, change "pi" to "pi2" and save. The linking error in the editor 
displaying ref.calc remains, whereas it has been fixed and none of the editors is dirty. Note that the error is correctly removed from the Problems View and the Project Explorer. 

If I suppress the above-mentioned patch, the problem doesn't occur (the linking error correctly disappears)

=============

Another procedure that can help you to identify the source of the problem:

1. Open in two editors side by side the following modules :

defi.calc
-------------
module defi

def pi: 3.14;
-------------

ref.calc (error-free)
-------------
module ref

import defi.*

pi * 4;
-------------

2. in defi.calc, change "pi" to "pi2" and save. A linking error appears in the editor displaying ref.calc, but it is grayed, which is not expected as none of the editors is dirty. Note that the error is correctly displayed from the Problems View and the Project Explorer. 

Here also, if I suppress the above-mentioned patch, the problem doesn't occur (the linking error correctly appears in red)

=============

I hope it'll help !
Comment 20 Flo Plateau CLA 2015-09-25 06:06:42 EDT
Hi Sven,

any news about this bug ? Are you confident it'll be fixed in 2.9 ? Do you have a more precise release date for 2.9 ? I'm asking all that because this bug occurs daily in my plugin, which is quite disturbing :-(.

Thanks in advance for your answers !
Flo
Comment 21 Sven Efftinge CLA 2015-10-20 06:14:22 EDT
(In reply to Flo Plateau from comment #20)
> Hi Sven,
> 
> any news about this bug ? Are you confident it'll be fixed in 2.9 ? Do you
> have a more precise release date for 2.9 ? I'm asking all that because this
> bug occurs daily in my plugin, which is quite disturbing :-(.
> 
> Thanks in advance for your answers !
> Flo

2.9 is planned for next week, which is already quite challenging.
We unfortunately won't come around to fix this for 2.9, but we plan to have some additional smaller releases in the coming weeks, where we could probably have a look at this.
Comment 22 Flo Plateau CLA 2015-10-26 13:32:50 EDT
(In reply to Sven Efftinge from comment #21)
> 2.9 is planned for next week, which is already quite challenging.
> We unfortunately won't come around to fix this for 2.9, but we plan to have
> some additional smaller releases in the coming weeks, where we could
> probably have a look at this.

OK, I'll keep a close eye on additional releases. Great thanks in advance !
Comment 23 Sven Efftinge CLA 2015-12-07 02:28:36 EST
Hi Flo,

I tried to reproduce the described behavior, but the two described situation from comment #19 worked for me. I did it with 2.9 where we have slightly changed the arithmetics language.

But I found the following:

Have the following two files saved & non-dirty:

defi.calc
-------------
module defi

def pi: 3.14;
-------------

ref.calc (error-free)
-------------
module ref

import defi.*

pi * 4;
-------------

no change def pi to def pi2 without saving.
There should be a non-persisted error annotation in the other editor, but it only comes when I save.
Comment 24 Sven Efftinge CLA 2015-12-07 02:44:51 EST
Sorry, the scenario I described in my last comment also works. It just takes a second, which made me think it doesn't work.
Returning this to fix. Please reopen with updated instructions to reproduce if it still happens for you.
Comment 25 Flo Plateau CLA 2015-12-14 04:29:38 EST
Hi Sven,

Concerning the scenario of Comment 19, I'll try to reproduce it in January 2016 (I don't have time before).

But what about the problem I describe in Comment #11, which you told me you understood (Comment #15 and Comment #18) ?
Comment 26 Flo Plateau CLA 2016-04-06 08:41:41 EDT
Hi Sven, 
I checked on Xtext 2.9.2: indeed, scenario of Comment 19 no more produces the described problem.

Still, the problem described in Comment 11 is still present :
DirtyStateAwareResourceDescriptions.getResourceDescription(URI) uses the global index when a resource is not dirty. But if the resource has just been saved, the resource is not dirty and the global index may not yet be up to date. Thus the returned resource description is not coherent with the resource content, which leads to the mentioned exceptions when we use the uris contained in the IResourceDescription to load EObjects of the resource.

I also get some ClassCastExceptions because of outdated uris in another contexts which have nothing to do with resource descriptions: Let 2 files "file1" and "file2", with file2 that references file1. I modify file1, changing the uri of the element referenced in file2. Then, sometimes, file2's proxy uris to file1 are not updated before the ValidationJob/OutlineRefreshJob/MarkOccurrenceJob are launched on it. If one of these jobs needs to get the referenced element in file1, then the ClassCastException occurs as the proxy uri of the reference is outdated.
Comment 27 Sven Efftinge CLA 2016-04-06 08:51:15 EDT
Could you open two distinct tickets for the outstanding issues you described?
Comment 29 Eclipse Webmaster CLA 2017-10-31 11:07:01 EDT
Requested via bug 522520.

-M.
Comment 30 Eclipse Webmaster CLA 2017-10-31 11:18:24 EDT
Requested via bug 522520.

-M.