Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 365826

Summary: Object revision not in sync with DB, but CLEAN and valid.
Product: [Modeling] EMF Reporter: Christophe Bouhier <dzonekl>
Component: cdo.coreAssignee: Project Inbox <emf.cdo-inbox>
Status: CLOSED WORKSFORME QA Contact:
Severity: normal    
Priority: P3 CC: stepper
Version: 4.0   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X - Carbon (unsup.)   
Whiteboard:
Attachments:
Description Flags
Application architecture none

Description Christophe Bouhier CLA 2011-12-07 00:59:56 EST
We have a problem with objects presenting themselves as valid while the DB shows a later revision. it seems, we sometimes do not work with the latest revision of an object which leads to problems accessing collections from that object. Typically the collection index is not correct and this leads to duplicate key violations on the DB. We have this issue on the following combinations: 


HibernateStore -> Postgresql
DBStore -> Postgresql
DBStore -> MySql


Some typical consequence of this: 

org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "metrics_metricsource_statistics_list_idx0"

java.sql.BatchUpdateException: Batch entry 0 INSERT INTO metrics_MetricSource_statistics_list (cdo_source, cdo_idx, cdo_value) VALUES('254', '1', '4575') was aborted. Call getNextException to see the cause.

Theory: 

I would like to know if a revision of an object which is not the latest (As we see in the DB), can indeed cause this and is intentional (by design). Next, how do I make sure, I have the latest revision of an object? Can I force an invalidation on the client side ? When the object is state CLEAN but not the latest revision? 
(Reloading doesn't help). Interresting enough we can see a later revision by creating a new CDOID from the object identifier like this: 

CDOID cdoID = ms.cdoID();
long longValue = ((AbstractCDOIDLong) cdoID).getLongValue();
CDOID createLongWithClassifier = CDOIDUtil.createLongWithClassifier(new CDOClassifierRef(
								MetricsPackage.Literals.METRIC_SOURCE), longValue);
CDOObject object = transaction.getObject(createLongWithClassifier);
System.out.println(" reloaded object =" +  object.cdoRevision().getVersion() + " timestamp=" + new Date(object.cdoRevision().getTimeStamp()));


How to reproduce:

Our app access CDO in several (parallel) ways, there is a UI client which has change subscription turned on. 

There are background jobs, which are scheduled and access CDO. There are server side jobs triggered with an HTTP request which contains the object ID. (See the architecture picture). The problem often occurs when triggering a manual job through HTTP which then accesses CDO. Commits from this job don't seem to invalidate the object on the server and send notifications to other sessions. The next time the UI access the same object, we get the problem described above.

Obviously, this is not a very reproduceable situation. To reproduce in a unit test the activity would be: 

Thread 1: Open a session and access resource A, and collection C from object A
Thread 2: Open a JVM session and access Object A with getObject(),  and collection C from object A, add one entry.  (The object is access from the longValue of the CDOID as it is received in the HTTP URL). 
---- invalidation doesn't happen. 
Thread 3: Scheduled job, opens a JVM session and access Object A with reference, problem occurs, not latest revision. 
Thread 1: access collection C from object A, not latest revision collection not fully visible.
Comment 1 Christophe Bouhier CLA 2011-12-07 01:02:11 EST
Created attachment 208036 [details]
Application architecture

Application architecture
Comment 2 Eike Stepper CLA 2011-12-10 10:22:55 EST
I think that this type of situation can normally occur unless you set one or more read locks for "consistent reads". In addition the repo must be configured with ensureReferentialIntegrity=true to prevent (unlocked) target objects from being deleted.

That said, I'd like to investigate whether we could/should throw better a exception...
Comment 3 Eike Stepper CLA 2011-12-15 13:23:09 EST
Christophe, do you agree with my arguments in comment 2?

Then I'd like to close this as WROKSFORME, unless you rename it and ask for a better exception ;-)
Comment 4 Christophe Bouhier CLA 2011-12-15 15:05:51 EST
Eike, 

I would need to try it out, also too much on my plate right now. 
please close it. I will get back if needed. 



(In reply to comment #3)
> Christophe, do you agree with my arguments in comment 2?
> 
> Then I'd like to close this as WROKSFORME, unless you rename it and ask for a
> better exception ;-)
Comment 5 Eike Stepper CLA 2011-12-16 01:26:29 EST
Okay, please feel free to reopen later ;-)