Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 353246

Summary: [DB] Duplicate entry / Violation of unique index in 'cdo_package_units_idx0'
Product: [Modeling] EMF Reporter: kaab <kaabab>
Component: cdo.dbAssignee: Eike Stepper <stepper>
Status: CLOSED FIXED QA Contact: Eike Stepper <stepper>
Severity: normal    
Priority: P3 CC: picard, saulius.tvarijonas, stefan, stepper
Version: 4.1   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Whiteboard:
Bug Depends on:    
Bug Blocks: 353448    
Attachments:
Description Flags
Test case to reproduce the error
none
Test case as patch (easier to apply)
none
Fix
none
Fix v2 none

Description kaab CLA 2011-07-27 14:42:39 EDT 
Build Identifier: I20110613-1736

+ When we try to create multiple resources concurrently we get an exception that seems directly related to the store we use.
+ We use a tree locking strategy to avoid conflicts happening at the resource creation level.
+ I have been able to reproduce this exception outside of our environement using cdo testing framework in the unit test provided.
+ The issue seem to be somehow related to timing
+ Issue was tested with Mysql Store and Hsql Db Store and should be reproducible with both (exception differs slightly but seem to refer to the same problem)

Reproducible: Always

Steps to Reproduce:
1. Run the unit test with a Mysql or HsqlDb store
2. You should see the exception appear in the console
-> For Mysql :
[ERROR] com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'http://www.eclipse.org/emf/CDO/tests/model1/1.0.0' for key 'cdo_package_units_idx0'
org.eclipse.net4j.db.DBException: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'http://www.eclipse.org/emf/CDO/tests/model1/1.0.0' for key 'cdo_package_units_idx0'
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.fillSystemTables(MetaDataManager.java:263)
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.fillSystemTables(MetaDataManager.java:292)
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.writePackageUnits(MetaDataManager.java:138)
	at org.eclipse.emf.cdo.server.internal.db.DBStoreAccessor.writePackageUnits(DBStoreAccessor.java:819)
	at org.eclipse.emf.cdo.spi.server.StoreAccessor.doWrite(StoreAccessor.java:76)
	at org.eclipse.emf.cdo.spi.server.StoreAccessorBase.write(StoreAccessorBase.java:149)
	at org.eclipse.emf.cdo.internal.server.TransactionCommitContext.write(TransactionCommitContext.java:425)
	at org.eclipse.emf.cdo.spi.server.InternalCommitContext$1.runLoop(InternalCommitContext.java:42)
	at org.eclipse.emf.cdo.spi.server.InternalCommitContext$1.runLoop(InternalCommitContext.java:1)
	at org.eclipse.net4j.util.om.monitor.ProgressDistributor.run(ProgressDistributor.java:96)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CommitTransactionIndication.indicatingCommit(CommitTransactionIndication.java:244)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CommitTransactionIndication.indicating(CommitTransactionIndication.java:92)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CDOServerIndicationWithMonitoring.indicating(CDOServerIndicationWithMonitoring.java:109)
	at org.eclipse.net4j.signal.IndicationWithMonitoring.indicating(IndicationWithMonitoring.java:84)
	at org.eclipse.net4j.signal.IndicationWithResponse.doExtendedInput(IndicationWithResponse.java:90)
	at org.eclipse.net4j.signal.Signal.doInput(Signal.java:326)
	at org.eclipse.net4j.signal.IndicationWithResponse.execute(IndicationWithResponse.java:63)
	at org.eclipse.net4j.signal.IndicationWithMonitoring.execute(IndicationWithMonitoring.java:63)
	at org.eclipse.net4j.signal.Signal.runSync(Signal.java:251)
	at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
->For HSQLDB :
[ERROR] java.sql.SQLException: Violation of unique index CDO_PACKAGE_UNITS_IDX0: duplicate value(s) for column(s) ID in statement [INSERT INTO cdo_package_units VALUES (?, ?, ?, ?)]
org.eclipse.net4j.db.DBException: java.sql.SQLException: Violation of unique index CDO_PACKAGE_UNITS_IDX0: duplicate value(s) for column(s) ID in statement [INSERT INTO cdo_package_units VALUES (?, ?, ?, ?)]
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.fillSystemTables(MetaDataManager.java:263)
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.fillSystemTables(MetaDataManager.java:292)
	at org.eclipse.emf.cdo.server.internal.db.MetaDataManager.writePackageUnits(MetaDataManager.java:138)
	at org.eclipse.emf.cdo.server.internal.db.DBStoreAccessor.writePackageUnits(DBStoreAccessor.java:819)
	at org.eclipse.emf.cdo.spi.server.StoreAccessor.doWrite(StoreAccessor.java:76)
	at org.eclipse.emf.cdo.spi.server.StoreAccessorBase.write(StoreAccessorBase.java:149)
	at org.eclipse.emf.cdo.internal.server.TransactionCommitContext.write(TransactionCommitContext.java:425)
	at org.eclipse.emf.cdo.spi.server.InternalCommitContext$1.runLoop(InternalCommitContext.java:42)
	at org.eclipse.emf.cdo.spi.server.InternalCommitContext$1.runLoop(InternalCommitContext.java:1)
	at org.eclipse.net4j.util.om.monitor.ProgressDistributor.run(ProgressDistributor.java:96)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CommitTransactionIndication.indicatingCommit(CommitTransactionIndication.java:244)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CommitTransactionIndication.indicating(CommitTransactionIndication.java:92)
	at org.eclipse.emf.cdo.server.internal.net4j.protocol.CDOServerIndicationWithMonitoring.indicating(CDOServerIndicationWithMonitoring.java:109)
	at org.eclipse.net4j.signal.IndicationWithMonitoring.indicating(IndicationWithMonitoring.java:84)
	at org.eclipse.net4j.signal.IndicationWithResponse.doExtendedInput(IndicationWithResponse.java:90)
	at org.eclipse.net4j.signal.Signal.doInput(Signal.java:326)
	at org.eclipse.net4j.signal.IndicationWithResponse.execute(IndicationWithResponse.java:63)
	at org.eclipse.net4j.signal.IndicationWithMonitoring.execute(IndicationWithMonitoring.java:63)
	at org.eclipse.net4j.signal.Signal.runSync(Signal.java:251)
	at org.eclipse.net4j.signal.Signal.run(Signal.java:147)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Comment 1 kaab CLA 2011-07-27 14:44:22 EDT 
Created attachment 200465 [details]
Test case to reproduce the error

Test Case to reproduce the issue
Comment 2 Eike Stepper CLA 2011-07-28 00:33:52 EDT 
We'll analyze this in 4.1 first and then eventually port thr fix to maintenance.

Stefan, is this issue a result of *some* SQL DBs not properly executing DDL in the transaction context?

Abdel, starting with CDO 4.0 you can nicely work around this issue by using the new method IRepository.setInitialPackages(EPackage...), which must be called *before te repository is started for the first time* .
Comment 3 Stefan Winkler CLA 2011-07-28 07:37:12 EDT 
(In reply to comment #2)
> Stefan, is this issue a result of *some* SQL DBs not properly executing DDL in
> the transaction context?
Maybe. I cannot say for sure.

Another possibility could be that the server does not notice that it receives the same package multiple times and it regards the same package as new in two different threads.
But I assume this mechanism is generally thread safe?
Comment 4 Eike Stepper CLA 2011-07-28 07:53:34 EDT 
(In reply to comment #3)
> Another possibility could be that the server does not notice that it receives
> the same package multiple times and it regards the same package as new in two
> different threads.
> But I assume this mechanism is generally thread safe?

That does ring a bell. I think we don't lock meta instances at all! Probably because they are always immutable and I did not consider that two commits with any new packages should compete for exclusive package registry access.

Assigning to myself...
Comment 5 Eike Stepper CLA 2011-08-03 07:18:16 EDT 
Created attachment 200797 [details]
Test case as patch (easier to apply)

BTW. the test case does pass for me.
Comment 6 kaab CLA 2011-08-03 11:38:10 EDT 
(In reply to comment #5)
> Created attachment 200797 [details]
> Test case as patch (easier to apply)
> 
> BTW. the test case does pass for me.

The test does pass but produces an exception, reproducing the exception was the objective behind the test, I just wanted to provide some fast way to reproduce this in CDO environment without going trough all the configuration needed for a CDO store, so I used the testing framework. Just to make sure, I've just re-run this test on a different machine and the error was still reproducible.
Comment 7 Eike Stepper CLA 2011-08-04 01:49:26 EDT 
Created attachment 200873 [details]
Fix
Comment 8 Eike Stepper CLA 2011-08-04 01:56:12 EDT 
Created attachment 200874 [details]
Fix v2
Comment 9 Eike Stepper CLA 2011-08-04 01:57:03 EDT 
Committed revision 8856:
- trunk/plugins/org.eclipse.emf.cdo.common
- trunk/plugins/org.eclipse.emf.cdo.server
- trunk/plugins/org.eclipse.emf.cdo.tests
Comment 10 Eike Stepper CLA 2011-08-04 01:57:44 EDT 
Resolved. 

Tested in PackageRegistryTest.testConcurrentPackageRegistration2()
Comment 11 Eike Stepper CLA 2012-09-21 07:17:23 EDT 
Closing.