Community
Participate
Working Groups
Build Identifier: The static PURGE_TRESHOLD set to 10000 is too high in our case. This leads to very infrequently and takes a long time to purge (>20min) This leads to all the threads that needs to assign IDs to wait till the lock is released and brings the whole application to halt during the purge operation. a may be useful insight informations would be, this particular application has large data objects and mostly reads data but rarely creates new ids. Attached jstack dump illustrates the system lock down during purge, notice the lock with the id 0x00002aaac27e77c8 Reproducible: Always Steps to Reproduce: I think, lowering or preferable making PURGE_THRESHOLD configurable should be cure to the solution. Also it seems like a teneo architect should be well aware that configuration to tune good his/her system for performance.
Please ignore my previous assessment. Due to the nature of the application we have we need large cache and therefore we run java with memories 8G 12G. While this proves to be very good for our case, it gives very large memory for Id and version cache. If no garbage collection needed for a long time, then id and version cache grows significantly. This results in purge taking too long to finish, therefore blocking hibernate bound threads for a long time. I think a conventional cache usage would be appropriate solution here. In addition to this I do not know the ways the id and version cache used in teneo, but we use TreeCache in our application. TreeCache is distributed and transactional. I think in a transactional system a non transactional cache (id & version cache) can pose problems which would be hard to track down. Regards, Hasan Ceylan
Hi Hasan, Some quick remarks: - the IdentifierCacheHandler is only really used if an object does not have its own id/version efeature. This is not adviced anyway for production environments. - you can set your own implementation of the IdentifierCacheHandler (see the setInstance method) Especially because the IdentifierCacheHandler is only really used for synthetic id/version I am not sure that it makes much sense to spend a lot of time changing it to another cache provider. Your remark about the purge treshold as a configurable setting makes sense for sure though. gr. Martin
Hello Martin, Thanks for fast handling as always. :) I was informed about this problem two days ago and frankly haven't had much time to throughly understand what IdentifierCache does and does not. So bear with that my comments are based on very limited knowledge. I initially modified teneo and modified the value from 10000 down to 100. I will have that version deployed to the production. Based on the fact that you agree with my first comment, this should work right? I'll inform of the outcome. All the best, Hasan
Yes that should work fine afaics. You can even replace the IdentifierCacheHandler with your own implementation which does not cache anything. This should work fine if all your objects have an explicit id/version efeature. gr. Martin
Martin, Bringing the value for PURGE_THRESHOLD to 100 seems to smoothen things out. FYI... Hasan
Ok, I changed this, will be in the next Teneo build
(In reply to comment #6) > Ok, I changed this, will be in the next Teneo build Hi Martin, After Hasan changed PURGE_TRESHOLD value to 100, the production didnt have any issue about performance for 2 months. Unfortunately in these days we come up with performance problems again. I profiled the application with jconsole. The threads in waiting state are increasing and application is running slow. What should we do for this issue? Maybe you can advice us, below is the part of thread dump. sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114) java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) java.util.concurrent.ConcurrentHashMap$Segment.remove(ConcurrentHashMap.java:515) java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:923) java.util.concurrent.ConcurrentHashMap$HashIterator.remove(ConcurrentHashMap.java:1108) org.eclipse.emf.teneo.hibernate.mapping.identifier.IdentifierCacheHandler.purgeMap(IdentifierCacheHandler.java:183) org.eclipse.emf.teneo.hibernate.mapping.identifier.IdentifierCacheHandler.purgeIDMap(IdentifierCacheHandler.java:167) org.eclipse.emf.teneo.hibernate.mapping.identifier.IdentifierCacheHandler.setID(IdentifierCacheHandler.java:122) org.eclipse.emf.teneo.hibernate.tuplizer.EMFTuplizer.setIdentifier(EMFTuplizer.java:157) org.hibernate.tuple.entity.AbstractEntityTuplizer.instantiate(AbstractEntityTuplizer.java:368) org.hibernate.persister.entity.AbstractEntityPersister.instantiate(AbstractEntityPersister.java:3620) org.hibernate.impl.SessionImpl.instantiate(SessionImpl.java:1303) org.hibernate.impl.SessionImpl.instantiate(SessionImpl.java:1292) org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1323) org.hibernate.loader.Loader.getRow(Loader.java:1230) org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:603) org.hibernate.loader.Loader.doQuery(Loader.java:724) org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:259) org.hibernate.loader.Loader.doList(Loader.java:2232) org.hibernate.loader.Loader.listUsingQueryCache(Loader.java:2161) org.hibernate.loader.Loader.list(Loader.java:2121) org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:118) org.hibernate.impl.SessionImpl.list(SessionImpl.java:1597) org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:306) org.hibernate.impl.CriteriaImpl.uniqueResult(CriteriaImpl.java:328) Thanks for all.. Regards, Ercan Canlıer
Hi Ercan, It is difficult to comment on this. I am surprised that something like a pure java action can actually lock a system. With me it always have been database locking issues. Have you tried even higher values of the purge treshold? How big does the cache get? You can also provide your own implementation of the IdentifierCacheHandler, it has a static set method to set your own instance in it. gr. Martin
Hi Martin, Thanks for fast reply. I dont know much about what this stuff really does. What i see while profiling the application with jconsole within top threads, their states, thread dumps is, the application is running slow after some time and threads are becoming to WAITING state. I will give it chance with higher values and try to be sure the problem is really teneo. Regards, Ercan Canlier
