Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 337397

Summary: Improve performance of CDORevisionCacheImpl.getObjectType(CDOID)
Product: [Modeling] EMF Reporter: Gábor Nagy <gnagy>
Component: cdo.coreAssignee: Eike Stepper <stepper>
Status: CLOSED FIXED QA Contact: Eike Stepper <stepper>
Severity: enhancement    
Priority: P3 CC: saulius.tvarijonas, stepper, vaisegid, vroldanbet
Version: 4.0Flags: stepper: review+
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Attachments:
Description Flags
Proposed patch for the performance issue.
stepper: iplog+
patch v2 - with type map self decreasing
none
Patch v3 - ready to be committed none

Description Gábor Nagy CLA 2011-02-17 03:21:38 EST
Build Identifier: 

While profiling our workload of loading lots of CDOObjects from a CDOResource, we noticed a performance issue in CDORevisionCacheImpl.getObjectType(CDOID), where linear searching happens on an increasingly large collection.

The proposed patch utilises a separate HashMap<CDOID, EClass> to speed up these lookups. 

Reproducible: Always
Comment 1 Gábor Nagy CLA 2011-02-17 03:22:15 EST
Created attachment 189156 [details]
Proposed patch for the performance issue.
Comment 2 Eike Stepper CLA 2011-02-22 11:07:30 EST
Thank you for your contribution. It looks good and I'd be happy to commit it to the 4.0 stream. Please confirm that:

1) The number of lines that you changed is smaller than 250.
2) You are the only author of these changed lines.
3) You apply the EPL to these changed lines.
Comment 3 Eike Stepper CLA 2011-02-22 15:16:51 EST
(In reply to comment #2)
> It looks good and I'd be happy to commit it to the 4.0 stream. 

Looking at it again I fear that your solution does not scale well. The new map will grow with the numbers of objects that are loaded and it will never shrink. This seems to be acceptable if the set of objects that are loaded is rather small and does not changed that much over time.
Comment 4 Gábor Nagy CLA 2011-02-23 03:41:36 EST
1) The number of lines that you changed is smaller than 250.
The patch is 64 lines, so...
2) You are the only author of these changed lines.
Yes, I am the only author.
3) You apply the EPL to these changed lines.
Yes, I apply the EPL.
Comment 5 Gábor Nagy CLA 2011-02-23 03:57:09 EST
(In reply to comment #3)
> (In reply to comment #2)
> > It looks good and I'd be happy to commit it to the 4.0 stream. 
> 
> Looking at it again I fear that your solution does not scale well. The new map
> will grow with the numbers of objects that are loaded and it will never shrink.
> This seems to be acceptable if the set of objects that are loaded is rather
> small and does not changed that much over time.

I see your point. I can think of three possible solutions:

- add a typeMap.remove(id) to removeRevision()

- use an LRU Map with a maximum size (see LinkedHashMap.removeEldestEntry())

- use a WeakHashMap and let the garbage collector manage size.
Comment 6 Eike Stepper CLA 2011-02-23 05:12:45 EST
(In reply to comment #5)
> - add a typeMap.remove(id) to removeRevision()

That does not sound good. That would remove the cached mapping with each object change.

> - use a WeakHashMap and let the garbage collector manage size.

That would only work if each logical ID has only ever one CDOID instance associated. Not sure if we can guarantee that.

> - use an LRU Map with a maximum size (see LinkedHashMap.removeEldestEntry())

That sounds good ;-)
Comment 7 Egidijus Vaisnora CLA 2011-02-26 04:13:01 EST
LRU map is good idea but we will need to answer what boundary to set, how to change it and support implementation (we don't have LRU map currently under CDO). Idea is to bound current <CDOID, EClass> map with already existing cache and its eviction policy.
How about to have Map<CDOID, Pair<EClass,Integer>>. After chat with Eike, I think it could be good solution (actually it was Eike idea :) ). We could keep track how many revisions are present for particular CDOID in the cache with "Integer". Tracking could be done in a methods addRevision or removeRevision. I suppose that removeRevision is always called on eviction from the cache but I could be wrong. Whenever "Integer == 0", we clear CDOID from the map.
Comment 8 Eike Stepper CLA 2011-02-26 04:36:07 EST
(In reply to comment #7)
> How about to have Map<CDOID, Pair<EClass,Integer>>. 

In principle okay, but I'd suggest to create a special value class TypeAndRefCounter.

> After chat with Eike, I
> think it could be good solution (actually it was Eike idea :) ). We could keep
> track how many revisions are present for particular CDOID in the cache with
> "Integer". Tracking could be done in a methods addRevision or removeRevision. I
> suppose that removeRevision is always called on eviction from the cache 

Yes, I think so.
Comment 9 Egidijus Vaisnora CLA 2011-02-26 05:40:46 EST
Created attachment 189880 [details]
patch v2 - with type map self decreasing
Comment 10 Eike Stepper CLA 2011-03-01 01:18:06 EST
Created attachment 190019 [details]
Patch v3 - ready to be committed

There was a bug in TypeAndRefCounter.decrease(). You used counter-- but it must be --counter.
Comment 11 Egidijus Vaisnora CLA 2011-03-01 03:01:58 EST
Committed to trunk, revision 7311
Comment 12 Eike Stepper CLA 2011-06-23 03:41:37 EDT
Available in R20110608-1407