Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 329179

Summary: Make cdoPrefetch() robust against timeouts
Product: [Modeling] EMF Reporter: David Hein <info>
Component: cdo.coreAssignee: Eike Stepper <stepper>
Status: RESOLVED FIXED QA Contact: Eike Stepper <stepper>
Severity: enhancement    
Priority: P3 CC: cbateman, claes.rosell, erik.lundstrom, stepper
Version: 4.16   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
See Also: https://git.eclipse.org/r/c/cdo/cdo/+/185928
https://bugs.eclipse.org/bugs/show_bug.cgi?id=576893
Whiteboard:
Bug Depends on: 576596    
Bug Blocks:    

Description David Hein CLA 2010-11-01 04:28:56 EDT
Build Identifier: 

i want to use CDOObject.cdoPrefetch(). For models with a large count of objects i got a TimeoutException. Is there any configuration that must be set to use this function or can i raise the timeout?

org.eclipse.emf.cdo.common.util.TransportException: java.util.concurrent.TimeoutException
    at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.send(CDOClientProtocol.java:404)
    at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.send(CDOClientProtocol.java:429)
    at org.eclipse.emf.cdo.internal.net4j.protocol.CDOClientProtocol.loadRevisions(CDOClientProtocol.java:159)
    at org.eclipse.emf.cdo.internal.common.revision.CDORevisionManagerImpl.loadRevisions(CDORevisionManagerImpl.java:352)
    at org.eclipse.emf.cdo.internal.common.revision.CDORevisionManagerImpl.getRevisions(CDORevisionManagerImpl.java:263)
    at org.eclipse.emf.cdo.internal.common.revision.CDORevisionManagerImpl.getRevision(CDORevisionManagerImpl.java:246)
    at org.eclipse.emf.cdo.internal.common.revision.CDORevisionManagerImpl.getRevision(CDORevisionManagerImpl.java:239)
    at org.eclipse.emf.cdo.internal.common.revision.CDORevisionManagerImpl.getRevision(CDORevisionManagerImpl.java:1)
    at org.eclipse.emf.internal.cdo.view.CDOViewImpl.prefetchRevisions(CDOViewImpl.java:361)
    at org.eclipse.emf.internal.cdo.view.CDOViewImpl.prefetchRevisions(CDOViewImpl.java:355)
    at org.eclipse.emf.internal.cdo.CDOObjectImpl.cdoPrefetch(CDOObjectImpl.java:159) 

Reproducible: Always
Comment 1 David Hein CLA 2010-11-01 04:29:56 EDT
I use CDO 4.0 with a DBStore (MySQL) and a horizontalAuditWithRanges mapping.
Comment 2 Eike Stepper CLA 2010-11-02 06:20:20 EDT
I will try to make use of the network monitoring framework (RequestWithMonitoring). Unlikely that I have time during the Eclipse Summit this week...
Comment 3 Eike Stepper CLA 2011-06-23 03:59:40 EDT
Moving all open enhancement requests to 4.1
Comment 4 Eike Stepper CLA 2012-08-14 22:58:24 EDT
Moving all open issues to 4.2. Open bugs can be ported to 4.1 maintenance after they've been fixed in master.
Comment 5 Eike Stepper CLA 2013-06-27 04:09:39 EDT
Moving all outstanding enhancements to 4.3
Comment 6 Eike Stepper CLA 2014-08-19 09:29:44 EDT
Moving all open enhancement requests to 4.4
Comment 7 Eike Stepper CLA 2014-08-19 09:38:23 EDT
Moving all open enhancement requests to 4.4
Comment 8 Eike Stepper CLA 2015-07-14 02:22:40 EDT
Moving all open bugzillas to 4.5.
Comment 9 Eike Stepper CLA 2016-07-31 01:05:41 EDT
Moving all unaddressed bugzillas to 4.6.
Comment 10 Eike Stepper CLA 2017-12-28 01:13:33 EST
Moving all open bugs to 4.7
Comment 11 Claes Rosell CLA 2019-04-29 08:10:53 EDT
This is an issue for me as well. Currently we need to do a pretty deep .prefetch() to achieve the performance that we need. However, with default timeout we get a TimeoutException. We can of cause avoid this by increasing the timeout but it would be nice to solve it in another way.

I made some early tests by just changing LoadRevisionsRequest and LoadRevisionsIndication to implement CDOClientRequestWithMonitoring and CDOServerReadIndicationWithMonitoring, respectively. This worked in some cases, but I received at least one "revision == null, ObjectNotFoundException" from AbstractCDOView, so I guess that there are more things in play here.

Eike, are there any obvious pitfalls here? Something obvious that I have missed?
Comment 12 Eike Stepper CLA 2019-11-08 02:16:10 EST
Moving all unresolved issues to version 4.8-
Comment 13 Eike Stepper CLA 2019-12-13 12:42:01 EST
Moving all unresolved issues to version 4.9
Comment 14 Eike Stepper CLA 2020-12-11 10:37:40 EST
Moving to 4.13.
Comment 15 Eclipse Genie CLA 2021-09-29 03:07:39 EDT
New Gerrit change created: https://git.eclipse.org/r/c/cdo/cdo/+/185928
Comment 16 Erik Lundström CLA 2021-09-29 03:13:42 EDT
As I stumbled upon this problem as well, I did an attempt to solve this using the suggested RequestWithMonitoring. 
We have used the contents of the Gerrit I just pushed (https://git.eclipse.org/r/c/cdo/cdo/+/185928) in production environment the last year, without problems. (I think we CDO version 4.7 was used).

Please review.
Comment 17 Eike Stepper CLA 2021-09-29 08:08:18 EDT
I'll come to this issue very soon because one of my clients also saw the timeouts. I also had the idea with RequestWithMonitoring, but i'd like to make sure that the 99% case of revision loading with just 1 revision does not pay the price...
Comment 18 Eike Stepper CLA 2021-10-13 06:14:40 EDT
The changes are committed:
https://git.eclipse.org/c/cdo/cdo.git/commit/?id=ae07fc3c71c3795255ea76f4d63996024cf91f74
Comment 19 Eike Stepper CLA 2021-10-13 06:19:19 EDT
Sorry, the previous link to the commit is from the corresponding Net4j enhancement (Bug 576596: Give signal indications the ability to reset the request timeout).

The commit of this enhancement is:
https://git.eclipse.org/c/cdo/cdo.git/commit/?id=5fb964e0ad863c78ed31e6d83d845bd8ef6b6871
Comment 20 Eike Stepper CLA 2021-11-09 22:51:22 EST
Please see also bug 576893 "Implement a CDOPrefetcherManager to prefetch and cache all valid revisions for a CDOViewSet". If you're interested you can study a working example in:

CDOCheckoutImpl.startPrefetcherManager()
CDOCheckoutImpl.stopPrefetcherManager()

THis functionality is exposed to the user in the checkout properties page.