Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 347190 - LAZY IndirectSet is fetched needlessly during a Query
Summary: LAZY IndirectSet is fetched needlessly during a Query
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Eclipselink (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: Tom Ware CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-25 12:28 EDT by Patric Rufflar CLA
Modified: 2022-06-09 10:03 EDT (History)
3 users (show)

See Also:


Attachments
patch which adds the testcase (10.56 KB, text/plain)
2011-05-25 12:29 EDT, Patric Rufflar CLA
no flags Details
persistence.xml which contains the test persistence unit (1.12 KB, text/plain)
2011-05-25 12:30 EDT, Patric Rufflar CLA
no flags Details
proposed fix 2.2 stream (15.51 KB, patch)
2011-07-18 10:43 EDT, Tom Ware CLA
no flags Details | Diff
proposed fix - trunk stream (14.99 KB, patch)
2011-07-18 11:30 EDT, Tom Ware CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Patric Rufflar CLA 2011-05-25 12:28:13 EDT
Build Identifier: 2.2.0

EclipseLink 2.2 seems to have a bug which causes that an IndirectSet (and possibly all indirection collections) will be resolved during a query needlessly.

I will add a diff which will add a test case which proves the behavior.
In addition, you can see in the log file (when using FINEST log level), that the same entity is fetched more than once during the same query.

Because I am providing my own entities, you have to put the persistence.xml in an appropriate place that it will be processed during the test.
Use log level finest, and you'll see that the IndirectSet is fetched (or place a breakpoint on IndirectSet.size()).

Please note, that the same query, using the same entities with the same data is not showing the problem when using EclipseLink 1.1/2.1.

In my situation, the bug causes that some IndirectSet are fetched without any reason and because a Set may contain millions of records (and would never be fetched under normal conditions) locks up the system.
So in fact, this bug is a blocker for us.

Reproducible: Always

Steps to Reproduce:
- Patching your 2.2 trunk using the diff I'am providing to get the test case and the entities
- Place the persistence.xml in a META-INF directory (or a jar file) so that the new persistence unit will be available for the test.
- Run the test testIndirectCollectionRefreshBehavior()
Comment 1 Patric Rufflar CLA 2011-05-25 12:29:45 EDT
Created attachment 196573 [details]
patch which adds the testcase
Comment 2 Patric Rufflar CLA 2011-05-25 12:30:34 EDT
Created attachment 196574 [details]
persistence.xml which contains the test persistence unit
Comment 3 Tom Ware CLA 2011-05-26 13:46:30 EDT
Targetting for 2.2.1

This issue occurs because our check to see whether to refresh the objects in the M-1 relationship checks a blank ValueHolder that has just been built as part of buildObject.  That valueholder appears to be instantiated because it has not been populated yet.

See ForeignReferenceMapping.buildCloneFromRow()


            Object oldAttribute = this.getAttributeValueFromObject(clone);
            setAttributeValueInObject(clone, clonedAttributeValue); // set this first to prevent infinite recursion
            if (this.indirectionPolicy.objectIsInstantiatedOrChanged(oldAttribute)){
                this.indirectionPolicy.instantiateObject(clone, clonedAttributeValue);

Here, for a 1-1 or M-1, oldAttribute will be a new instance of valueholder and EclipseLink will think it is instantiated therefore instantiating the object

This is a bigger deal for a M-1 than a 1-1 because triggering the M-1 will mean we have to deal with the 1-M relationship on the other side and trigger it.
Comment 4 Tom Ware CLA 2011-05-26 15:01:39 EDT
Possible solution is to remove this block from ForeignReferenceMapping.buildCloneFromRow()



        if (executionSession.isUnitOfWork() && sourceQuery.shouldRefreshIdentityMapResult()){
            Object oldAttribute = this.getAttributeValueFromObject(clone);
            setAttributeValueInObject(clone, clonedAttributeValue); // set this first to prevent infinite recursion
            if (this.indirectionPolicy.objectIsInstantiatedOrChanged(oldAttribute)){
                this.indirectionPolicy.instantiateObject(clone, clonedAttributeValue);
            }
        }


Cascade refresh tests will need to be added.
Comment 5 Tom Ware CLA 2011-05-26 15:05:02 EDT
Alternate solution:

See why oldAttribute has been added as an Empty ValueHolder and if it is weaving-related, we can check isNewlyWeavedValueHolder in WeavedBasicIndirectionPolicy.objectIsInstantiatedOrChanged()
Comment 6 Tom Ware CLA 2011-07-18 10:43:19 EDT
Created attachment 199839 [details]
proposed fix 2.2 stream
Comment 7 Tom Ware CLA 2011-07-18 11:30:27 EDT
Created attachment 199842 [details]
proposed fix - trunk stream

The reason for the issue is a set of changes related to handling relationships with non-cacheable entities

Those changes included an update that would refresh across LAZY relationships in certain cases. The issue with the change was that while we were building the entity the relationship would hold a null, and we would treat it as though it needed refreshing.

The fix is to detect the null in the partially built object.
Comment 8 Tom Ware CLA 2011-07-19 13:38:40 EDT
Fixed in 2.2.1, 2.3.1 and trunk

Fix adds a way to as an IndirectionPolicy if it has finished building it's object and checks for fully built objects before refeshing

Reviewed by Andrei Ilitchev

Added test to CacheableJUnitTestModel

Tested with JPA LRG and Core LRG
Comment 9 Patric Rufflar CLA 2011-07-19 14:32:33 EDT
Thanks, Tom.
Comment 10 Alfredo Osorio CLA 2013-11-01 11:58:47 EDT
This patch made that when you have a query with QueryHint.REFRESH to true and you have an entity with a default list initialized example

public class Department {

}
Comment 11 Alfredo Osorio CLA 2013-11-01 12:07:46 EDT
This patch made that when you have a query with QueryHint.REFRESH to true and you have an entity with a default list initialized example

public class Department {
   
  private List<Employee> employees = new ArrayList<Employee>();

  @OneToMany(mappedBy="department", cascade=CascadeType.ALL)
  public List<Employee> getEmployees() {
	  return this.employees;
  }
  
  public void setEmployees(List<Employee> employees) {
	  this.employees = employees;
  }

}

It fetches the OneToMany relationship even though it is LAZY. If you remove the new ArrayList<Employee>() it doesn't fetch the data. This has something to do with the way org.eclipse.persistence.mappings.ForeignReferenceMapping.buildCloneFromRow:
 boolean wasAttributeValueFullyBuilt = isAttributeValueFullyBuilt(clone);

determines if the relationship was initialized, it wasn't it just have an empty list not created by EclipseLink.

Maybe it also has something to do with:
org.eclipse.persistence.internal.indirection.TransparentIndirectionPolicy.objectIsInstantiated(Object)

public boolean objectIsInstantiated(Object object) {
        if (object instanceof IndirectContainer) {
            return ((IndirectContainer)object).isInstantiated();
        } else {
            return true;// it must be a "real" collection
        }
    }

Maybe it shouldn't return true because it is an instance of ArrayList and it should return false in the else statement.
Comment 12 Alfredo Osorio CLA 2013-11-01 12:34:40 EDT
I'm interested in resolving this bug if you could give me an advice to solve it, I'll be happy to provide a patch.
Comment 13 Eclipse Webmaster CLA 2022-06-09 10:03:04 EDT
The Eclipselink project has moved to Github: https://github.com/eclipse-ee4j/eclipselink