Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 347190

Summary: LAZY IndirectSet is fetched needlessly during a Query
Product: z_Archived Reporter: Patric Rufflar <patric>
Component: EclipselinkAssignee: Tom Ware <tom.ware>
Status: CLOSED FIXED QA Contact:
Severity: critical    
Priority: P3 CC: alfredo.osorio, eclipselink.orm-inbox, tom.ware
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
patch which adds the testcase
none
persistence.xml which contains the test persistence unit
none
proposed fix 2.2 stream
none
proposed fix - trunk stream none

Description Patric Rufflar CLA 2011-05-25 12:28:13 EDT
Build Identifier: 2.2.0

EclipseLink 2.2 seems to have a bug which causes that an IndirectSet (and possibly all indirection collections) will be resolved during a query needlessly.

I will add a diff which will add a test case which proves the behavior.
In addition, you can see in the log file (when using FINEST log level), that the same entity is fetched more than once during the same query.

Because I am providing my own entities, you have to put the persistence.xml in an appropriate place that it will be processed during the test.
Use log level finest, and you'll see that the IndirectSet is fetched (or place a breakpoint on IndirectSet.size()).

Please note, that the same query, using the same entities with the same data is not showing the problem when using EclipseLink 1.1/2.1.

In my situation, the bug causes that some IndirectSet are fetched without any reason and because a Set may contain millions of records (and would never be fetched under normal conditions) locks up the system.
So in fact, this bug is a blocker for us.

Reproducible: Always

Steps to Reproduce:
- Patching your 2.2 trunk using the diff I'am providing to get the test case and the entities
- Place the persistence.xml in a META-INF directory (or a jar file) so that the new persistence unit will be available for the test.
- Run the test testIndirectCollectionRefreshBehavior()
Comment 1 Patric Rufflar CLA 2011-05-25 12:29:45 EDT
Created attachment 196573 [details]
patch which adds the testcase
Comment 2 Patric Rufflar CLA 2011-05-25 12:30:34 EDT
Created attachment 196574 [details]
persistence.xml which contains the test persistence unit
Comment 3 Tom Ware CLA 2011-05-26 13:46:30 EDT
Targetting for 2.2.1

This issue occurs because our check to see whether to refresh the objects in the M-1 relationship checks a blank ValueHolder that has just been built as part of buildObject.  That valueholder appears to be instantiated because it has not been populated yet.

See ForeignReferenceMapping.buildCloneFromRow()


            Object oldAttribute = this.getAttributeValueFromObject(clone);
            setAttributeValueInObject(clone, clonedAttributeValue); // set this first to prevent infinite recursion
            if (this.indirectionPolicy.objectIsInstantiatedOrChanged(oldAttribute)){
                this.indirectionPolicy.instantiateObject(clone, clonedAttributeValue);

Here, for a 1-1 or M-1, oldAttribute will be a new instance of valueholder and EclipseLink will think it is instantiated therefore instantiating the object

This is a bigger deal for a M-1 than a 1-1 because triggering the M-1 will mean we have to deal with the 1-M relationship on the other side and trigger it.
Comment 4 Tom Ware CLA 2011-05-26 15:01:39 EDT
Possible solution is to remove this block from ForeignReferenceMapping.buildCloneFromRow()



        if (executionSession.isUnitOfWork() && sourceQuery.shouldRefreshIdentityMapResult()){
            Object oldAttribute = this.getAttributeValueFromObject(clone);
            setAttributeValueInObject(clone, clonedAttributeValue); // set this first to prevent infinite recursion
            if (this.indirectionPolicy.objectIsInstantiatedOrChanged(oldAttribute)){
                this.indirectionPolicy.instantiateObject(clone, clonedAttributeValue);
            }
        }


Cascade refresh tests will need to be added.
Comment 5 Tom Ware CLA 2011-05-26 15:05:02 EDT
Alternate solution:

See why oldAttribute has been added as an Empty ValueHolder and if it is weaving-related, we can check isNewlyWeavedValueHolder in WeavedBasicIndirectionPolicy.objectIsInstantiatedOrChanged()
Comment 6 Tom Ware CLA 2011-07-18 10:43:19 EDT
Created attachment 199839 [details]
proposed fix 2.2 stream
Comment 7 Tom Ware CLA 2011-07-18 11:30:27 EDT
Created attachment 199842 [details]
proposed fix - trunk stream

The reason for the issue is a set of changes related to handling relationships with non-cacheable entities

Those changes included an update that would refresh across LAZY relationships in certain cases. The issue with the change was that while we were building the entity the relationship would hold a null, and we would treat it as though it needed refreshing.

The fix is to detect the null in the partially built object.
Comment 8 Tom Ware CLA 2011-07-19 13:38:40 EDT
Fixed in 2.2.1, 2.3.1 and trunk

Fix adds a way to as an IndirectionPolicy if it has finished building it's object and checks for fully built objects before refeshing

Reviewed by Andrei Ilitchev

Added test to CacheableJUnitTestModel

Tested with JPA LRG and Core LRG
Comment 9 Patric Rufflar CLA 2011-07-19 14:32:33 EDT
Thanks, Tom.
Comment 10 Alfredo Osorio CLA 2013-11-01 11:58:47 EDT
This patch made that when you have a query with QueryHint.REFRESH to true and you have an entity with a default list initialized example

public class Department {

}
Comment 11 Alfredo Osorio CLA 2013-11-01 12:07:46 EDT
This patch made that when you have a query with QueryHint.REFRESH to true and you have an entity with a default list initialized example

public class Department {
   
  private List<Employee> employees = new ArrayList<Employee>();

  @OneToMany(mappedBy="department", cascade=CascadeType.ALL)
  public List<Employee> getEmployees() {
	  return this.employees;
  }
  
  public void setEmployees(List<Employee> employees) {
	  this.employees = employees;
  }

}

It fetches the OneToMany relationship even though it is LAZY. If you remove the new ArrayList<Employee>() it doesn't fetch the data. This has something to do with the way org.eclipse.persistence.mappings.ForeignReferenceMapping.buildCloneFromRow:
 boolean wasAttributeValueFullyBuilt = isAttributeValueFullyBuilt(clone);

determines if the relationship was initialized, it wasn't it just have an empty list not created by EclipseLink.

Maybe it also has something to do with:
org.eclipse.persistence.internal.indirection.TransparentIndirectionPolicy.objectIsInstantiated(Object)

public boolean objectIsInstantiated(Object object) {
        if (object instanceof IndirectContainer) {
            return ((IndirectContainer)object).isInstantiated();
        } else {
            return true;// it must be a "real" collection
        }
    }

Maybe it shouldn't return true because it is an instance of ArrayList and it should return false in the else statement.
Comment 12 Alfredo Osorio CLA 2013-11-01 12:34:40 EDT
I'm interested in resolving this bug if you could give me an advice to solve it, I'll be happy to provide a patch.
Comment 13 Eclipse Webmaster CLA 2022-06-09 10:03:04 EDT
The Eclipselink project has moved to Github: https://github.com/eclipse-ee4j/eclipselink