Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 190453

Summary: OutOfMemoryError when generating large TPTP test reports.
Product: z_Archived Reporter: Paul Slauenwhite <paulslau>
Component: TPTPAssignee: Paul Slauenwhite <paulslau>
Status: CLOSED FIXED QA Contact:
Severity: major    
Priority: P1 CC: dmorris, jcayne, jerome.gout, jkubasta, jptoomey, sluiman
Version: unspecifiedKeywords: plan
Target Milestone: ---Flags: jptoomey: review+
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on: 213294, 213697    
Bug Blocks:    
Attachments:
Description Flags
Screen capture.
none
Partial patch. none

Description Paul Slauenwhite CLA 2007-06-01 08:28:51 EDT
OutOfMemoryError when generating TPTP test reports.

Using the default Eclipse initialization settings (see eclipse.ini), the following OutOfMemoryError is throw when generating TPTP test reports on the TPTP test bucket:

java.lang.OutOfMemoryError
	at org.apache.xerces.util.XMLStringBuffer.append(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.scanCharReferenceValue(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.scanAttributeValue(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanAttribute(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
	at org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:179)
	at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl.java:180)
	at org.eclipse.hyades.models.common.facades.behavioral.impl.FacadeResourceImpl.doLoad(FacadeResourceImpl.java:181)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1354)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1155)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoad(ResourceSetImpl.java:256)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper(ResourceSetImpl.java:271)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:398)
	at org.eclipse.hyades.models.common.util.ResourceUtil.load(ResourceUtil.java:272)
	at org.eclipse.hyades.models.common.util.ResourceUtil.load(ResourceUtil.java:246)
	at org.eclipse.hyades.models.common.util.ExecutionUtil$1.visit(ExecutionUtil.java:67)
	at org.eclipse.hyades.models.common.util.ResourceUtil.getFiles(ResourceUtil.java:203)
	at org.eclipse.hyades.models.common.util.ResourceUtil.getFiles(ResourceUtil.java:219)
	at org.eclipse.hyades.models.common.util.ResourceUtil.getFiles(ResourceUtil.java:219)
	at org.eclipse.hyades.models.common.util.ResourceUtil.getFiles(ResourceUtil.java:219)
	at org.eclipse.hyades.models.common.util.ResourceUtil.getFiles(ResourceUtil.java:135)
	at org.eclipse.hyades.models.common.util.ExecutionUtil.findExecutionResults(ExecutionUtil.java:80)
	at org.eclipse.hyades.test.tools.ui.common.internal.report.ReportWindowWizard.isAvailable(ReportWindowWizard.java:354)
	at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard.getWizardElements(ReportWizard.java:91)
	at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard.initPages(ReportWizard.java:74)
	at org.eclipse.hyades.ui.internal.wizard.HyadesWizard.init(HyadesWizard.java:80)
	at org.eclipse.hyades.test.ui.internal.navigator.action.OpenReportAction.run(OpenReportAction.java:45)
	at org.eclipse.jface.action.Action.runWithEvent(Action.java:498)
	at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:545)
	at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:490)
	at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:402)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:938)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3673)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3284)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2365)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2329)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2204)
	at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:289)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:461)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:153)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:363)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:176)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:615)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:497)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:436)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1162)

OS:
Windows XP Pro SP2

TPTP:
TPTP-4.4.0-200705080100B

JRE:
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build pwi32devifx-20070323 (if
ix 117674: SR4 + 116644 + 114941 + 116110 + 114881))
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223ifx-2007
0323 (JIT enabled)
J9VM - 20070322_12058_lHdSMR
JIT  - 20070109_1805ifx3_r8
GC   - WASIFIX_2007)
JCL  - 20070131

The worka-around is to increase the JVM heap size from 256 MB to 1024 MB in eclipse.ini.
Comment 1 Paul Slauenwhite CLA 2007-06-01 08:29:16 EDT
Please investigate and provide a sizing.
Comment 2 Paul Slauenwhite CLA 2007-06-01 08:30:31 EDT
Created attachment 69684 [details]
Screen capture.
Comment 3 Paul Slauenwhite CLA 2007-08-27 11:07:15 EDT
Required for 4.5.
Comment 4 Paul Slauenwhite CLA 2007-11-05 14:44:38 EST
Please provide a sizing for this defect before Monday, November 12, 2007.
Comment 5 Paul Slauenwhite CLA 2007-11-07 06:30:01 EST
Bulk update of the target field to reflect the change in iteration numbers.
Comment 6 Paul Slauenwhite CLA 2007-11-07 06:30:24 EST
Bulk update of the target field to reflect the change in iteration numbers.
Comment 7 Paul Slauenwhite CLA 2007-11-26 09:42:16 EST
Work-around (Eclipse):

Start Eclipse with a larger VM heap.  For example, add the following to the <Eclipse installation directory>\eclipse\eclipse.ini:

-Xmx1024m

Work-around (ASF/ANY):

Run the report from an Eclipse instance with a larger VM heap.  For example, add the following to the ANT script:

<property name="tptp.automation.vmargs" value="-Xmx1024m" />
Comment 8 Paul Slauenwhite CLA 2007-11-26 10:22:48 EST
Work-around provided so retargetting to i6.
Comment 9 Paul Slauenwhite CLA 2007-12-12 08:08:51 EST
The resources (e.g. execution histories) that are not used in org.eclipse.hyades.models.common.util.ExecutionUtil.findExecutionResults(List, long, long) need to be unloaded:

executionResult.eResource().unload();
Comment 10 Paul Slauenwhite CLA 2007-12-21 10:32:47 EST
This defect appears to be the result of a BIRT memory leak (see 213697).

After profiling our code, there is one place where we can do a better job at releasing unused memory:  

org.eclipse.hyades.models.common.util.ExecutionUtil.java: There are several places where we load an execution result resource for a test suite to confirm the ID and time stamp using the resource set of the test suite.  In most cases, we never use/return the loaded execution result resource.  However, the execution result resource is not unloaded or cannot be GCed.  We should be loading execution result resources using a new resource set so when the execution result resource is not used, it can be GCed after the method returns.

Attaching patch for the above fix to be integrated in I5. 

Joe: Can you review this patch?
Comment 11 Paul Slauenwhite CLA 2007-12-21 10:33:24 EST
Created attachment 85715 [details]
Partial patch.
Comment 12 Joe Toomey CLA 2008-01-02 14:20:35 EST
Looks good to me.

Thanks,
--Joe
Comment 13 Paul Slauenwhite CLA 2008-01-24 13:46:29 EST
Profiled/tested the patch using the default 256 MB max heap size and the TPTP test bucket with no OutOfMemoryError exceptions or memory leaks.

Checked in CVS (HEAD):

Comment 14 Harm Sluiman CLA 2008-01-24 18:58:22 EST
(In reply to comment #13)
> Profiled/tested the patch using the default 256 MB max heap size and the TPTP
> test bucket with no OutOfMemoryError exceptions or memory leaks.
> 
> Checked in CVS (HEAD):
> 

Just curious, how did you confirm no memory leaks?
Comment 15 Paul Slauenwhite CLA 2008-01-25 07:04:16 EST
(In reply to comment #14)
> (In reply to comment #13)
> > Profiled/tested the patch using the default 256 MB max heap size and the TPTP
> > test bucket with no OutOfMemoryError exceptions or memory leaks.
> > 
> > Checked in CVS (HEAD):
> > 
> 
> Just curious, how did you confirm no memory leaks?
> 

While increasing the number of execution results to be reported, I used the TPTP profiler to monitor a constant memory usage.  Previously, BIRT and the utility method in our model code had positive memory usage.
Comment 16 Paul Slauenwhite CLA 2008-02-12 09:40:54 EST
Verified in 4.5.0-200802100400 (using the M4 BIRT).
Comment 17 Paul Slauenwhite CLA 2008-02-12 09:44:23 EST
(In reply to comment #16)
> Verified in 4.5.0-200802100400 (using the M4 BIRT).
> 
 
Due to BIRT 2.3.0 M5 blocking defects:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=218578
https://bugs.eclipse.org/bugs/show_bug.cgi?id=218018