Community
Participate
Working Groups
OutOfMemoryError when running numerous RunAndRenderTasks. When creating/configuring/running numerous (e.g. 160) RunAndRenderTasks consecutively, the following exception is thrown: !SESSION 2007-12-20 17:08:31.616 ----------------------------------------------- eclipse.buildId=I20071213-1700 java.fullversion=J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223ifx-20070323 (JIT enabled) J9VM - 20070322_12058_lHdSMR JIT - 20070109_1805ifx3_r8 GC - WASIFIX_2007 BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US Framework arguments: -product org.eclipse.sdk.ide Command-line arguments: -product org.eclipse.sdk.ide -data D:\Workspaces\workspace_TPTP/../runtime-configuration -dev file:D:/Workspaces/workspace_TPTP/.metadata/.plugins/org.eclipse.pde.core/TPTP/dev.properties -os win32 -ws win32 -arch x86 !ENTRY org.eclipse.hyades.test.ui 4 0 2007-12-20 19:04:28.428 !MESSAGE java.lang.OutOfMemoryError !STACK 0 java.lang.OutOfMemoryError at org.eclipse.datatools.enablement.oda.xml.util.MemoryCacheDataSource.<init>(MemoryCacheDataSource.java:35) at org.eclipse.datatools.enablement.oda.xml.util.XMLDataInputStreamCreator.getCreator(XMLDataInputStreamCreator.java:127) at org.eclipse.datatools.enablement.oda.xml.util.XMLDataInputStreamCreator.getCreator(XMLDataInputStreamCreator.java:105) at org.eclipse.datatools.enablement.oda.xml.util.SaxParserConsumer.<init>(SaxParserConsumer.java:108) at org.eclipse.datatools.enablement.oda.xml.impl.ResultSet.<init>(ResultSet.java:65) at org.eclipse.datatools.enablement.oda.xml.impl.Query.executeQuery(Query.java:139) at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaQuery.doExecuteQuery(OdaQuery.java:443) at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaQuery.executeQuery(OdaQuery.java:408) at org.eclipse.birt.data.engine.odaconsumer.PreparedStatement.execute(PreparedStatement.java:553) at org.eclipse.birt.data.engine.executor.DataSourceQuery.execute(DataSourceQuery.java:495) at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery$OdaDSQueryExecutor.executeOdiQuery(PreparedOdaDSQuery.java:348) at org.eclipse.birt.data.engine.impl.QueryExecutor.execute(QueryExecutor.java:779) at org.eclipse.birt.data.engine.impl.ServiceForQueryResults.executeQuery(ServiceForQueryResults.java:232) at org.eclipse.birt.data.engine.impl.QueryResults.getResultIterator(QueryResults.java:158) at org.eclipse.birt.report.engine.data.dte.QueryResultSet.<init>(QueryResultSet.java:92) at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:129) at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:224) at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1706) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:77) at org.eclipse.birt.report.engine.executor.GridItemExecutor.execute(GridItemExecutor.java:64) at org.eclipse.birt.report.engine.internal.executor.dup.SuppressDuplicateItemExecutor.execute(SuppressDuplicateItemExecutor.java:42) at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.execute(WrappedReportItemExecutor.java:45) at org.eclipse.birt.report.engine.internal.executor.l18n.LocalizedReportItemExecutor.execute(LocalizedReportItemExecutor.java:33) at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:63) at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:85) at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:106) at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:138) at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run(RunAndRenderTask.java:66) at org.eclipse.tptp.platform.report.birt.core.provisional.BIRTReportGenerator.generateHTMLReportFragment(BIRTReportGenerator.java:293) at org.eclipse.tptp.platform.report.birt.core.provisional.BIRTReportGenerator.generateHTMLReport(BIRTReportGenerator.java:190) at org.eclipse.tptp.test.report.birt.internal.elements.TabularTPFTestSuiteReportElement.generate(TabularTPFTestSuiteReportElement.java:300) at org.eclipse.tptp.platform.report.birt.ui.internal.BIRTReportsWizard.generate(BIRTReportsWizard.java:60) at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard$FinishOperation.execute(ReportWizard.java:193) at org.eclipse.ui.actions.WorkspaceModifyOperation$1.run(WorkspaceModifyOperation.java:104) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1800) at org.eclipse.ui.actions.WorkspaceModifyOperation.run(WorkspaceModifyOperation.java:116) at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:407) at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:347) at org.eclipse.jface.wizard.WizardDialog.run(WizardDialog.java:934) at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard.performFinish(ReportWizard.java:148) at org.eclipse.jface.wizard.WizardDialog.finishPressed(WizardDialog.java:746) at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:373) at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:623) at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:227) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:952) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3750) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3361) at org.eclipse.jface.window.Window.runEventLoop(Window.java:825) at org.eclipse.jface.window.Window.open(Window.java:801) at org.eclipse.hyades.test.ui.internal.navigator.action.OpenReportAction.run(OpenReportAction.java:49) at org.eclipse.jface.action.Action.runWithEvent(Action.java:498) at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:582) at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:499) at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:410) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:952) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3750) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3361) at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2381) at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2345) at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2211) at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:473) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:288) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:468) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:362) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:175) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:561) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:501) at org.eclipse.equinox.launcher.Main.run(Main.java:1239) at org.eclipse.equinox.launcher.Main.main(Main.java:1215) After profiling, the call to IRunAndRenderTasks.run() takes 20 - 40 MB and apparently not releasing the memory. This defect can be reproduced by using the following steps: 1) Download TPTP (December 22 build or later), EMF/BIRT/GMF/DTP, and the TPTP BIRT Test Reports (see the as-is section). 2) Extract all of the TPTP test plug-ins from (see http://www.eclipse.org/tptp/home/documents/process/TPTP_Testing_Strategy.html): http://dev.eclipse.org/viewcvs/index.cgi/test-results/monitor/?root=TPTP_Project http://dev.eclipse.org/viewcvs/index.cgi/test-results/test/?root=TPTP_Project http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/?root=TPTP_Project http://dev.eclipse.org/viewcvs/index.cgi/test-results/trace/?root=TPTP_Project 3) Open the Test Perspective and multiple select the following files in the Test Navigator: http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.tests/TP1/?root=TPTP_Project 4) Select Report, select HTML Tabular Test Report (BIRT), and specify a report window from Dec 10 - 21 (see http://help.eclipse.org/help33/topic/org.eclipse.tptp.test.report.birt.doc.user/tasks/treporttest.xhtml). 5) Click Finish and note the OutOfMemoryError. Note: The above scenario was tested with a max heap size of 1.5 GB and still produced the OutOfMemoryError.
Note, this can be reproduce in any TPTP build after December 10.
We allocated over-sized memory cache for the input stream so caused this bug. Fix it by limit the size of cache and release it when the connection is closed. Fix it now!
(In reply to comment #2) > We allocated over-sized memory cache for the input stream so caused this bug. > Fix it by limit the size of cache and release it when the connection is closed. > Fix it now! > Is this fix included in the BIRT 2_3_0-N20071228 build? Does it require a new DTP build or will DTP 1.6M4 suffice?
Using BIRT 2_3_0-N20071228, this symptom is still found, although we are able to get farther along in the reporting process. If this driver does not include the fix, please provide the build ID that includes this fix. Here is the session info and stack trace: !SESSION 2008-01-02 12:07:32.093 ----------------------------------------------- eclipse.buildId=I20071213-1700 java.fullversion=J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223ifx-20070323 (JIT enabled) J9VM - 20070322_12058_lHdSMR JIT - 20070109_1805ifx3_r8 GC - WASIFIX_2007 BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US Command-line arguments: -os win32 -ws win32 -arch x86 -clean -data D:\AC_Hyades_WSAD_Drivers\4.5.0-200801010400\eclipse\workspace !ENTRY org.eclipse.hyades.test.ui 4 0 2008-01-02 13:56:20.312 !MESSAGE java.lang.OutOfMemoryError !STACK 0 java.lang.OutOfMemoryError at org.eclipse.datatools.enablement.oda.xml.util.MemoryCacheDataSource.<init>(MemoryCacheDataSource.java:35) at org.eclipse.datatools.enablement.oda.xml.util.XMLDataInputStreamCreator.getCreator(XMLDataInputStreamCreator.java:127) at org.eclipse.datatools.enablement.oda.xml.util.XMLDataInputStreamCreator.getCreator(XMLDataInputStreamCreator.java:105) at org.eclipse.datatools.enablement.oda.xml.util.SaxParserConsumer.<init>(SaxParserConsumer.java:108) at org.eclipse.datatools.enablement.oda.xml.impl.ResultSet.<init>(ResultSet.java:65) at org.eclipse.datatools.enablement.oda.xml.impl.Query.executeQuery(Query.java:139) at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaQuery.doExecuteQuery(OdaQuery.java:443) at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaQuery.executeQuery(OdaQuery.java:408) at org.eclipse.birt.data.engine.odaconsumer.PreparedStatement.execute(PreparedStatement.java:553) at org.eclipse.birt.data.engine.executor.DataSourceQuery.execute(DataSourceQuery.java:495) at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery$OdaDSQueryExecutor.executeOdiQuery(PreparedOdaDSQuery.java:348) at org.eclipse.birt.data.engine.impl.QueryExecutor.execute(QueryExecutor.java:782) at org.eclipse.birt.data.engine.impl.ServiceForQueryResults.executeQuery(ServiceForQueryResults.java:232) at org.eclipse.birt.data.engine.impl.QueryResults.getResultIterator(QueryResults.java:158) at org.eclipse.birt.report.engine.data.dte.QueryResultSet.<init>(QueryResultSet.java:92) at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:129) at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:224) at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1708) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:77) at org.eclipse.birt.report.engine.executor.GridItemExecutor.execute(GridItemExecutor.java:64) at org.eclipse.birt.report.engine.internal.executor.dup.SuppressDuplicateItemExecutor.execute(SuppressDuplicateItemExecutor.java:42) at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.execute(WrappedReportItemExecutor.java:45) at org.eclipse.birt.report.engine.internal.executor.l18n.LocalizedReportItemExecutor.execute(LocalizedReportItemExecutor.java:33) at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:63) at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:85) at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:106) at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:138) at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run(RunAndRenderTask.java:66) at org.eclipse.tptp.platform.report.birt.core.provisional.BIRTReportGenerator.generateHTMLReportFragment(BIRTReportGenerator.java:293) at org.eclipse.tptp.platform.report.birt.core.provisional.BIRTReportGenerator.generateHTMLReport(BIRTReportGenerator.java:190) at org.eclipse.tptp.test.report.birt.internal.elements.TabularTPFTestSuiteReportElement.generate(TabularTPFTestSuiteReportElement.java:300) at org.eclipse.tptp.platform.report.birt.ui.internal.BIRTReportsWizard.generate(BIRTReportsWizard.java:60) at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard$FinishOperation.execute(ReportWizard.java:193) at org.eclipse.ui.actions.WorkspaceModifyOperation$1.run(WorkspaceModifyOperation.java:104) at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:1800) at org.eclipse.ui.actions.WorkspaceModifyOperation.run(WorkspaceModifyOperation.java:116) at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:407) at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:347) at org.eclipse.jface.wizard.WizardDialog.run(WizardDialog.java:934) at org.eclipse.hyades.test.ui.internal.wizard.ReportWizard.performFinish(ReportWizard.java:148) at org.eclipse.jface.wizard.WizardDialog.finishPressed(WizardDialog.java:746) at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:373) at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:623) at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:227) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:952) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3750) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3361) at org.eclipse.jface.window.Window.runEventLoop(Window.java:825) at org.eclipse.jface.window.Window.open(Window.java:801) at org.eclipse.hyades.test.ui.internal.navigator.action.OpenReportAction.run(OpenReportAction.java:49) at org.eclipse.jface.action.Action.runWithEvent(Action.java:498) at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:582) at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:499) at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:410) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:952) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3750) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3361) at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2381) at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2345) at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2211) at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:473) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:288) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:468) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:362) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:175) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:615) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:561) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:501) at org.eclipse.equinox.launcher.Main.run(Main.java:1239)
Hi Paul, The fix is targeting Dtp 1.6M5, for when it is spotted the Dtp 1.6M4 has already been delivered, and BIRT is currently in M4 release cycle, which depends on Dtp1.6M4. So the fix to the problem will not be included in BIRT build until M4 release (ETA:2008/1/7). Could you please download a Dtp latest build and replace following two jars in your BIRT? org.eclipse.datatools.enablement.oda.xml org.eclipse.datatools.enablement.oda.xml.ui And for your convenience I've attached in this bug two jars from last night's Dtp build.
Created attachment 86047 [details] Jar1.
Created attachment 86048 [details] Jar2.
This fix to the bug will only be shipped with BIRT 2.3M5.
Verified using the dtp-sdk_1.6M5_N010208.zip. I do not agree with deferring this fix to M5. This defect is blocking TPTP report generation (as I would imagine others), so it should be blocking the DTP M4 release as well. Can this fix be included in M4?
Reopening for consideration in M4.
DTP 1.6M4a has been posted to address this issue. The only changes in 1.6M4 --> 1.6M4a are as follows: 1. Updated plug-ins o.e.d.enablement.oda.xml and o.e.d.enablement.oda.xml.ui with those from the current nightly build. 2. Updated the o.e.d.enablement.oda.feature and o.e.d.enablement.oda.designer.feature definitions for the new plug-in versions. 3. Updated the o.e.d.source plug-in for the source attachments to the oda.xml and oda.xml.ui plug-ins. 4. Updated the o.e.d.sdk feature definition for the new o.e.d.source plug-in version. The M4a download is now available on the DTP download page and the DTP milestone update site (as used by Ganymede). NOTE that the URL for the download zips will change tomorrow from a "drop" zip to a mirrored one, once the Eclipse mirrors have had a chance to replicate the 1.6M4a zips.
(In reply to comment #11) Thanks Lin and John for your help. Closing.