Community
Participate
Working Groups
Due to BIRT bug #212387, we had to use the parameter "useOldAliasMetadatabehavior=true" on all connection URLs to our MySQL DB in the past. Now that this bug is fixed (which I'm not really sure about), I wanted to remove the mentioned parameter in the connection URL. But then the reports raise an OutOfMemoryError. To clarify: the execption is thrown in the "Preview" view of the Report Designer and also when the report is deployed in my development environment. However, in "Edit Data Set" -> "Preview Results" the result set is shown. The stack trace is given at the end of the bug report. A test report is attached. In a productive environment, where I have more memory available (-Xmx4096m -XX:MaxPermSize=256m), the same report does not raise an OutOfMemoryError, but instead: org.eclipse.birt.report.engine.api.EngineException: There is an error in loading files of data set cache Malformed input around byte 35 In deveopment environment (where the OutOfMemoryError occurs) I stepped into the code and realized the following: The byte[] ret in IOUtil:1125 has a length of 822MB! In convertBytes2String() line 1233, BIRT tries to allocate a second array of that size, which of course fails with the available memory in development environment - but not in production! This explains the difference between the two. So why is BIRT trying to allocate that much memory? Jason Weathersby recommended to open this bug. Please also see http://www.birt-exchange.org/org/forum/index.php/topic/20330-there-is-an-error-in-loading-files-of-data-set-cache Exception in thread "Thread-65" java.lang.OutOfMemoryError: Java heap space at org.eclipse.birt.core.util.IOUtil.convertBytes2String(IOUtil.java:1233) at org.eclipse.birt.core.util.IOUtil.readUTF(IOUtil.java:1127) at org.eclipse.birt.core.util.IOUtil.readString(IOUtil.java:795) at org.eclipse.birt.data.engine.executor.cache.ResultObjectUtil.readData(ResultObjectUtil.java:185) at org.eclipse.birt.data.engine.executor.dscache.CacheUtilFactory$DiskLoadUtil.loadObject(CacheUtilFactory.java:551) at org.eclipse.birt.data.engine.executor.dscache.DataSetFromCache.loadObject(DataSetFromCache.java:76) at org.eclipse.birt.data.engine.executor.dscache.DataSetFromCache.fetch(DataSetFromCache.java:58) at org.eclipse.birt.data.engine.executor.cache.OdiAdapter.fetch(OdiAdapter.java:161) at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:105) at org.eclipse.birt.data.engine.executor.cache.ExpandableRowResultSet.next(ExpandableRowResultSet.java:63) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.populateData(SmartCacheHelper.java:311) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.initInstance(SmartCacheHelper.java:283) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.getResultSetCache(SmartCacheHelper.java:244) at org.eclipse.birt.data.engine.executor.cache.SmartCache.<init>(SmartCache.java:69) at org.eclipse.birt.data.engine.executor.transform.pass.PassUtil.populateOdiResultSet(PassUtil.java:142) at org.eclipse.birt.data.engine.executor.transform.pass.PassUtil.pass(PassUtil.java:62) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.doSinglePass(PassManager.java:183) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.pass(PassManager.java:93) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.populateResultSet(PassManager.java:74) at org.eclipse.birt.data.engine.executor.transform.ResultSetPopulator.populateResultSet(ResultSetPopulator.java:196) at org.eclipse.birt.data.engine.executor.transform.CachedResultSet.<init>(CachedResultSet.java:152) at org.eclipse.birt.data.engine.executor.dscache.DataSourceQuery.execute(DataSourceQuery.java:143) at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery$OdaDSQueryExecutor.executeOdiQuery(PreparedOdaDSQuery.java:399) at org.eclipse.birt.data.engine.impl.QueryExecutor.execute(QueryExecutor.java:1045) at org.eclipse.birt.data.engine.impl.ServiceForQueryResults.executeQuery(ServiceForQueryResults.java:232) at org.eclipse.birt.data.engine.impl.QueryResults.getResultIterator(QueryResults.java:158) at org.eclipse.birt.report.engine.data.dte.QueryResultSet.<init>(QueryResultSet.java:98) at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:168) at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:265) at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1875) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:80) at org.eclipse.birt.report.engine.executor.DataItemExecutor.execute(DataItemExecutor.java:75)
Created attachment 177386 [details] test report
False alarm, sorry. I'm confused. Now that I tried to execute the attached report again, I get: Error evaluating Javascript expression. Script engine error: ReferenceError: "thisimportPackage" is not defined. (#1) And in fact, there is a typo in one of my import statements. The OutOfMemoryError now only occurrs in the "Preview" mode, but not when deployed.
Well, the bug still exists. I corrected the typo mentioned in the comment above. The corrected report is attached again. The situation now is the following: - the report works in my development environment - the report fails in the productive environment. Error message: org.eclipse.birt.report.engine.api.EngineException: There is an error in loading files of data set cache Malformed input around byte 35 It's very strange. Please let me know if I should perfom futher tests to narrow the search.... Remo
Created attachment 177512 [details] corrected report
Anyone?
Hi Remo, I have the same problem here. Did you manage to solve it? This is my stacktrace: SEVERE: An OutOfMemory error happened while running the report. 2011-07-06 03:24:13,527 ERROR [STDERR] Exception in thread "TestScheduler_Worker-0" 2011-07-06 03:24:13,527 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space 2011-07-06 03:24:13,527 ERROR [STDERR] at org.eclipse.birt.core.util.IOUtil.convertBytes2String(IOUtil.java:1123) 2011-07-06 03:24:13,527 ERROR [STDERR] at org.eclipse.birt.core.util.IOUtil.readUTF(IOUtil.java:1020) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.core.util.IOUtil.readString(IOUtil.java:689) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.data.dte.DteMetaInfoIOUtil.loadDteMetaInfo(DteMetaInfoIOUtil.java:128) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.data.dte.DteMetaInfoIOUtil.loadDteMetaInfo(DteMetaInfoIOUtil.java:113) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.data.dte.DataPresentationEngine.loadDteMetaInfo(DataPresentationEngine.java:95) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.data.dte.DataPresentationEngine.<init>(DataPresentationEngine.java:73) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.data.DataEngineFactory.createDataEngine(DataEngineFactory.java:98) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.executor.ExecutionContext.openDataEngine(ExecutionContext.java:758) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.executor.ExecutionContext.getDataEngine(ExecutionContext.java:779) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.internal.document.v4.AbstractReportExecutor.<init>(AbstractReportExecutor.java:79) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.internal.document.v4.ReportPageExecutorV4.<init>(ReportPageExecutorV4.java:40) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.internal.document.ReportPageExecutor.<init>(ReportPageExecutor.java:41) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.RenderTask$PageRangeRender.render(RenderTask.java:449) 2011-07-06 03:24:13,528 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.RenderTask.render(RenderTask.java:206) 2011-07-06 03:24:13,528 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReportFile(ReportGenerator.java:189) 2011-07-06 03:24:13,528 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReportDocument(ReportGenerator.java:124) 2011-07-06 03:24:13,528 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReport(ReportGenerator.java:70) 2011-07-06 03:24:13,528 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.startAutomaticGeneration(ReportGenerator.java:306) 2011-07-06 03:24:13,528 ERROR [STDERR] at com.trintech.cms.reports.ReportGeneratorJob.execute(ReportGeneratorJob.java:66) 2011-07-06 03:24:13,529 ERROR [STDERR] at org.quartz.core.JobRunShell.run(JobRunShell.java:203) 2011-07-06 03:24:13,529 ERROR [STDERR] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520) 2011-07-06 03:24:25,429 ERROR [STDERR] Jul 6, 2011 3:24:25 AM org.eclipse.birt.report.model.parser.ModuleReader readModule SEVERE: Line Number:53 Error Code:Error.XMLParserException.SAX_ERROR Exception:org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x0) was found in the value of attribute "id" and element is "cell". Message:SAX detected an error with the basic XML syntax of the file. 2011-07-06 03:24:25,460 ERROR [STDERR] Jul 6, 2011 3:24:25 AM org.eclipse.birt.report.model.parser.ModuleReader readModule SEVERE: Line Number:53 Error Code:Error.XMLParserException.SAX_ERROR Exception:org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x0) was found in the value of attribute "id" and element is "cell". Message:SAX detected an error with the basic XML syntax of the file. 2011-07-06 03:24:25,469 ERROR [STDERR] Jul 6, 2011 3:24:25 AM org.eclipse.birt.report.engine.api.impl.ReportEngineHelper openReportDesign SEVERE: invalid design file file:/data/jboss/jboss-4.2.0.GA/server/producao/deploy/birt.war/reports/acquirer/reports/schedulePayments.rptdesign 2011-07-06 03:24:25,470 ERROR [STDERR] Jul 6, 2011 3:24:25 AM org.eclipse.birt.report.engine.api.impl.ReportEngineHelper openReportDesign SEVERE: invalid design file file:/data/jboss/jboss-4.2.0.GA/server/producao/deploy/birt.war/reports/acquirer/reports/schedulePayments.rptdesign 2011-07-06 03:24:25,471 ERROR [STDERR] Jul 6, 2011 3:24:25 AM org.eclipse.birt.report.engine.api.impl.ReportDocumentReader getReportRunnable SEVERE: Failed to get the report runnable Write before this I got this error: 2011-07-06 03:22:07,764 ERROR [com.trintech.cms.reports.ReportGenerator] PAYware CMS Exception generateReportFile en ReportGenerator null-/data/jboss/jboss-4.2.0.GA/server/producao/files/reports/acquirer/reports/schedulePayments.document-/data/jboss/jboss-4.2.0.GA/server/producao/files/reports/schedulePayments_20110706_1_1_MASTERCARD_SANTANDER_pt - pdf 2011-07-06 03:22:07,764 ERROR [STDERR] java.lang.NullPointerException 2011-07-06 03:22:07,764 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.RenderTask.<init>(RenderTask.java:108) 2011-07-06 03:22:07,764 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.RenderTask.<init>(RenderTask.java:76) 2011-07-06 03:22:07,764 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.ReportEngineHelper.createRenderTask(ReportEngineHelper.java:458) 2011-07-06 03:22:07,764 ERROR [STDERR] at org.eclipse.birt.report.engine.api.impl.ReportEngine.createRenderTask(ReportEngine.java:496) 2011-07-06 03:22:07,764 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReportFile(ReportGenerator.java:162) 2011-07-06 03:22:07,764 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReportDocument(ReportGenerator.java:124) 2011-07-06 03:22:07,764 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.generateReport(ReportGenerator.java:70) 2011-07-06 03:22:07,765 ERROR [STDERR] at com.trintech.cms.reports.ReportGenerator.startAutomaticGeneration(ReportGenerator.java:306) 2011-07-06 03:22:07,765 ERROR [STDERR] at com.trintech.cms.reports.ReportGeneratorJob.execute(ReportGeneratorJob.java:66) 2011-07-06 03:22:07,765 ERROR [STDERR] at org.quartz.core.JobRunShell.run(JobRunShell.java:203) 2011-07-06 03:22:07,765 ERROR [STDERR] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520) 2011-07-06 03:24:13,507 ERROR [STDERR] Jul 6, 2011 3:24:13 AM org.eclipse.birt.report.engine.api.impl.RenderTask render SEVERE: An OutOfMemory error happened while running the report. 2011-07-06 03:24:13,527 ERROR [STDERR] Jul 6, 2011 3:24:13 AM org.eclipse.birt.report.engine.api.impl.RenderTask render SEVERE: An OutOfMemory error happened while running the report. Thanks in advance!
No, unfortunately it's still unresolved.
We encounter the same error! - works in development environment - doesn't work in production This is serious, why doesn't anybody look into this error?
Engineering is looking into it. Thanks. Lin
I get this error running a BIRT report using the BIRT 3.7.1 runtime under Glassfish 3.1.1 with a JNDI data source (which connects to DB2 Express-C 9.7.2 using its JDBC 4-compliant driver). When I run the same report in the same environment with a username and password embedded in the BIRT report definition rather than using JNDI, however, I DON'T get this error. (That, in fact, is my workaround until this problem is resolved.) So whatever the problem is, there appears to be a difference between database connections obtained through JNDI and those opened directly by BIRT using JDBC. (I've also tried two different versions of DB2's JDBC 4 driver, and get consistent results from both.) By the way, I am using the "useJDBC4ColumnNameAndLabelSemantics=2" parameter on my DB2 connections, which sounds like it is analogous to MySQL's "useOldAliasMetadatabehavior=true" flag, about which this bug was originally opened. Here is my stacktrace: [#|2011-11-07T10:39:17.974-0500|INFO|glassfish3.1.1|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=23;_ThreadName=Thread-2;|java.lang.OutOfMemoryError: Java heap space at org.eclipse.birt.core.util.IOUtil.readUTF(IOUtil.java:1175) at org.eclipse.birt.core.util.IOUtil.readString(IOUtil.java:812) at org.eclipse.birt.data.engine.executor.cache.ResultObjectUtil.readData(ResultObjectUtil.java:187) at org.eclipse.birt.data.engine.executor.dscache.CacheUtilFactory$DiskLoadUtil.loadObject(CacheUtilFactory.java:552) at org.eclipse.birt.data.engine.executor.dscache.DataSetFromCache.loadObject(DataSetFromCache.java:86) at org.eclipse.birt.data.engine.executor.dscache.DataSetFromCache.fetch(DataSetFromCache.java:67) at org.eclipse.birt.data.engine.executor.cache.OdiAdapter.fetch(OdiAdapter.java:222) at org.eclipse.birt.data.engine.executor.cache.RowResultSet.doNext(RowResultSet.java:113) at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:91) at org.eclipse.birt.data.engine.executor.cache.ExpandableRowResultSet.next(ExpandableRowResultSet.java:63) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.populateData(SmartCacheHelper.java:316) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.initInstance(SmartCacheHelper.java:285) at org.eclipse.birt.data.engine.executor.cache.SmartCacheHelper.getResultSetCache(SmartCacheHelper.java:246) at org.eclipse.birt.data.engine.executor.cache.SmartCache.<init>(SmartCache.java:73) at org.eclipse.birt.data.engine.executor.transform.pass.PassUtil.populateOdiResultSet(PassUtil.java:142) at org.eclipse.birt.data.engine.executor.transform.pass.PassUtil.pass(PassUtil.java:62) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.doSinglePass(PassManager.java:211) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.prepareDataSetResultSet(PassManager.java:94) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.pass(PassManager.java:125) at org.eclipse.birt.data.engine.executor.transform.pass.PassManager.populateResultSet(PassManager.java:74) at org.eclipse.birt.data.engine.executor.transform.ResultSetPopulator.populateResultSet(ResultSetPopulator.java:198) at org.eclipse.birt.data.engine.executor.transform.CachedResultSet.<init>(CachedResultSet.java:154) at org.eclipse.birt.data.engine.executor.DataSourceQuery.execute(DataSourceQuery.java:1032) at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery$OdaDSQueryExecutor.executeOdiQuery(PreparedOdaDSQuery.java:441) at org.eclipse.birt.data.engine.impl.QueryExecutor.execute(QueryExecutor.java:1124) at org.eclipse.birt.data.engine.impl.ServiceForQueryResults.executeQuery(ServiceForQueryResults.java:232) at org.eclipse.birt.data.engine.impl.QueryResults.getResultIterator(QueryResults.java:173) at org.eclipse.birt.report.engine.data.dte.QueryResultSet.<init>(QueryResultSet.java:98) at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:168) at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:267) at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1905) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:80)
Hi Kevin, could you attach your rptdesign Thanks peng
Created attachment 206723 [details] Report that manifests problem when executed with JNDI Connection BIRT report containing Dynamic images sourced from BLOB columns that fails with the previously indicated error.
Created attachment 206724 [details] Identical report with BLOBs removed that does NOT manifest problem with JNDI Connection I had a hunch that dynamic images from BLOB columns might be part of the problem, and I was right. I removed all references to BLOBs from the report and it works fine with a JNDI Connection. Note that I had to completely remove the BLOB values from the Data Set queries to get it to work. It wasn't enough to just remove the image objects from the report layout. Only a maximum of 4 different images are in the query, however, and in my test case they ranged from 30K to <900K in their compressed (JPEG) form. (And with no image objects referencing them, nothing should have been trying to expand them to full-size images, suggesting that the problem probably lies in the handling of BLOB values in general rather than images in particular.)
I accidentally left an unused image attached to the two samples I uploaded. It can be removed without effect.
Hi Remo, The dataset you used contains some clob or blob data? Thanks Peng
You probably can't tell from the report design, but the Text column in the ElementText query in my samples comes from a CLOB column, and it's functioning properly with a JNDI Connection. It appears to be only the BLOBs that are an issue.
(In reply to comment #15) > Hi Remo, > The dataset you used contains some clob or blob data? > > Thanks > Peng Yes, there is image data read from a blob column.
fixed, this problem is caused by the different version of mysql driver. Now we use alias as expression name in CacheUtilFactory.
Is it possible that the fix will affect drivers for other databases? (I'm using DB2.)
I think it will not affect other drivers or database
Verified in 3_7_2-20111216.
