Bug 365395

Description Yaytay 2011-12-02 03:03:50 EST

I have a table that has 500 INT rows and 100000 columns.
I have created a report that runs a simple "select * from megatable" and puts the result into a table.
After consuming >3GB of RAM BIRT gives up on processing the query with an out of memory error.

This are two specific problems:
1. getMetadata is called by the UI, and results in a complete run of the query.
BIRT is going to be used for a great many queries that are not feasible to run at design time - if getMetadata has to run the query at design time it should always limit the output.

2. ODA tries to cache the results of the query from getMetadata, but does so by getting a complete ResultSet in RAM, which doesn't fit (and is very inefficient anyway).

My opinion, which is based on not knowing much about the internals of ODA, is that:
1. getMetadata should never run a full query.
This may be less efficient for small queries, maybe a way around that is to set MaxRows to something reasonable (500?) and then to cache the result only if it fits in that.
2. When BIRT processes a resultset it should always stream it from source to destination (rptdocument) so it never tries to fit the whole thing in RAM.
MaxFetchSize should achieve this.