Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 365299

Summary: FlatFile datasource is very slow, and reads the file one byte at a time
Product: [Tools] Data Tools Reporter: Yaytay <jim-eclipsebugs>
Component: Enablement:ODAAssignee: Mingxia Wu <mwu>
Status: NEW --- QA Contact: Xiaoying Gu <bluesoldier>
Severity: normal    
Priority: P3 CC: bluesoldier, lchan
Version: 1.9.1   
Target Milestone: ---   
Hardware: PC   
OS: Windows Vista   
Whiteboard:

Description Yaytay CLA 2011-12-01 06:31:05 EST
Build Identifier: 3.7.1

Running a BIRT report using a 36MB FlatFile datasource (500 columns, 10000 rows) takes a very long time - >30minutes.
Running SysInternals Process Monitor shows that it is reading one byte from the file every ~0.1ms (it's actually a bit quicker than that, because that would take an hour to read).
Looking in the source for org.eclipse.datatools.connectivity.oda.flatfile shows that it is using a bare FileInputStream.

Is there a good reason why this couldn't be wrapped in a BufferedInputStream?

Reproducible: Always

Steps to Reproduce:
1. Create a CSV with 500 columns and 10000 rows.
2. Add that file as a data source to a BIRT report.
3. Run the report, and use Process Monitor to track file accesses.