Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 365299 - FlatFile datasource is very slow, and reads the file one byte at a time
Summary: FlatFile datasource is very slow, and reads the file one byte at a time
Status: NEW
Alias: None
Product: Data Tools
Classification: Tools
Component: Enablement:ODA (show other bugs)
Version: 1.9.1   Edit
Hardware: PC Windows Vista
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Mingxia Wu CLA
QA Contact: Xiaoying Gu CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-12-01 06:31 EST by Yaytay CLA
Modified: 2012-07-31 02:27 EDT (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Yaytay CLA 2011-12-01 06:31:05 EST 
Build Identifier: 3.7.1

Running a BIRT report using a 36MB FlatFile datasource (500 columns, 10000 rows) takes a very long time - >30minutes.
Running SysInternals Process Monitor shows that it is reading one byte from the file every ~0.1ms (it's actually a bit quicker than that, because that would take an hour to read).
Looking in the source for org.eclipse.datatools.connectivity.oda.flatfile shows that it is using a bare FileInputStream.

Is there a good reason why this couldn't be wrapped in a BufferedInputStream?

Reproducible: Always

Steps to Reproduce:
1. Create a CSV with 500 columns and 10000 rows.
2. Add that file as a data source to a BIRT report.
3. Run the report, and use Process Monitor to track file accesses.