Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 362348 - Out of memory parsing a large binary file to check if it is a patch file
Summary: Out of memory parsing a large binary file to check if it is a patch file
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Team (show other bugs)
Version: 3.5.2   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform Team Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2011-10-28 13:39 EDT by Chris McGee CLA
Modified: 2020-07-16 15:43 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chris McGee CLA 2011-10-28 13:39:56 EDT
I'm getting an out of memory exception in the eclipse patch file parser trying to parse a large binary file (not actually a patch):

java.lang.OutOfMemoryError
at java.lang.StringBuffer.toString(StringBuffer.java:790)
at org.eclipse.compare.internal.core.patch.LineReader.readLine(LineReader.java:165)
at org.eclipse.compare.internal.core.patch.PatchReader.parse(PatchReader.java:185)
at org.eclipse.compare.internal.core.patch.PatchReader.parse(PatchReader.java:105)
at org.eclipse.compare.patch.ApplyPatchOperation.internalParsePatch(ApplyPatchOperation.java:215)
at org.eclipse.compare.patch.ApplyPatchOperation.parsePatch(ApplyPatchOperation.java:99)


Instead, the parser should fail gracefully when the buffer exceeds a certain size.
Comment 1 Tomasz Zarna CLA 2011-11-02 05:48:03 EDT
(In reply to comment #0)
> the parser should fail gracefully when the buffer exceeds a certain size.

Do you have any specific size in mind? The parser can handle larger files if you adjust JVM memory settings, but it won't help much if you're trying to parse a binary file. I think a better approach would be to abort as soon as we realize we're dealing with a binary. However, you will still get in trouble if you try to parse a proper patch as large as the binary.
Comment 2 Chris McGee CLA 2011-11-02 09:35:58 EDT
Perhaps if there is an extremely large line it could be a clue that it is a binary file. Maybe it is unrealistic for a line to exceed something like 1MB.
Comment 3 Tomasz Zarna CLA 2011-11-02 09:50:31 EDT
(In reply to comment #2)
> Perhaps if there is an extremely large line it could be a clue that it is a
> binary file. Maybe it is unrealistic for a line to exceed something like 1MB.

Yup, we do a similar thing when guessing type in org.eclipse.compare.internal.CompareUIPlugin.guessType(ITypedElement). If any of the first 10 lines is longer than 1000 bytes we assume the input is not a text file.
Comment 4 Eclipse Genie CLA 2020-07-16 15:43:48 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.