Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 356098 - Ensure eglx.lang.StringLib tokenizer functions use character instead of byte lengths
Summary: Ensure eglx.lang.StringLib tokenizer functions use character instead of byte ...
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: EDT (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: Matt Heitz CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-29 11:55 EDT by Will Smythe CLA
Modified: 2017-02-23 14:14 EST (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Will Smythe CLA 2011-08-29 11:55:12 EDT 
Me: the token functions should be based not on byte lengths, but on character lengths. This caught me a few weeks ago, and will certainly catch other developers who will assume a character tokenizer function works off character length, not byte length.

Matt: I checked the places we use indexes, and found that the index parameter to StrLib.getNextToken was being used as a byte index not a character index in the Java runtime.  It wasn't supposed to be that way, so you can open that defect too.  I don't know if JavaScript has the same problem. 

We should keep the byte-based behavior of StrLib in eglx.rbd intact (or this will  really screw up applications that get ported from RBD/CE into EDT IDE - and it will be difficult/impossible to fix).
Comment 1 Matt Heitz CLA 2011-08-29 15:14:43 EDT 
Fixed StringLib.getNextToken, and updated eglx.rbd.StrLib's version of the function.
Comment 2 Lisa Lasher CLA 2011-11-02 10:31:37 EDT 
closing this defect for Will