Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 356098

Summary: Ensure eglx.lang.StringLib tokenizer functions use character instead of byte lengths
Product: z_Archived Reporter: Will Smythe <smythew>
Component: EDTAssignee: Matt Heitz <mheitz>
Status: CLOSED FIXED QA Contact:
Severity: critical    
Priority: P3 CC: greer, mheitz
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:

Description Will Smythe CLA 2011-08-29 11:55:12 EDT 
Me: the token functions should be based not on byte lengths, but on character lengths. This caught me a few weeks ago, and will certainly catch other developers who will assume a character tokenizer function works off character length, not byte length.

Matt: I checked the places we use indexes, and found that the index parameter to StrLib.getNextToken was being used as a byte index not a character index in the Java runtime.  It wasn't supposed to be that way, so you can open that defect too.  I don't know if JavaScript has the same problem. 

We should keep the byte-based behavior of StrLib in eglx.rbd intact (or this will  really screw up applications that get ported from RBD/CE into EDT IDE - and it will be difficult/impossible to fix).
Comment 1 Matt Heitz CLA 2011-08-29 15:14:43 EDT 
Fixed StringLib.getNextToken, and updated eglx.rbd.StrLib's version of the function.
Comment 2 Lisa Lasher CLA 2011-11-02 10:31:37 EDT 
closing this defect for Will