Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 360882 - myString = stringLib.format(mySmallFloat.. gives wrong string
Summary: myString = stringLib.format(mySmallFloat.. gives wrong string
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: EDT (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-10-13 14:30 EDT by broy2 CLA
Modified: 2017-02-23 14:16 EST (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description broy2 CLA 2011-10-13 14:30:54 EDT 
program formatSmallFloat type BasicProgram {}
    mySmallFloat smallFloat;
    myString string;
    function main()
        mySmallFloat = 2.345678;
        myString = stringLib.format(mySmallFloat, "***.**********");
        SysLib.writeStdout("Should be **2.3456780000 > " + myString);
    end
end  

myString should be **2.3456780000
instead, it's **2.3456780910
Comment 1 Jeff Douglas CLA 2011-10-18 15:15:45 EDT 
The number formatter method in the runtime only supports bigdecimal and so the smallfloat is being converted to a bigdecimal, causing an imprecise value.

The runtime needs to have additional logic for floating point numbers.

My belief is that this is not a P1 level item, so reducing it to P3. If you disagree, then we can discuss.
Comment 2 Jeff Douglas CLA 2011-11-10 10:32:27 EST 
fixed
Comment 3 broy2 CLA 2011-11-14 14:16:54 EST 
Verified in 20111114 build.