Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 254517 - XMLContentDescriber defaults to UTF-8 for Shift_JIS encoding
Summary: XMLContentDescriber defaults to UTF-8 for Shift_JIS encoding
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 3.5   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.5 M4   Edit
Assignee: Szymon Brandys CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 254504
  Show dependency tree
 
Reported: 2008-11-06 15:55 EST by Nick Sandonato CLA
Modified: 2008-11-14 07:35 EST (History)
1 user (show)

See Also:

Attachments
Shift_JIS encoded XML file (246 bytes, text/xml)
2008-11-06 15:55 EST, Nick Sandonato CLA 		no flags Details
The screenshot (72.51 KB, image/jpg)
2008-11-12 08:30 EST, Szymon Brandys CLA 		no flags Details
shift_JIS with 1.5 (41.93 KB, image/jpeg)
2008-11-12 09:05 EST, Nick Sandonato CLA 		no flags Details
Fix v01 (4.07 KB, patch)
2008-11-14 07:04 EST, Szymon Brandys CLA 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Sandonato CLA 2008-11-06 15:55:24 EST 
Created attachment 117254 [details]
Shift_JIS encoded XML file

Fixing bug 251748 seems to have introduced a problem with the XMLContentDescriber where it will set the CHARSET to UTF-8 when the encoding="Shift_JIS". It looks like this was a result of using the InputStreamReader with a charsetName="UTF-8" when the content is actually Shift_JIS.  The result of trying to do a readLine on the inputstreamreader is a MalformedInputException.

I've attached an example file.
Comment 1 Szymon Brandys CLA 2008-11-12 08:30:10 EST 
Could you double check against M3? I can open the file without errors and its encoding is properly resolved to Shift_JIS. I'm attaching the screen shot of what I see in Eclipse when the file is opened.
Comment 2 Szymon Brandys CLA 2008-11-12 08:30:57 EST 
Created attachment 117651 [details]
The screenshot
Comment 3 Nick Sandonato CLA 2008-11-12 09:05:33 EST 
Created attachment 117659 [details]
shift_JIS with 1.5

This might be a VM-dependent issue.  I launched Eclipse 3.5M3 with 1.6 and it seemed like the charset was correctly set on the file. When I relaunched Eclipse with 1.5, I got an editor part that said "Character Encoding Problems". I've attached a screenshot of what I'm seeing with the 1.5 VM.
Comment 4 Szymon Brandys CLA 2008-11-12 10:23:41 EST 
This issue is reproducible only on IBM VM 1.5. Sun VM works fine.
Comment 5 Szymon Brandys CLA 2008-11-14 07:04:23 EST 
Created attachment 117894 [details]
Fix v01
Comment 6 Szymon Brandys CLA 2008-11-14 07:35:07 EST 
Released to HEAD.