Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 149309

Summary: [Widgets] Unable to display japanese characters in text control
Product: [Eclipse Project] Platform Reporter: Mike Tabb <mtabb>
Component: SWTAssignee: Platform-SWT-Inbox <platform-swt-inbox>
Status: CLOSED WONTFIX QA Contact: Felipe Heidrich <eclipse.felipe>
Severity: normal    
Priority: P3 CC: akurtakov, deupree, Karice_McIntyre, kitlo, Mike_Wilson, mtabb, philippe_mulet, Silenio_Quarti, snorthov
Version: 3.1.1Keywords: triaged
Target Milestone: ---   
Hardware: Other   
OS: HP-UX   
Whiteboard:
Attachments:
Description Flags
Encoded text file
none
Sample java program none

Description Mike Tabb CLA 2006-06-30 10:31:14 EDT 
2 scenarios fail in this case:
1.  The user copies and pastes from a text file into a styled text widget.  The text goes from readable in the text file to :: :: :: in the styled text widget.  It seems like the characters are being converted.  Could be a bad conversion.  Attached is a sample program that contains a styled text widget and an encoded text file to copy from.  The line that fails starts with the header Kanji3:.

2.  The user manually types in the same characters that fail in the paste , and they turn into colons as well.

This defect is preventing users from inserting this data into a database due to the invalid characters that the widget is converting to , which is why the blocker severity.  This renders the product unusable.
Comment 1 Mike Tabb CLA 2006-06-30 10:32:33 EDT 
Created attachment 45587 [details]
Encoded text file
Comment 2 Mike Tabb CLA 2006-06-30 10:33:06 EDT 
Created attachment 45588 [details]
Sample java program
Comment 3 Mike Tabb CLA 2006-06-30 10:41:00 EDT 
In addition , a copy/paste succeeds when using a non-swt motif based application.
Comment 4 Mike Tabb CLA 2006-06-30 10:44:13 EDT 
This fails when the locale is eucJP or SJIS
Comment 5 Felipe Heidrich CLA 2006-06-30 11:54:07 EDT 
what is the encoding of the file? 
does it work with Eclipse 3.2?
what is your OS version ?
here I have:
$ uname -a
HP-UX donald B.11.00 U 9000/785 2009553889 unlimited-user license

Eclipse 3.1.x is finished, we don't build it anymore.
Comment 6 Mike Tabb CLA 2006-07-05 13:34:25 EDT 
Answers to questions:

what is the encoding of the file? Unicode
does it work with Eclipse 3.2? This does not work on 3.2
what is your OS version ?  
HP-UX hpsrv99 B.11.11 U 9000/785 2007600467 unlimited-user license
Comment 7 Mike Tabb CLA 2006-07-10 11:36:41 EDT 
After doing some investigation it appears that the characters are coming in from the clipboard correctly in the local encoding.  For example if the ucs character u+5516 is copied from the html document it comes into the converter class in the mbcsToWcs() method as the following for each locale:

UTF8 - e5 94 96
SJIS - 88 a0
EUC - b0 a2

when the native call to OS.iconv() is used in the converter class it correctly converts the character to U+5516.

Eclipse code in Converter class , line 198:
  OS.iconv (cd, inBuffer, inBytesLeft, outBuffer, outBytesLeft);

However when one of the characters in question is copied xFB70 it enters into the mbcsToWcs method correctly in the local encoding for SJIS , but the call to OS.iconv() converts it incorrectly.  This character is part of the IBM extended character set.  Is it possible that HP-UX does not have support for this charset?  Just a thought.  It would make sense as to why this conversion is failing.
Comment 8 Kit Lo CLA 2006-08-03 12:40:34 EDT 
I found this technical document in HP-UX 11i Version 2 Release Notes: http://docs.hp.com/en/5990-6737/ch10s03.html

Hope it's helpful.

Also, the problem is applicable against 3.2.
Comment 9 Grant Gayed CLA 2006-08-03 12:52:03 EDT 
possibly related: bug 137621
Comment 10 Karice McIntyre CLA 2006-08-03 13:27:53 EDT 
Felipe,
A couple of questions have been posed to me because of this one, but you are the expert :-)
1. Is the conversion table used by SWT configurable (i.e. to use a custom converter) on HPUX?

2. Does the rendering of Japanese characters work better on later versions of HPUX?

Mike, 
Is it possible for you to try your test case using the latest 3.3 I-build to see what the result is?  The fix for bug 137621 (which may be related) is in these builds.
Comment 11 Steve Northover CLA 2006-09-07 17:07:54 EDT 
SWT uses iconv to convert from the platform character representation to Unicode.
Comment 12 Steve Northover CLA 2008-06-19 11:38:10 EDT 
>This character is part of the IBM extended
>character set.  Is it possible that HP-UX does
>not have support for this charset?

If iconv() can't convert a character, the other native applications that use Unicode and iconv() can't convert it either.  Felipe, can you confirm that when the offending characters are removed, the converion works?
Comment 13 Philipe Mulet CLA 2008-06-24 07:40:06 EDT 
Is this issue really a blocker ? If so, shouldn't it be planned for 3.4.1 ?
If it was only critical, it imagine it could wait until 3.5 maybe ?
In any case, if it remains blocker or critical, it should be have its target set.
Comment 14 Felipe Heidrich CLA 2008-06-24 11:05:22 EDT 
Steve: I waiting on INFRA to get access to a HP machine.
Comment 15 Felipe Heidrich CLA 2009-08-21 14:54:28 EDT 
Your bug has been moved to triage, visit http://www.eclipse.org/swt/triage.php for more info.
Comment 16 Leo Ufimtsev CLA 2017-08-03 12:37:44 EDT 
This is a one-off bulk update. (The last one in the triage migration).

Moving bugs from swt-triaged@eclipse to platform-swt-inbox@eclipse.org and adding "triaged" keyword as per new triage process:
https://wiki.eclipse.org/SWT/Devel/Triage

See Bug 518478 for details.

Tag for notification/mail filters:
@TriageBulkUpdate
Comment 17 Alexander Kurtakov CLA 2017-12-06 01:55:42 EST 
SWT no longer provides HPUX builds.