Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 342795

Summary: Chinese(Traditional) problem in \uff1a
Product: [Technology] Babel Reporter: JHU, Li-Wei <a9926281>
Component: translationsAssignee: Kit Lo <kitlo>
Status: RESOLVED FIXED QA Contact:
Severity: trivial    
Priority: P3 CC: denis.roy, kis36205, kitlo
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
Eclipse - Traditional Chinese
none
CDT - Traditional Chinese none

Description JHU, Li-Wei CLA 2011-04-14 03:28:00 EDT
Build Identifier: M20110210-1200

when install Babel Chinese(Traditional) package
some word include ":" is display \uff1a

Reproducible: Always
Comment 1 Kit Lo CLA 2011-07-28 12:26:48 EDT
Can you provide some specific examples?
Comment 2 Bing-Yi Wong CLA 2012-02-29 21:05:35 EST
It's obvious.  Please try to use Babel yourself or at least try to google about "eclipse uff1a".  Here's a thread about the problem description, screenshots, AND the solution:

http://www.javaworld.com.tw/jute/post/view?bid=10&id=266254&sty=1&tpg=1&age=0

Seems like it's a very simple bug but just nobody want to check it in.

This kind of bugs are much more serious than they look, because as you can see, some people just teach others to give up using Babel package.
Comment 3 Kit Lo CLA 2012-03-04 19:20:01 EST
Created attachment 212042 [details]
Eclipse - Traditional Chinese

We requested specific examples last year so we could debug and fix the problem, but did not receive any response. Thanks Bing-Yi for providing the information.

As far as I understand, the forum mentioned 4 problems:
1. "\uff1a" displayed in Outline View
2. "\uff1a" displayed in Content Assist popup window
3. "\uff1a" displayed in status area
4. "Unmatched braces in the pattern" when pressing F3

The first 2 problems have been fixed in October of last year. I fixed the last 2 problems yesterday. Please try the latest integration build. Here is the link for Babel Language Packs for Indigo (Eclipse 3.7) in Tradional Chinese:

http://build.eclipse.org/technology/babel/babel_language_packs/I20120304-0400/indigo/BabelLanguagePack-eclipse-zh_TW_3.7.0.v20120304074031.zip

Please note that old builds are deleted periodically. Just search for the latest build here:

http://build.eclipse.org/technology/babel/babel_language_packs/

or wait for the official release of Babel language packs in the summer.
Comment 4 Bing-Yi Wong CLA 2012-03-05 10:53:36 EST
Hi Kit,

Thank you for the quick fix and sorry for being rude in my last comment.  I just tried on my Windows machine and didn't see any uff1a, feeling really happy now.  I'll try it on Linux machine tomorrow.
Comment 5 Bing-Yi Wong CLA 2012-03-05 22:22:27 EST
Hi Kit,
I noticed that uff1a patterns still exist in translation package for cdt.  If cdt translation is installed, uff1a patterns appear in outline view and type hierarchy view.  Not sure if more views are affected.  Could you please do a quick search & replace for all zh_TW packages?
Comment 6 Bing-Yi Wong CLA 2012-03-05 22:56:39 EST
Hi,
I just come up with a question: why do we need to translate a colon?
Comment 7 Kit Lo CLA 2012-03-06 22:03:45 EST
Created attachment 212183 [details]
CDT - Traditional Chinese

I fixed the problems in Outline View, Type Hierarchy View, and Call Hierarchy View for CDT. Please try:

http://build.eclipse.org/technology/babel/babel_language_packs/N20120306-1700/indigo/BabelLanguagePack-tools.cdt-zh_TW_3.7.0.v20120306051632.zip

To answer your question in comment 6, it has been a normal practice for Asian translators (including Traditional Chinese, Simplified Chinese, and sometimes Japanese and Korean) to translate single-byte English punctuation marks (e.g. colon and period) into double-byte punctuation marks.

Eclipse developers sometimes prefix some special symbols with the escape character (back slash) to avoid confusion. Usually the extra necessary escape character before single-byte English special symbols do not cause any harm. However, Java does not recognize the escape character before double-byte colon, and resulted in the problem we saw.
Comment 8 Kit Lo CLA 2012-03-06 22:25:06 EST
Denis, to completely fix this problem, we have to replace all occurrences of '\\:' to ':' in the translations table. (Note that the colons are double-byte colons.)

I came up with this query to review the changes before updating the table:

select value, replace(value, '\\:', ':') from translations where value like '%!\\:%' escape '!' and language_id in (8, 9, 11, 12) and is_active;

I added the "language_id in (8, 9, 11, 12)" condition to narrow the query to the 4 Asian double-byte languages. The query and update worked okay in my database. However, as we recently noticed that the Babel database is getting very big. The above query took forever on the Babel server and my ssh session eventually timed out. I couldn't review the query before the update.

Can you think of a way to refine the query or can you help perform the query and update on a console or something so that the query and update can complete without being timed out? Thanks!
Comment 9 Bing-Yi Wong CLA 2012-03-07 01:14:31 EST
Hi Kit,

I don't see any uff1a now.  Thanks.

BTW, I saw another minor bug in Project->Properties, [Run/Debug Settings], there's a {0}' pattern which should be the project name.  The bug looks like to be in the main translation package but not cdt.

And thanks about the explanation about translating a colon.
Comment 10 Kit Lo CLA 2012-03-07 01:24:22 EST
Bing-Yi,

For the "Project->Properties" problem, does it happen in English version? If it only happens when Babel Traditional Chinese language pack is used, then please open a separate bug report against Babel. Attach a screen capture will help. Thanks!
Comment 11 Bing-Yi Wong CLA 2012-03-07 09:21:41 EST
Hi Kit,
I just filed a bug as Bug 373527 for {0}' pattern.
Comment 12 Denis Roy CLA 2012-03-07 11:38:13 EST
The query took 37 seconds on the actual database server, and shows 990 rows.  If you want, hit me up on IM and I can arrange for us to share a screen from the babel server into the database server.  We can then work together to resolve this.
Comment 13 Kit Lo CLA 2012-03-07 12:36:12 EST
Thanks, Denis! The query looks good! Please apply the update to the database. (And you may delete the tmp table.) Thanks!
Comment 14 Denis Roy CLA 2012-03-07 13:03:26 EST
mysql> update translations set value = replace(value, '\\:', ':') where value like '%!\\:%' escape '!' and language_id in (8, 9, 11, 12) and is_active;
Query OK, 990 rows affected (43.74 sec)
Rows matched: 990  Changed: 990  Warnings: 0


How does it look?
Comment 15 Kit Lo CLA 2012-03-07 13:17:20 EST
Perfect! Marking defect as resolved.

All "\uff1a" strings in all language packs for all projects in Babel should be fixed in next build.