Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 220717

Summary: default encoding should be UTF-8
Product: Community Reporter: Masayuki Fuse <fuse>
Component: BugzillaAssignee: Eclipse Webmaster <webmaster>
Status: CLOSED FIXED QA Contact:
Severity: minor    
Priority: P3 CC: antoine, d_a_carver, Ed.Merks, hceylan, nboldt, remy.suen
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on: 309105, 333434    
Bug Blocks: 275102, 308815    

Description Masayuki Fuse CLA 2008-02-28 04:39:19 EST
On IE7, I tried to navigate https://bugs.eclipse.org/bugs/xml.cgi?id=36651 and got "Unable to display page error"

The bug contains DBCS 論理ビュー. I guess that is not converted to XML UTF-8.
Comment 1 Denis Roy CLA 2008-02-28 10:16:21 EST
I'm not sure what I can do to fix this.  It works fine with Firefox, can you try that?
Comment 2 Masayuki Fuse CLA 2008-02-28 21:07:41 EST
I confirmed that the page was displayed in FireFox but the DBCS was garbled. It's not a vaild XML file.

I now understand why this happened. This bugzilla web page default encoding is Latin-1. When non Latin-1 character is entered, that will be garbled on web and xml file. I'd like to ask to change the default web encoding to UTF-8.
Comment 3 Denis Roy CLA 2008-08-18 11:45:58 EDT
According to the Bugzilla docs I need to run a conversion script on the bugzilla databases before this can happen.  I'll see what that involves in the hopes of getting this done for Bugzilla 3.2.
Comment 4 David Carver CLA 2008-09-26 10:24:12 EDT
*** Bug 248500 has been marked as a duplicate of this bug. ***
Comment 5 Denis Roy CLA 2009-03-03 13:36:45 EST
*** Bug 248706 has been marked as a duplicate of this bug. ***
Comment 6 Denis Roy CLA 2009-08-18 15:49:38 EDT
I'll need to apply this patch:

https://bugzilla.mozilla.org/show_bug.cgi?id=508181
Comment 7 Denis Roy CLA 2009-09-14 16:36:49 EDT
I'll recode the databases to UTF-8 this weekend.
Comment 8 Denis Roy CLA 2010-09-13 10:49:56 EDT
*** Bug 325061 has been marked as a duplicate of this bug. ***
Comment 9 Denis Roy CLA 2010-12-07 09:22:38 EST
I'll really really try to do this when I upgrade to Bugzilla 3.6
Comment 10 Denis Roy CLA 2010-12-20 13:44:32 EST
I'll attempt the recode on the sandbox database.
Comment 11 Denis Roy CLA 2010-12-22 14:45:10 EST
I installed the Encode::Detect stuff on one of the Bugzilla servers (what a pain!) and it seems like the recode has succeeded on the test instance.

However, checksetup.pl is telling me this:


We are about to convert your table storage format to UTF8. This
allows Bugzilla to correctly store and sort international characters.
if you have any non-UTF-8 data in your database,
it ***WILL BE DELETED*** by this process. So, before
you continue with checksetup.pl, if you have any non-UTF-8
data (or even if you're not sure) you should press Ctrl-C now
to interrupt checksetup.pl, and run contrib/recode.pl to make all 
the data in your database into UTF-8. You should also back up your
database before continuing. This will affect every single table
in the database, even non-Bugzilla tables.

I'm inquiring to the Bugzilla folks about this.  This does not inspire me.
Comment 12 Denis Roy CLA 2010-12-22 15:09:31 EST
"This will affect every single table in the database, even non-Bugzilla tables"

Ok, after re-reading this a few dozen times and examining the source to Mysql.pm I think I've interpreted this wrong, and I'm good to go.
Comment 13 Denis Roy CLA 2011-01-02 11:05:17 EST
Encoding is now set to UTF-8.  I'll close this as fixed; please reopen if you see something wrong.
Comment 14 Masayuki Fuse CLA 2011-01-05 02:32:10 EST
Verified the fixed. Closing. Thanks