Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 375700

Summary: Orion client nls bundles format cannot be consumed by Babel
Product: [ECD] Orion Reporter: Malgorzata Janczarska <malgorzata.tomczyk>
Component: ClientAssignee: Malgorzata Janczarska <malgorzata.tomczyk>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: john.arthorne, johnjbarton, ken_walker, kitlo, mamacdon, simon_kaegi, Szymon.Brandys
Version: unspecified   
Target Milestone: 0.5 RC1   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on: 376148    
Bug Blocks: 374414, 378716    
Attachments:
Description Flags
Language pack sample for Polish none

Description Malgorzata Janczarska CLA 2012-03-30 06:01:03 EDT
To make our nls bundles consumable by Babel, we should:
1. make bundle of each language the exactly the same format, differing only by translation
2. make bundles "installable" after Orion is already there, by copying files on top of the existing instance of Orion. User should be able to install later a new language without disturbing the support for other languages.
Comment 1 Malgorzata Janczarska CLA 2012-04-04 12:07:50 EDT
My proposed solution for this problem is committed to branch Bug375700. John, Ken, I would appreciate a review, see my doubts below. John B. I know you are using our editor in your application, could you verify if you application works correctly for editor from branch Bug375700.

(In reply to comment #0)
> To make our nls bundles consumable by Babel, we should:
> 1. make bundle of each language the exactly the same format, differing only by
> translation
This was easy, we can change the format of our messages in a way they will still be supported by require.js and can be translated.

> 2. make bundles "installable" after Orion is already there, by copying files on
> top of the existing instance of Orion. User should be able to install later a
> new language without disturbing the support for other languages.
This is difficult because the "master" messages file still needs a list of supported locales. From what I learned from Kit we do not have process that would add a new locale to existing file, so that I needed somehow to generate the list of locales based on what files are available in Orion. To do it without any server-site help I had two alternatives:
1. List directories in given "nls" directory and add them to supported locales
2. Find out what locale is set, check if it exists in "nls" directory and if so add it to locale list. In this case require.js does not have the full list of supported locales, but in fact it only needs information if the current locale is supported or not.
I decided not to implement number (1) because it would require to change settings of server for each nls directory - it would have to list its content. Even if I changed it for our Orion server we would have to require this for every server that uses our client or parts of it, for instance where embedded editor is used.
I implemented (2), but it involved a little hacking, see nlsutil.js for that. First I had to check for locale, also in require.js config if it's available, then I had to make a call to check if file for given locale exists. The problem is that if locale is not supported I will for sure get 404 and I don't see how could I avoid it. In worst case scenario I would have two 404's, for instance when browser locale is pt-br, we have to check for messages in "pt-br", but also for messages in "pt". This doesn't look good in console, calls also need to be synchronous.
Comment 2 John J. Barton CLA 2012-04-04 18:26:04 EDT
on branch Bug375700 I get

Failed to load resource: the server responded with a status of 405 (Method Not Allowed) http://127.0.0.2:8080/atopwi/inspector/front-end/orion/textview/nls/en-us/messages.js
Failed to load resource: the server responded with a status of 405 (Method Not Allowed) http://127.0.0.2:8080/atopwi/inspector/front-end/orion/textview/nls/en-us/messages.js
Failed to load resource: the server responded with a status of 404 (Not Found) http://127.0.0.2:8080/orion.client/bundles/org.eclipse.orion.client.editor/web/orion/textview/nls/en-us/messages.js
Failed to load resource: the server responded with a status of 404 (Not Found) http://127.0.0.2:8080/orion.client/bundles/org.eclipse.orion.client.editor/web/orion/editor/nls/en-us/messages.js
Uncaught Error: Load timeout for modules: i18n!orion/textview/nls/messages i18n!orion/editor/nls/messages orion/textview/nls/en-us/messages orion/editor/nls/en-us/messages 
http://requirejs.org/docs/errors.html#timeout :8080/MetaObject/requirejs/require.js:79
Comment 3 Malgorzata Janczarska CLA 2012-04-05 04:43:01 EDT
John, could you try now. In previous version I used not very common method HEAD, but GET should work just as well and should be supported by every server.
Comment 4 Malgorzata Janczarska CLA 2012-04-12 13:18:51 EDT
We are getting close to establishing the correct format. Hopefully we will be able to merge it on the beginning of M2.
Comment 5 Malgorzata Janczarska CLA 2012-04-13 12:22:57 EDT
Created attachment 213988 [details]
Language pack sample for Polish

This zip contains a sample of language pack. It contains one feature containing 4 plugin fragments: one per each of our plugins. Adding this feature to Orion adds Polish translation.
Comment 6 Malgorzata Janczarska CLA 2012-04-16 09:24:24 EDT
We agreed with Kit on the nls bundle and file format. Kit wants to start extracting our nls file this week, so if no objections I will merge branch Bug375700 tomorrow. I am of cause still open for any comments or suggestions.
Comment 7 Szymon Brandys CLA 2012-04-16 09:51:11 EDT
The language packs and file formats look good. Let's merge this early in M2. Do we have all strings in client bundles externalized? If not, I would raise separate bugs to ensure that all strings are properly externalized.
Comment 8 Malgorzata Janczarska CLA 2012-04-16 10:13:12 EDT
(In reply to comment #7)
> The language packs and file formats look good. Let's merge this early in M2. Do
> we have all strings in client bundles externalized? If not, I would raise
> separate bugs to ensure that all strings are properly externalized.
I will rise separate bugs for creating a tool for externalizing strings and then for using it for parts of UI.
Comment 9 Szymon Brandys CLA 2012-04-17 03:59:29 EDT
Please create a wiki page describing how our globalization works, how to externalize strings etc. People could start to externalize strings before we have the tool then.
Comment 10 Malgorzata Janczarska CLA 2012-04-17 10:25:22 EDT
I pushed changes that change the format of message files that are accepted by Kit. I will now work to describe it on a wiki page:
http://wiki.eclipse.org/Orion/Internationalization
Comment 11 Malgorzata Janczarska CLA 2012-05-08 05:49:46 EDT
I just got information from Kit that the format we created is not compatible with the newest version of CHKPII (product used for translations) that has been released May 2. They support Dojo 1.7 format, but only the most common one. Kit opened a PMR for them to support the format we are using, but since bug 378716 was opened recently this is yet another reason to change the format again. I am now thinking to create a filter in our configurator that would decorate the main messages file with a list of supported locales.
I preferred to have it on the client side, so our UI could be taken "as it is" and new language would be supported just by adding the language bundle. When I move the logic to configurator adding new language bundle will only work when our UI is deployed using the configurator. In some cases, for instance for embedded editor the deployer will have to support some other way of installing new language packs: by modifying the messages master files in language pack install or by doing it dynamically like us.
Comment 12 Malgorzata Janczarska CLA 2012-05-21 16:11:02 EDT
We established a bundle format. I am externalizing rest of the strings and we'll be able to close this bug when first language pack in the new format is verified.
Comment 13 Malgorzata Janczarska CLA 2012-05-23 06:46:15 EDT
We have to wait for the bundle generation. Moving to RC1. This bug only needs verification now.
Comment 14 Malgorzata Janczarska CLA 2012-05-30 09:12:04 EDT
(In reply to comment #13)
> We have to wait for the bundle generation. Moving to RC1. This bug only needs
> verification now.
Kit generated language packs in the new format. I installed them and they work, so we may proudly say that we have a proper language pack format established.