This Bugzilla instance is deprecated, and most Eclipse projects now use GitHub or Eclipse GitLab. Please see the deprecation plan for details.
Bug 306576 - Externalization/translation of strings in xmi files
Summary: Externalization/translation of strings in xmi files
Status: RESOLVED FIXED
Alias: None
Product: e4
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 1.0   Edit
Hardware: PC Windows XP
: P3 normal with 1 vote (vote)
Target Milestone: 4.1   Edit
Assignee: Project Inbox CLA
QA Contact: John Arthorne CLA
URL:
Whiteboard:
Keywords:
Depends on: 330602 331010 331260
Blocks: 275658 324957
  Show dependency tree
 
Reported: 2010-03-19 15:34 EDT by John Arthorne CLA
Modified: 2011-08-15 14:23 EDT (History)
17 users (show)

See Also:


Attachments
Add local fields to contain tranlated values (114.24 KB, patch)
2010-11-04 15:40 EDT, Eric Moffatt CLA
no flags Details | Diff
TranslationService implementation (11.46 KB, text/plain)
2010-11-05 11:19 EDT, Thomas Schindl CLA
no flags Details
latest source (35.45 KB, patch)
2010-11-06 05:40 EDT, Thomas Schindl CLA
no flags Details | Diff
patch (36.74 KB, patch)
2010-11-06 06:23 EDT, Thomas Schindl CLA
no flags Details | Diff
patch with use of LocaleProvider (37.35 KB, patch)
2010-11-06 08:32 EDT, Thomas Schindl CLA
no flags Details | Diff
Patch which uses BundleLocalization-Service (44.06 KB, patch)
2010-11-13 12:45 EST, Thomas Schindl CLA
no flags Details | Diff
patch (108.21 KB, patch)
2010-11-15 16:03 EST, Thomas Schindl CLA
no flags Details | Diff
patch (111.05 KB, text/plain)
2010-11-15 16:19 EST, Thomas Schindl CLA
no flags Details
patch (136.42 KB, text/plain)
2010-11-16 16:37 EST, Thomas Schindl CLA
no flags Details
patch (170.02 KB, patch)
2010-11-20 08:27 EST, Thomas Schindl CLA
no flags Details | Diff
patch (184.00 KB, text/plain)
2010-11-20 12:20 EST, Thomas Schindl CLA
no flags Details
Start on translation consumation (15.92 KB, patch)
2010-11-21 14:25 EST, Thomas Schindl CLA
no flags Details | Diff
patch (190.71 KB, patch)
2010-11-21 14:49 EST, Thomas Schindl CLA
no flags Details | Diff
reinject when locale is updated (192.65 KB, text/plain)
2010-11-21 16:29 EST, Thomas Schindl CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description John Arthorne CLA 2010-03-19 15:34:00 EDT
Currently our e4xmi files have various hard-coded English strings that are presented to the user (in particular the "label" attributes). Is there some mechanism already defined for externalization/translation of NLS strings from these xmi files? If not, we need to define one. Creating a new xmi file from scratch for each language won't scale, and won't allow switching languages after the product is installed.
Comment 1 Eric Moffatt CLA 2010-03-19 16:07:49 EDT
Also, if we want to maintain the current ability to shut down and restart with a new locale and have the translation occur the NLS mapping would have to take place at rendering time.

What this means is that let's say we use %<label> a la PDE...then the model for a given part would *always* have '%tag' as a label and it would be the task of the stack renderer to do the translation on the fly when setting the text (note that we already have code there to handle the '*' adornment for dirty parts).

Since there are numerous places that a part's label can show up in the UI (tab, show view dialog, quick access...) we could provide a simple API like:

String translate(String key)

Conversely, we could override the generated EMF code for 'getLabel' to automatically apply the translation, meaning that all the other code wouldn't have to change.
Comment 2 Oleg Besedin CLA 2010-04-12 13:32:22 EDT
Few more things to consider:
- performance (both memory and CPU): in 3.x we only do translation once, at the first application start up. (Barring special things like bundles being installed or using "-nl" command line switch)
- the popular request is to be able to change UI language on the fly, without Eclipse restart
- another use-case to consider is a server-based processing where a single app provides UI in multiple languages.
Comment 3 Ed Merks CLA 2010-04-12 13:34:15 EDT
I think messing with the model itself is a bad idea.  After all, the question will remain: what's actually serialized?  Of course the getter better return that value, at least reflectively. It seems the most sensible for the renderer to be responsible for rendering in a suitable locale.
Comment 4 Paul Webster CLA 2010-04-12 14:50:11 EDT
From Kenn Hussey:

Yeah, I realized that was probably the case after I sent the message. In that case, you're looking for something similar to what we've done in UML2 for profiles/stereotypes (which are essentially dynamic EMF models). There, we've implemented a mechanism that does dynamic look-up of strings in .properties files that are co-located with the model, possibily in a plug-in/fragment. If you're interested, check out the source for org.eclipse.uml2.common.util.UML2Util#getString(...).
Comment 5 Ralf Sternberg CLA 2010-04-16 05:49:50 EDT
From the user perspective, this case is similar to the translation of strings in plugin.xml files, thus the solution should be close to the multi-locale support for the extension registry (bug 244468), i.e. the translation should obtain the current locale from the LocaleProvider service and if the translated strings are cached, it should respect the multiLocale parameter to be able to optimize the single-locale case.
Comment 6 Thomas Schindl CLA 2010-07-13 08:41:56 EDT
some random thoughts:
a) we need to take into account that the user has access to the Label-Attribute 
   and can modify at runtime
b) we could apply translations when the application model is constructed 
   (probably even after the delta has been applied?) and whenever the user 
   switches the language

I think a and b require us to have an extra attribute which holds the orginal key as transient value so that we can reapply language changes after the application has started.

This is naturally NOT needed if we decide that the translation is handled in the renderers.
Comment 7 Eric Moffatt CLA 2010-09-22 11:06:50 EDT
We could add a number of transient attributes to the model (one for each of Label, TTip, Image & axxessibilityPhrase) but this seems a bit heavyweight for folks that don't need NLS.

See bug 318866 for a non-NLS example of where the part can change its image *without* wanting to lose the original URI.
Comment 8 Eric Moffatt CLA 2010-09-22 11:10:50 EDT
There are actually two different mechanisms used by the 3.x version of eclipse (and the current compatibility layer):

    * Classes that extend the org.eclipse.osgi.util.NLS class: For eclipse this is used to define the 'static' strings used to construct dialogs or displayable error messages...This is a proven, viable technique and could easily be adopted by e4 developers.
    * PDE support: This is more appilcable to the compatibility layer since it's already based on using extension points. 

Our translation discussions are mostly about replacing the second form of use but we shouldn't forget that the current WorkbenchMessages approach is available in e4 without change. This mechanism is specifically designed to minimize the memory footprint and access for large sets of translatable strings...
Comment 9 Thomas Schindl CLA 2010-10-27 17:36:34 EDT
I've had time this week to think about bit about this. I've already been experminenting a bit on this in my SimpleIDE and here's my take on it:

First of all I think the translation is a "decoration" process done by the renderer.

a) We need a Translation infrastructure:
----------------------------------------
public interface TranslationService {
  public void translate(String category, String key);
  public void translate(String category, String key, Locale locale);

  // Using DS we the category is provided through the DS-XML so the we
  // can create the real instances on demand
  public void addTranslationProvider(String category, TranslationProvider provider);
  public void removeTranslationProvider(String category, TranslationProvider provider);
}

public interface TranslationProvider {
  public void translate(String key);
  public void translate(String key, Locale locale);
}

The thing here is that it is completely up to the TranslationProvider whether they store the translations permentantly in memory, ... . And it naturally flipping the language (at least from the model side) a dynamic process (probably we should introduce a EventBus-Event which informs the renderers to update their values)

b) i18n-model attributes
------------------------
All attributes in our model who allow must be analyzed by the renderer and send to the translation service. The attribute format of a to translated element looks like this:

%(.+):(.+) e.g. %my.translation.category:firstname

because the translation is passed on to our TranslationService people can modify the attribute of our model elements and the look up is going to their own translation service.

c) Dynamic values:
------------------
The resulting value from the translation service has to support dynamic values. I'm not 100% sure yet but I think it would make sense probably of the lookup is going to the IEclipseContext of the UI-Element. So the return value of the translation service could be:

Changes since ${mycontextVar,short,date}

and the mycontextVar is looked up in the IEclipseContext and the label updates itself when the myContextVar changes.


One final question do people think this TranslationService stuff could be pushed downwards into Equinox, is there anything going on in the OSGi-Specs when it comes to translations?
Comment 10 Oleg Besedin CLA 2010-10-28 10:03:23 EDT
(In reply to comment #9)
> First of all I think the translation is a "decoration" process done by the
> renderer.
> ...

Performance is the issue here, that was the driving force for NLS and plugin XML handing we have in 3.x. In 3.x translation is done once on the first startup of the install. (Unless bundles change.) To get comparable performance, we'll need to follow that pattern.
Comment 11 Thomas Schindl CLA 2010-10-28 11:12:12 EDT
(In reply to comment #10)
> (In reply to comment #9)
> > First of all I think the translation is a "decoration" process done by the
> > renderer.
> > ...
> 
> Performance is the issue here, that was the driving force for NLS and plugin
> XML handing we have in 3.x. In 3.x translation is done once on the first
> startup of the install. (Unless bundles change.) To get comparable performance,
> we'll need to follow that pattern.

So what do you propose? To translate the keys when they are first used and then set back to the model? It's naturally an option but when I take a look at the number of translated labels when the my workbench comes up and who need to displayed initially its not more than 50 (I counted the Top-Menu, the View-Headers).

Please note that things comeing from e.g. Extension Point translations will only go through one additional method call because of the proposal so IMHO the advantages outweight this by far.

How would you translate those labels? On first startup and then store them in the model which makes the delta bigger? 

What happens if the user changes the language in between? 
Can the user flip the language of the application while running? 

If the workbench thinks it has no need for this it can simply replace all labels at startup and then no decoration has to be done by the renderer, e.g. the compat layer could do that easily.
Comment 12 Thomas Schindl CLA 2010-10-28 11:25:40 EDT
One more note maybe. 

I don't want to make the current translation stuff bad but see possibilities to improve. E.g. why do I hold the Translations of e.g. Dialogs-Resistent in memory only because we opened the dialog hours ago? 

Why do we load many translations not even needed in our descriptors because the workbench parts are not (yet) shown, ...?

As said in the other comment people can easily opt out from the "decoration" process by doing what Oleg suggest, simply replacing the values on startup - but don't forget the delta apply overhead which might hit you.
Comment 13 Eric Moffatt CLA 2010-11-04 15:40:53 EDT
Created attachment 182417 [details]
Add local fields to contain tranlated values


Also allows PerspectiveStacks to be PSC elements.

These fields will be used to reference the translated strings/icons...
Comment 14 Eric Moffatt CLA 2010-11-04 15:42:08 EDT
Committed in >20101104. Applied the patch.

This is to get the model changes in to prepare for the translation work.
Comment 15 Thomas Schindl CLA 2010-11-05 04:42:49 EDT
Why are those local-fields needed? I'd really like to discuss the way we address this. May proposal would not need this but it looks like you are heading in a different direction.
Comment 16 Paul Webster CLA 2010-11-05 07:37:08 EDT
I'm interested to see how we can support bug 275658

PW
Comment 17 Thomas Schindl CLA 2010-11-05 07:55:20 EDT
(In reply to comment #16)
> I'm interested to see how we can support bug 275658
> 
> PW

My take on it would be to send out an event on EventBroker which informs about a dynamic language switch. On the Eclipse 4.0 front this means our renderers have to listen for it and retranslate elements.

The second problem is that the inner parts (View/Editor) have to do the same. Until 226340 is not fixed this 2nd problem could not be supported but at least the things under control of Eclipse 4.0 Application Platform are prepared.
Comment 18 Thomas Schindl CLA 2010-11-05 10:09:03 EDT
Eric, asked me about the code i tested out in my SimpleIDE. Though I don't think the approach I've taken there you can see what I already tried.

The classes in question are:
* org.eclipse.e4.demo.simpleide.services.INLSLookupFactoryService
* org.eclipse.e4.demo.simpleide.internal.NLSLookupFactoryServiceImpl (made 
  available as OSGI-Service)

The key idea was that people provide an interface like (org.eclipse.e4.demo.simpleide.handlers.Messages) and the NLSLookupFactory creates a dynamic proxy, where method name is simply the key in .properties-File.

Beside that the system provided support to use annotations e.g. define that the lookup instance should be cached for reuse, ... .

Though the concept used there is powerful I'm not 100% it is the way to go. I think the approach i outlined in comment 9 is better.
Comment 19 Thomas Schindl CLA 2010-11-05 11:19:15 EDT
Created attachment 182484 [details]
TranslationService implementation

patch with:
* ITranslationService + Implementation
* ITranslationProvider
* Tweaked StackRenderer to use TranslationService

I'm currently using a String for the translation key. Would it be better to use an int? Next thing would be to tweak the compat layer to use the "%value" from the plugin.xml when creating the model elements and register an appropriate translation-provider.
Comment 20 Thomas Schindl CLA 2010-11-05 11:25:48 EDT
I'm still not sure about those new attributes. 

They are a temporal storage for the translation key - hence transient. It only makes sense to have them in the Ecore if we expect the developer to interface with them - something i don't see at the moment. So I think we could simply add those to our implementation class without exposing them in the .ecore and the MUILabel-Interface
Comment 21 Thomas Schindl CLA 2010-11-06 05:40:38 EDT
Created attachment 182549 [details]
latest source

Services are the same than before but this shows the translation stuff applied to the contacts demo where it is used in the XMI and Java code to translate.
Comment 22 Thomas Schindl CLA 2010-11-06 06:23:35 EDT
Created attachment 182550 [details]
patch

One more way I've been already exploring in SimpleIDE. This now adds 2 more methods allows people to create Message-Interfaces so that they don't have strings lurking around in their Java-Code.

Example:

@Inject
public MyView(ITranslationService ts) {
  this.m = ts.createTranslationInstance("mycat", Messages.class);

  Label l = ...
  l.setText(m.my_label()); // ts.translate("mycat","my_label");
}

You see I'm trying to explore multiple ways of doing things :-)
Comment 23 Thomas Schindl CLA 2010-11-06 08:32:06 EDT
Created attachment 182553 [details]
patch with use of LocaleProvider

This patch makes use of the LocaleProvider introduced in bug 294463 just like the Extension Registry (bug 244468)
Comment 24 Eric Moffatt CLA 2010-11-08 11:22:54 EST
Just to clarify...the extra fields are not added simply to support translation. 

They're there are part of a compatibility requirement; in 3.x-land a rendered WorkbenchPart can override the label, tooltip and icon *independently* of the information in the model (i.e. we can't over-write the info in the model because it needs to be there the next time the part needs to be rendered). Examples of this include the JUnit view (where the image changes to give a thumbnail-like 'progress' and the Problems view where the tooltip changes based on the number of errors/warnings...

For translation they *may* provide a convenient place to cache the result of a translation as well. Much of this convenience would be from the ability to have the renderers listen for changes to the 'local' fields and react as needed.
Comment 25 Eric Moffatt CLA 2010-11-08 15:16:05 EST
I'm +1 with having a Translation Service and doing all translations 'on the fly'. I also like the specific per-locale version of the API even though it would be of limited use until we get some more sophisticated scenarios.

From the various implementation examples I see that you are aware that the hard part of this is on the 'input' side; 

When and how does a new bundle tie *its* translations into the service?
How do we handle 'key' clashes between bundles ? Just tacking on the bundleId to the string would work but seems sub-optimal at best.
Comment 26 Thomas Schindl CLA 2010-11-09 13:04:10 EST
(In reply to comment #22)
> Created an attachment (id=182550) [details]
> patch
> 
> One more way I've been already exploring in SimpleIDE. This now adds 2 more
> methods allows people to create Message-Interfaces so that they don't have
> strings lurking around in their Java-Code.
> 
> Example:
> 
> @Inject
> public MyView(ITranslationService ts) {
>   this.m = ts.createTranslationInstance("mycat", Messages.class);
> 
>   Label l = ...
>   l.setText(m.my_label()); // ts.translate("mycat","my_label");
> }
> 
> You see I'm trying to explore multiple ways of doing things :-)

I'm still banging my head against the problem how to consume Translations in the business code. What I'm not happy yet is the way we use to register the TranslationProvider as an OSGI-Service which is a bit cumbersome. The @Translation-Annotation together with an extension of the DI-System can solve this for business code.

------------8<------------
@Translations(category="mycat", baseResource="/OSGI-INF/l10n/mymessages")
private Messages m;

public MyView() {

}

@PostConstruct
private void init(Composite parent) {
  Label l = ...
  l.setText(m.my_label());
}
------------8<------------

Now the question is how we could do this for our model elements. We could add a translation element in our Application-Model. So we'd add a model element:

--------8<--------
Translation {
  String category;
  String baseResource;
}
--------8<--------

In the backend this would always call out to ITranslationService APIs:
1. registerTranslationProvider(String category, ITranslationProvider provider);
2. createTranslationInstance(String category, Class<T> clazz);

I think this would follow the e4 way of doing things because our users would normally NOT talk to the ITranslationService-API. Only if they'd need something specific (e.g. a string in the none default locale, register their own things, ... ).

What do you think?
Comment 27 Thomas Schindl CLA 2010-11-10 15:33:32 EST
so that the public is also informed:

I've had a phone call with Eric today where we discuss my proposal and defined the following action items (Eric please add something in case I forgot):
* Registration of Translations is too complex (involves registering a 
  TranslationProvider through DS) we need to get closer to what we have in 3.x
  where the MANIFEST.MF - Bundle-Localization and/or the default location of 
  those informations in OSGI-INF/l10n/bundle.properties

  We somehow need to manage to integrate this standard thing into our 
  ITranslationService so that those translations are registered as an 
  ITranslationProvider

* Translation Strings in our XMI-Files. The current proposal would require a 
  developer to specify a translation like this:
  org.eclipse.e4.demo.contacts#listviewlabel
  but if you look closely we could get away with org.eclipse.e4.demo.contacts 
  if we'd somehow restore the information who contributed the application model 
  element

* Support for Field and Method base Translations: The current code only creates 
  Method-based look ups in business code. We'd like to support a similar 
  approach of the current NLS-thing but instead of using static fields we'd 
  like to use instance fields instead so that we can support multi locales at 
  once.
  
  Translation based upon interface:

  -------------8<-------------
  public interface IMessage {
   public String mykey(); 
  }

  public class MyView {

    private IMessages m;

    public MyView(Composite parent, ITranslationService s) {
       this.m = s.createTranslationInstance(IMessages.class);
       Label l = ...
       l.setText(m.mykey());
    }
  }
  -------------8<-------------

  Translation based upon class:
  -------------8<-------------
  public class Messages {
   public String mykey; 
  }

  public class MyView {

    private Messages m;

    public MyView(Composite parent, ITranslationService s) {
       this.m = s.createTranslationInstance(IMessage.class);
       Label l = ...
       l.setText(m.mykey);
    }
  }
  -------------8<-------------
Comment 28 Thomas Schindl CLA 2010-11-10 17:53:30 EST
On the default registration of Bundle-Localization and OSGI-INF/l10n/bundle I think we could do the same DS is doing. We install our workbench bundle as a listener and register a translation provider / bundle - using the same logic DS uses.

I've asked at the equinox use group if there's an API to get access to the OSGI-Translation stuff because when those are stored in OSGI-INF/l10n/bundle no Bundle-Localization-Header is found in the MANIFEST.MF
Comment 29 Eric Moffatt CLA 2010-11-11 15:19:44 EST
Here's some feedback after talking with Oleg a bit regarding the current 3.x implementation...

We can look here for examples of the current lookup strategy:

Those are internal method that do properties files lookup:

NLS: NLS.buildVariants() from org.eclipse.osgi
Registry: ManifestLocalization.getResourceBundle()

In general it would be best to allow the product integrators to keep their existing mechanism which appears to be fragment based. We should likely just do a parallel lookup for the model's '.properties file (as opposed to using the file defined in the manifest). The suggestion was to expect that, given that the application's model is called "myApp.e4xmi' that there would be a file in the same directory (bundle?) called 'myApp.properties'. If this file is not there the presumption would be that no translation is required.

Oleg, how hard would it be to rig up something that mimics our current translation's install shape (fragments...) for the contacts demo? This would give us something to test against.

When its working all the current 'override' mechanisms should work:
1) Changing the OS locale setting

2) Overriding the OS locale through the command line argument (is there a pref for this that can be set in plugincustomization ?)

In addition we can then test whether our new 'dynamic' story works as well...
Comment 30 Thomas Watson CLA 2010-11-11 15:56:47 EST
(In reply to comment #28)
> On the default registration of Bundle-Localization and OSGI-INF/l10n/bundle I
> think we could do the same DS is doing. We install our workbench bundle as a
> listener and register a translation provider / bundle - using the same logic DS
> uses.
> 
> I've asked at the equinox use group if there's an API to get access to the
> OSGI-Translation stuff because when those are stored in OSGI-INF/l10n/bundle no
> Bundle-Localization-Header is found in the MANIFEST.MF

There is a service published by the core framework that the extension registry uses the get a ResourceBundle from a bundle for a particular locale:

org.eclipse.osgi.service.localization.BundleLocalization

With the method 

public ResourceBundle getLocalization(Bundle bundle, String locale);

Isn't that what you need?  This gets the resource bundle for the properties file specified in the Bundle-Localization header.  Please don't resort to internals ;-)
Comment 31 Thomas Schindl CLA 2010-11-13 12:45:03 EST
Created attachment 183081 [details]
Patch which uses BundleLocalization-Service

This patch introduces the following things:
* support to lookup translations defined through MANIFEST.MF
  => The current logic is that if the category passed in is NOT yet known try
     if the category is the symbolic name then use the service suggested by Tom 
     Watson
  => This has the advantage that we do those things on demand and not eagerly

* support to create and register Translations it can be used like this:

  package com.mybundle.views;

  public interface ViewMessages {
   String bla();
  }


  public class MyView(Composite parent, ITranslationService service) {
   Label l ....
   ViewMessages msg = service.createTranslationInstance(ViewMessages.class);
   l.setText(msg.bla());
  }

  The translation keys are looked up in 
  com.mybundle.views.ViewMessages.properties, ...

* support to for class-based translations. The above is using interfaces but it 
  is also possible now to use classes. The difference to NLS-based translations 
  is that translations are store in instance fields

* support to customize these programmatic registration using an @Translation-
  Annotation - this is not yet working because of what ever reason I can not 
  get access to the Annotation information at runtime

Things to come:
- fixing the @Translation - anybody can point out my error?
- adding support to findout who contributed an model element
- adding support for temporal translations who stay not resident in memory but 
  are gone when the instance created is gone too
Comment 32 Thomas Schindl CLA 2010-11-15 15:38:21 EST
Uh - I found the reason my @Translation is not working. It's quite obvious once you see it. The bundle declares itself compatible with CDC-1.1, when I remove CDC-1.1 and J2SE-1.4  

Do we still keep this CDC-1.1 compability? Maybe the services bundle is not the right place for the translation work!
Comment 33 Oleg Besedin CLA 2010-11-15 15:43:59 EST
(In reply to comment #29)
> Oleg, how hard would it be to rig up something that mimics our current
> translation's install shape ...

I'll do my best to look into this and into Tom's patches in a little bit, probably in about a week - week and a half.
Comment 34 Thomas Schindl CLA 2010-11-15 16:03:29 EST
Created attachment 183165 [details]
patch

Addresses the following things:
* working @Translation (removed CDC-1.1 and J2SE-1.4)
* extended EModelService to lookup the contributor
  - currently only working for MContributions by simply reading the 
    contributionURI
  - added slot in MApplicationElement to store contributor informations

Things missing:
* Support for all none MContribution-Elements
* Add temporal support for @Translations to allow small memory footprint if 
  required
Comment 35 Thomas Schindl CLA 2010-11-15 16:19:55 EST
Created attachment 183172 [details]
patch

Fixed contribution loading and main model loading to fill the new contributor feature.

Missing:
* Temporal support for @Translation
Comment 36 Thomas Schindl CLA 2010-11-16 16:37:55 EST
Created attachment 183269 [details]
patch

This adds a test suite and fixes minor problems
Comment 37 Thomas Schindl CLA 2010-11-17 02:43:28 EST
Kai, you might be interested in this because we are using your application to experiment a bit with our API.

Next thing in the queue for your application is to switch the language dynamically :-)

While working on this I found one major problem with translations. If you are running under the default locale e.g. german and have language bundles:

mymessages.properties
mymessages_de.properties

you are unable force your the application to use English because of the way Java-ResourceBundle stuff is working. I've discovered this while working on my test suite. The problem is that the OSGI-Bundle translations work the way you'd expect them to work but the things we loading through ResourceBundle.getBundle() is not.
Comment 38 Thomas Schindl CLA 2010-11-18 14:44:38 EST
I've filed bug 330602 on equinox because their BundleLocalization service and e.g. Bundle#getHeaders() is not following the OSGI spec.

On the first look the problem seems easy but fixing it will be a major problem because today we are only shiping bundle.properties which means people who have e.g. bundle_de.properties can't e.g. access their english texts any more!

In my opinion the real problem is how ResourceBundle#getBundle() is spec'ed because it defines to fall back to default locale before going to the base resource.

This means if equinox decides to fix their problem we must also somehow publish bundle_en.properties. Everyone interested should join the equinox bug.
Comment 39 Thomas Schindl CLA 2010-11-18 14:48:31 EST
The important thing for us is to be consitent because currently our translation behave differently depending on if they come through equinox or by our own standard Java resource loading implementation.

If equinox decides to not fix BundleLocalization-Service we need to "fix" our stuff to behave incorrect but at least consitent. Anyways our ITranslationService should spec how the language look up is done.
Comment 40 Brian de Alwis CLA 2010-11-19 16:35:47 EST
I've been following this, but haven't yet had the time to give Tom's patches a whirl.  But I did want to thank Tom for pushing this forward.
Comment 41 Thomas Schindl CLA 2010-11-20 08:27:18 EST
Created attachment 183518 [details]
patch

This patch shows the full language switching at runtime in action:
* adds an event which is send out (needs improvement because currently I use the 
  EventAdmin, we'd better use IEventBroker) when the locale is switched
* language switching in demo

TODOS:
* MenuManager does not allow to update the label of the menu (Paul?)
* ITranslationService/Workbench instance and not as OSGI-Service?
Comment 42 Thomas Schindl CLA 2010-11-20 12:20:33 EST
Created attachment 183523 [details]
patch

a) ITranslationService is now created through DI so it uses the event broker
b) Made MenuManager work by setting overrides

=> The whole rendering changes are not meant to be final but to demonstrate how a full UI language update looks like. You can try it yourself. Apply the patch and launch the contacts demo. You should have a menu to change the language to german and english.

There are 2 tests failing because somehow the translation provider registration is not happening on the right order. I'm still waiting for the final equinox resolution of the localization bug so that we can spec our service API and use the same mechanism OSGi will use when it comes to english translations.
Comment 43 Thomas Schindl CLA 2010-11-21 14:25:41 EST
Created attachment 183539 [details]
Start on translation consumation

This is the start to add an DI-Extension to consume translations through DI. The main idea is the same than the for Events and Preferences:

public class MyView {
  @Inject
  @Messages
  private MyMessages messages;
}

but I've hit a blocker. I'm not able to access the translation service because I can't access the active context while the injection is done. 

Oleg can we add this information so that I can look it up in ExtendedObjectSupplier#get() - I've debugged and it is available in the PrimaryObjectSupplier so in theory we can access it is available through the IRequestor if the interface would have a method getContext()
Comment 44 Thomas Schindl CLA 2010-11-21 14:27:49 EST
> Oleg can we add this information so that I can look it up in
> ExtendedObjectSupplier#get() - I've debugged and it is available in the
> PrimaryObjectSupplier so in theory we can access it is available through the
> IRequestor if the interface would have a method getContext()

Ok. I see this not possible because IRequestor has no idea about contexts :-(
Comment 45 Thomas Schindl CLA 2010-11-21 14:49:25 EST
Created attachment 183541 [details]
patch

Ok. Found a way to get to the context but I'm using internals :-( but now one can write

-----8<-----
@Inject
public MyView(@Messages MyMessages messages) {
}
-----8<-----

I think I'm happy now with the state of consuming messages - only one little tiny thing I need to implement is that messages are reinjected when we use a instance fields for translations (reinject the message class) and naturally I really don't want to use internals to get access to context informations.
Comment 46 Thomas Schindl CLA 2010-11-21 16:29:47 EST
Created attachment 183543 [details]
reinject when locale is updated

Ok now the reinjection is also working. I think I'm done now and I'm really satisfied with the API :-)
Comment 47 Oleg Besedin CLA 2010-11-23 11:57:26 EST
(In reply to comment #46)
> Ok now the reinjection is also working. I think I'm done now and I'm really
> satisfied with the API :-)

I'd say it is a cool demo, but as a piece of "framework" software it needs a lot of work. 

1) [most importantly] dynamic language switch needs significant extra code added to consumer's side. 
For example, look at the changes to DetailComposite in the patch. 
As a general solution, this is simply a non-starter. The whole point is to make developer's life simpler; this goes into the opposite direction.

2) do we need a replaceable translation service?
Let me use a real-life analogy: my son has two cranes. One is build from Lego set, another is a regular plastic toy. The Lego set is cool as you can replace parts and, say, put it on a Lego train platform. However, the Lego crane can't lift anything and breaks every few minutes. Why? Because it uses modularity in wrong places :-).
Back to the story: I haven't seen any requests to make translation service replaceable. I have seen few requests to improve the one translation service we provide in some way or another. 
I don't think we need a replaceable translation service. Rather, we need one that works and is transparent to 99% of developers. Ideally, it will be written in a way to make replacement easier when the need arises. It also needs to be available at the service level, not workbench level.

3) [least important for now] the code needs to be more robust. For instance, I am getting an NPE when simply trying to run e4 SDK with the patch (due to tooltips missing on some tool items).

===

I'd suggest restructuring patch into 3 or 4 portions, preferably on separate enhancement requests, so that they are more manageable:

(I) model elements remember where they come from (this is important for other things, not only for translation);
(II) translate "%string" from XML files using existing Eclipse mechanisms; ideally this should allow in future to supply translation depending on context 
(III) consider ability to switch locales on the fly
(IV) consider ability to support multiple locales at the same time for server-side apps
Comment 48 Thomas Schindl CLA 2010-11-24 03:39:42 EST
(In reply to comment #47)
> (In reply to comment #46)
> > Ok now the reinjection is also working. I think I'm done now and I'm really
> > satisfied with the API :-)
> 
> I'd say it is a cool demo, but as a piece of "framework" software it needs a
> lot of work. 
> 
> 1) [most importantly] dynamic language switch needs significant extra code
> added to consumer's side. 
> For example, look at the changes to DetailComposite in the patch. 
> As a general solution, this is simply a non-starter. The whole point is to make
> developer's life simpler; this goes into the opposite direction.

The main problem is that the structure of the code in there was never written with the indention to update the labels. Once we have the core pieces in place we can think about helpers we could provide to switch the language.

If you leave out the dynamic switching for now I think the developers job got not harder than before and even less code because it uses convention over configuration.

> 
> 2) do we need a replaceable translation service?
> Let me use a real-life analogy: my son has two cranes. One is build from Lego
> set, another is a regular plastic toy. The Lego set is cool as you can replace
> parts and, say, put it on a Lego train platform. However, the Lego crane can't
> lift anything and breaks every few minutes. Why? Because it uses modularity in
> wrong places :-).
> Back to the story: I haven't seen any requests to make translation service
> replaceable. I have seen few requests to improve the one translation service we
> provide in some way or another. 

The current usage of NLS and friends is one of most pressing problems to I'd say 75% of all customers I've doing RCP consulting in the last 3-4 years. Many of them are NOT happy with the fact that translations can only come from those .property-Files and this is something I tried to 

> I don't think we need a replaceable translation service. Rather, we need one
> that works and is transparent to 99% of developers. Ideally, it will be written
> in a way to make replacement easier when the need arises. It also needs to be
> available at the service level, not workbench level.

After having read your comments I think you are right:
* TranslatioService is place at the OSGi-Level as an OSGi-Service
* LocaleService is placed at the Workbench-Level (+@Messages stuff) and 
  provided through DI

> 
> 3) [least important for now] the code needs to be more robust. For instance, I
> am getting an NPE when simply trying to run e4 SDK with the patch (due to
> tooltips missing on some tool items).

Well as stated in one of my comments. The useage of the translations has been hacked into the Renderer stuff to show case what is possible with the API.
Comment 49 Konstantin Komissarchik CLA 2010-12-02 12:01:33 EST
I admit that I haven't read through the entire thread... But I just wanted to ask if anyone considered automatic translation lookup that doesn't rely on the developer manually selecting and managing lookup keys or property files. 

In case it would be useful, here is what we do in Sapphire to solve a very similar problem...

1. The developer just writes strings in the XML file directly in their native language.

2. We have an algorithm that can take such a string and produce a unique, reproducible and reasonably compact key. Many solutions can fit here.

3. At build-time, we scan the XML files and export the master resource files. The XML files are not modified. This step is here mostly to fit into the existing translator workflows. Given sufficient tooling, this step would not be necessary.

4. At run-time, when a string is needed, we take the original string, run it through the algorithm to find the key and then lookup the key in the prop file corresponding with the desired locale.

Obviously this approach optimizes for developer time over CPU time, but we haven't seen any perf problems even on very large applications.
Comment 50 Thomas Schindl CLA 2010-12-02 12:17:19 EST
How the lookup of informations is done is completely up to you, by default we have support for .property lookups but you can register your own ITranslationProvider which uses you algorithm.

So say you have a ITranslationProvider which uses Googles' Translation API you would contribute something like this:

--------8<--------
import com.google.api.translate.Language;
import com.google.api.translate.Translate;

public GoogleTranslationProvider implements ITranslationProvider {
  public String translate(Locale locale, String category, String key) {
      Translate.setHttpReferrer("http://code.google.com/a/eclipselabs.org/p/eclipse-translator-view/");
      return Translate.execute(term, Language.AUTO_DETECT, getLang(locale));

  }

  private Language getLang(Locale locale) {
    // ...
  }
}
--------8<--------

Then you register this ITranslationProvider e.g. using DS under the category "google.translate" and then you can write in your UI-Code something like this:

@Inject
ITranslationService service

@PostConstruct
public void init(Composite comp) {
  service.translate(Locale.GERMAN,"google.translate","My name is Tom");
}


Cool isn't it :-)

Tom
Comment 51 Konstantin Komissarchik CLA 2010-12-02 12:26:52 EST
Interesting... One problem I see with this approach is that the expected input into the model is unspecified. The developer has to know which translator is going to be used in order to know whether to enter "%dialogTitle", "534322" or "My Dialog" into the model editor.
Comment 52 Thomas Schindl CLA 2010-12-02 12:53:04 EST
Well what I showed you is the possibility the core translation service will provide. 

When we talk about the model it is like this:

If the user simply enters: %myvalue the lookup is the lookup is done with a category which is equal to the contributing bundles symbolic name.

When the model element is contributed by com.mybundle, the renderer is going to call the translation service like this:

service.translate(getWorkbenchLocale(),"com.mybundle","myvalue");

and because normally no provider for com.mybundle is found it will fallback to the BundleLocalization provided by the OSGi-Bundle.

The %myvalue is identical to %com.mybundle#myvalue so when you the user wants to use the google service he/she has to specify the correct category which would be in my example "google.translate" and so the value in the file should be "google.translate#My name is Tom"
Comment 53 Thomas Schindl CLA 2011-08-15 14:23:32 EDT
I think we can close this because we shipped with translations. If someone disagrees please reopen.