Community
Participate
Working Groups
Build ID: M20080911-1700 Steps To Reproduce: I apologise for the fairly generic nature of this bug report, but I have been finding quite a few unexternalised strings in Eclipse projects, and was asked for bugzilla reports, so I'm trying to do them efficiently! 1. Install the Babel Pseudo English langpacks from http://www.eclipse.org/babel/downloads.php (update site: http://download.eclipse.org/technology/babel/update-site/ganymede ) 2. Run eclipse with the option "-nl en_AA" to activate the langpack 3. Open Eclipse's Preferences window and visit the above-mentioned node. Note that most of the other node names have prefixes like eclipseNNNN: to indicate the fact that they have been externalised, whereas the above-mentioned node is appearing in plain English. The same thing probably applies to any sub-nodes in the preferences tree, and in many cases the contents of the corresponding preferences pages. Please run the Source/Externalize Strings wizard, and PDE Tools/Externalize Strings, ideally against all the DTP plugins. There are bound to be other strings which need externalising, besides the strings which are visible in Preferences. More information:
I just checked in the code... The string "Data Management" as the root item for DTP preferences is in fact externalized in the plugin.properties file of org.eclipse.datatools.connectivity.ui as datatools.core.ui.preferences.data.node. This appears to be true in HEAD (for Galileo/1.7) and our 1.6.2 branch.
(In reply to comment #1) > I just checked in the code... The string "Data Management" as the root item for > DTP preferences is in fact externalized in the plugin.properties file of > org.eclipse.datatools.connectivity.ui as > datatools.core.ui.preferences.data.node. This appears to be true in HEAD (for > Galileo/1.7) and our 1.6.2 branch. > Thank you for looking into it. If I'm reading ViewCVS correctly, that version of plugin.properties went into DTP 1.6.1. My test setup is using 1.6.1.v200809191145, so the question is why that string wasn't pseudo-localised? That file does appear in the translation editor: http://babel.eclipse.org/babel/translate.php?project=datatools&version=1.6&file=org.eclipse.datatools.connectivity/plugins/org.eclipse.datatools.connectivity.ui/plugin.properties&string=datatools.core.ui.preferences.data.node So it looks as if Babel langpack generation is at fault. I just downloaded the latest datatools langpack from http://download.eclipse.org/technology/babel/babel_language_packs/, and the resulting zip only contains five plugin fragments. It should include the fragment org.eclipse.datatools.connectivity.ui.nl_en_AA, but doesn't. Reassigning to Babel for some help.
I see that hasn't actually reassigned it the way I wanted. Would you mind assigning to babel.server-inbox@eclipse.org? PS: going by the filesizes, it's not just the locale en_AA which has a problem. I don't think any of the other languages have had all their datatools fragments included either.
I figured that the problem is in the language pack generation code (generate1.php) where we use the preg_replace() function to strip the source folder name. # strip source folder (bug 221675) (org.eclipse.plugin/source_folder/org/eclipse/plugin) $pattern = '/^([a-zA-Z0-9_-]+)\.([a-zA-Z0-9_-]+)\.([a-zA-Z0-9\._-]+)(.*)\/(\1)([\.\/])(\2)([\.\/])(.*)\.properties$/i'; $replace = '${1}.${2}.${3}/${5}${6}${7}${8}${9}.properties'; $file_row['name'] = preg_replace($pattern, $replace, $file_row['name']); $file_row['name'] for the DTP file is "org.eclipse.datatools.connectivity/plugins/org.eclipse.datatools.connectivity.ui/plugin.properties" The result after the replace became "org.eclipse.datatools.connectivity/org.eclipse.datatools.connectivity.ui/plugin.properties" The translation of the "org.eclipse.datatools.connectivity.ui/plugin.properties" file went into "org.eclipse.datatools.connectivity.nl", instead of "org.eclipse.datatools.connectivity.ui.nl". I'm not very good at regex. Patch anyone?
Good catch. You know, I don't think we should keep on using regexps in there. It sounds like a scary business. That's just my 2 cents, I don't have a solution at hand.
(In reply to comment #5) > Good catch. You know, I don't think we should keep on using regexps in there. > It sounds like a scary business. It has been said that regex is the worst form of string parsing, except all the others that have been tried. (Apologies to Churchill.) But it helps to add whitespace and comments using the /x modifier (which *is* supposed to be allowed in PHP regexes). This is the spaced-out equivalent to the existing regex: /^ ([a-zA-Z0-9_-]+)\. # \1 org. ([a-zA-Z0-9_-]+)\. # \2 eclipse. ([a-zA-Z0-9\._-]+) # \3 datatools.connectivity (.*)\/ # \4 \/plugins\/ (\1) # \5 org ([\.\/]) # \6 . (\2) # \7 eclipse ([\.\/]) # \8 . (.*) # \9 datatools.connectivity.ui\/plugin \.properties # .properties $/ix Still a terrifying regex, and it looks unnecessarily complex, but without looking up the context, I don't know whether it's safe to simplify. I assume we want the replacement to be "org.eclipse.datatools.connectivity.ui/org.eclipse.datatools.connectivity.ui/plugin.properties"? In other words, the first path segment should always be the same as the second? If so, this should do the trick (I'm not sure how to concatenate the lines of the string in PHP): $pattern = '/^ ([a-zA-Z0-9_-]+)\. # \1 org. ([a-zA-Z0-9_-]+)\. # \2 eclipse. ([a-zA-Z0-9\._-]+) # \3 datatools.connectivity (.*)\/ # \4 \/plugins\/ (\1) # \5 org ([\.\/]) # \6 . (\2) # \7 eclipse ([\.\/]) # \8 . ([^\/]*) # \9 datatools.connectivity.ui \/ # / (.*) # \10 plugin \.properties # .properties $/ix'; $replace = '${5}${6}${7}${8}${9}/${5}${6}${7}${8}${9}/${10}.properties'; I haven't tested this in the PHP, but I've tested the regexes with http://www.solmetra.com/scripts/regex/index.php with exactly one test string.
No, sorry that won't do, mainly because this is yet another particular case to support. I'm not sure there is a way out of this with regexps. I also am pretty sure this won't work for external organizations that would want to bundle with Babel. And to be honest, I don't think we can keep using map files. There is a need to run the syncup script when repository locations change. There is the need to be a bit more explicit about what we consume. There is the need for the committer to be even more in control. There might be a way out with one more regexp for all I know, but I don't feel like experimenting with it.
I'll take the credit for writing that mess of a regexp. Although it probably could be simplified, I ain't gonna. I agree that all these corner cases and regexps are ugly, but we don't have any standardization of CVS/SVN plugin naming/storage conventions at Eclipse.org. It would be nice to try to implement some, but that would be like moving a mountain and it won't happen overnight. We could (should?) perhaps just leave well enough alone and simply add : if(plugin = org.eclipse.datatools) { different_regexp } else { big_regexp_to_catch_everything_else }
(In reply to comment #7) > No, sorry that won't do, mainly because this is yet another particular case to > support. I'm not sure there is a way out of this with regexps. I don't follow you, Antoine. Even though the regex comments (which start with "#") are specific to org.eclipse.datatools.connectivity.ui, I don't think the actual regex is. It should work in the general case, unless I've got my assumptions wrong. Have I?
Sean, if you submit a patch to process_map_files.php I promise we'll try it out.
Created attachment 124635 [details] Patch to use deepest org.something folder as plugin ID when generating I dug through the database export to find the files table, and I can see that my regex was indeed too specific. That's what I get for testing with exactly one string. Actually, I don't think it even worked for the string I tested it with, because I think I had the wrong goal string. But I know where the other strings come from now. So I picked a few representative cases: org.eclipse.datatools.connectivity/plugins/org.eclipse.datatools.connectivity.ui/src/org/eclipse/datatools/connectivity/ui/messages.properties org.eclipse.datatools.connectivity/plugins/org.eclipse.datatools.connectivity.ui/plugin.properties org.eclipse.compare/plugins/org.eclipse.compare/plugin.properties org.eclipse.compare/plugins/org.eclipse.compare/compare/org/eclipse/compare/internal/AddFromHistoryAction.properties common/docs/org.eclipse.wst.common.api.doc/plugin.properties org.eclipse.ui.workbench.texteditor/src/org/eclipse/ui/texteditor/RegExMessages.properties org.eclipse.ant.core/plugin.properties Now, with the regex in my patch: 1. All the plugin.properties map to: org.plugin.id/plugin.properties (Should be similar for any other properties files in plugin roots, perhaps feature.properties.) 2. All the others map to: org.plugin.id/org/package/filename.properties I think this was the intent of the original regex. My regex, like the original, essentially assumes that an org.* plugin will only contain org packages. In other words the first-level domain must match: no com.* packages in org.* plugins. I think this assumption is the only way to write a regex which can generically strip out source_folders like "src/", other than customising a path/regex for each and every plugin. It might also break if you have a plugin named src.something with a source folder called src! Unlike the original (perhaps, not sure), my regex assumes that plugin directories are always named [a-z].something, eg org.some.thing, and that the path components within a plugin (eg org/some/thing/messages.properties) never include a dot in the directory names. Pretty sure Java doesn't allow package names like that anyway. The regex basically takes the deepest org.some.thing folder it can find, and uses that as the plugin ID. Any higher-level directories such as "org.eclipse.datatools.connectivity/plugins/" are thrown away. If the pathname only has one org.some.thing folder, it is used, same as before. I have replaced [a-zA-Z0-9_-]+ with [a-z] since (a) the regex is case-insensitive, and (b) I don't think the first component of a package/plugin name would ever contain digits, underscores or dashes. Feel free to change this if you disagree. I agree that getting rid of map files, and this sort of regex, would be good, perhaps by parsing the plugin's .classpath and/or build.properties, but in the meantime I think this should help with cases like this bug. Please let me know how it goes!
I've just noticed that the CDT langpacks suffer from the same problem, since all the properties files for all CDT plugins are ending up in the one plugin: org.eclipse.cdt.nl_en. CDT seems to have a similar directory structure, eg http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/xlc/org.eclipse.cdt.errorparsers.xlc/?root=Tools_Project. The regex in my patch seems to do the right thing with it.
Many projects have extra folders near the root in CVS to organize the files. I believe the last folder with the *.* pattern (like org.eclipse.*) should be the actual plugin folder. I was going to suggest search backwards to look for the plugin folder name, but I'm not a regex expert. Sounds like Sean's regex to find the deepest org.something folder is the same idea. I tried that in my development environment. It solved the building problems for datatools, cdt, and equinox. I think this is a good patch! I'm going to check the patch in and try that on the staging server. Thanks Sean for the patch!
*** Bug 246152 has been marked as a duplicate of this bug. ***
*** Bug 268770 has been marked as a duplicate of this bug. ***
what is the current status?
Problem has been fixed. Please try the latest language packs.
I try to install "modeling.gmf" language pack for german (see Bug 268770) Following Error occurs now: An error occurred while collecting items to be installed No repository found containing: org.eclipse.babel.nls_modeling.gmf_de/org.eclipse.update.feature/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.bridge.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.bridge.ui.dashboard.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.bridge.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.codegen.edit.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.codegen.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.codegen.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.common.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.graphdef.codegen.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.graphdef.codegen.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.graphdef.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.map.edit.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.action.ide.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.action.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.printing.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.services.action.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.services.dnd.ide.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.services.dnd.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.services.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.common.ui.services.properties.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.actions.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.dnd.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.geoshapes.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.printing.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.printing.render.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.properties.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.providers.ide.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.providers.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.render.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.resources.editor.ide.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.diagram.ui.resources.editor.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.draw2d.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.draw2d.ui.render.awt.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.draw2d.ui.render.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.clipboard.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.commands.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.type.core.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.type.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.emf.ui.properties.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.gef.ui.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.notation.edit.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.notation.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.runtime.notation.providers.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.tooldef.edit.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.tooldef.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.tooling.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.validate.nl_de/osgi.bundle/3.4.0.v20090322043401 No repository found containing: org.eclipse.gmf.xpand.nl_de/osgi.bundle/3.4.0.v20090322043401
Created attachment 129620 [details] GMF with language packs Michael, I tried a fresh copy of Eclipse Ganymede with GMF. I do not have the problem. Translations for GMF are coming up okay. No messages in the log file. Did you use the Babel language pack update site, or the downloadable zip? Could you try that again with a fresh install and a new workspace? Let's open a separate bug if you have furthur problems with GMF. (I think the datatools problem reported here have been resolved.) Thanks!
With a fresh install and a new workspace everything works fine... Thank's for help
