Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 380695

Summary: restore native launchers in equinox.starterkit.product and equinox.core features
Product: [Technology] CBI Reporter: Igor Fedorenko <igor>
Component: build helpAssignee: CBI Inbox <cbi-inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: andrea.ross, aniefer, david_williams, gunnar, igor, john.arthorne, pwebster, pwebster, thanh.ha, tjwatson
Version: 1.0   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on: 390837    
Bug Blocks: 372792    
Attachments:
Description Flags
root resources patch none

Description Igor Fedorenko CLA 2012-05-25 11:55:11 EDT
From https://bugs.eclipse.org/bugs/show_bug.cgi?id=380050#c3

> > http://git.eclipse.org/c/cbi/rt.equinox.bundles.git/commit/?h=cbinopom&id=6e10e477c66298d2c49e2b3a4ebb6541f2b09efd
> 
> This seems risky (wrong?) since it appears to remove the executable from our
> starter kits.


Need to investigate what it'd take to reintroduce win32.x86 and win32.x86_64 root files.
Comment 1 Andrea Ross CLA 2012-07-09 21:56:37 EDT
marking this bug as CBI 1.0
Comment 2 Andrea Ross CLA 2012-07-24 16:58:20 EDT
Paul, as per Igor's git comment... do you know where this comes from?
Comment 3 Paul Webster CLA 2012-07-25 08:36:12 EDT
Tom, do you know where these come from?

PW
Comment 4 Thomas Watson CLA 2012-07-25 09:00:56 EDT
Ultimately they should come from the rt.equinox.binaries repository at:

http://git.eclipse.org/c/equinox/rt.equinox.binaries.git/tree/org.eclipse.equinox.executable/bin/win32/win32

But I am not sure how the following URLs are found during the actual build:

root.win32.win32.x86=file:executables/win32.win32.x86/eclipsec.exe
root.win32.win32.x86_64=file:executables/win32.win32.x86_64/eclipsec.exe

CC'ing Andrew Niefer to see if he knows/remembers.  What also confuses me is why it is only needed for win32 binaries.  Looking at the starter kits for mac and linux seem to include the launchers without specifying them as root files in the build.properties.
Comment 5 Andrew Niefer CLA 2012-07-25 10:29:23 EDT
(In reply to comment #4)
> Ultimately they should come from the rt.equinox.binaries repository at:
> 
> http://git.eclipse.org/c/equinox/rt.equinox.binaries.git/tree/org.eclipse.equinox.executable/bin/win32/win32
> 
> But I am not sure how the following URLs are found during the actual build:
> 
> root.win32.win32.x86=file:executables/win32.win32.x86/eclipsec.exe
> root.win32.win32.x86_64=file:executables/win32.win32.x86_64/eclipsec.exe
> 
> CC'ing Andrew Niefer to see if he knows/remembers.  What also confuses me is
> why it is only needed for win32 binaries.  Looking at the starter kits for mac
> and linux seem to include the launchers without specifying them as root files
> in the build.properties.

During a PDE product build, launchers and associated root files are contributed to the product via the automatic inclusion of the org.eclipse.equinox.executable feature.  This feature contains a customBuildCallbacks.xml script which fetches the binaries from the rt.equinox.binaries.git repo at build time. The specific executables included in each platform are listed in org.eclipse.equinox.executable/target.build.properties.  For win32.x86 and win32.x86_64 this only includes the normal launcher (listed as launcher.exe).

For windows, there was never any automatic support for including the second "console" launcher (eclipsec.exe) in a product (bug 185205).  This would be why the two windows platforms are explicitly listed there to include the console launcher.   However, I'm not sure how the file:executables uri is found, there must be ant script somewhere that would copy them into the equinox.core feature.
Comment 6 Paul Webster CLA 2012-09-17 20:38:22 EDT
Thanh, we're also missing appropriate  eclipse.platform.releng.aggregator/eclipse.platform.releng.eclipsebuilder/equinox/buildConfigs copied over to tychobuilder.

eclipse.platform.releng.aggregator/eclipse.platform.releng.eclipsebuilder/equinox/buildConfigs/osgistarter.config.launcher generates org.eclipse.osgi.configuration

PW
Comment 7 Paul Webster CLA 2012-09-17 20:41:12 EDT
David, are the buildConfigs in a regular PDE build called by the build script (as opposed to being include from the master feature)?

PW
Comment 8 David Williams CLA 2012-09-17 21:04:46 EDT
(In reply to comment #7)
> David, are the buildConfigs in a regular PDE build called by the build
> script (as opposed to being include from the master feature)?
> 
> PW

Both, I think. Search for 'buildConfigs' in eclipseBuilder and you can see where they are used "directly" by ant scripts. I believe those cases are all for "packaging" ... but, maybe that's that you are after?
Comment 9 Thanh Ha CLA 2012-09-18 09:52:51 EDT
(In reply to comment #6)
> Thanh, we're also missing appropriate 
> eclipse.platform.releng.aggregator/eclipse.platform.releng.eclipsebuilder/
> equinox/buildConfigs copied over to tychobuilder.
> 
> eclipse.platform.releng.aggregator/eclipse.platform.releng.eclipsebuilder/
> equinox/buildConfigs/osgistarter.config.launcher generates
> org.eclipse.osgi.configuration

osgistarter.config.launcher and equinox.starterkit.product are now being built in tychoeclipsebuilder.

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_2_maintenance&id=785e5bf6581d9dea776ad46e9475d9406b40bf8d
Comment 10 Paul Webster CLA 2012-09-18 14:03:54 EDT
Info I found:

The root.<os>.<ws>.<arch> we need in more than one place, like the eclipse.platform.releng.tychoeclipsebuilder/rcp.config, eclipse.platform.releng.tychoeclipsebuilder/osgistarter.config.launcher, the org.eclipse.equinox.executable feature, etc.

I believe that PDE build is using the root.* to generate root file IUs that just allow eclipsec.exe (and for the equinox.executable feature, the other root.* files like libcairo-swt.so for gtk/linux/x86_64) and including a filter:

root.linux.gtk.x86_64=bin/gtk/linux/x86_64,gtk_root ->
      <filter>
        (&amp;(osgi.arch=x86_64)(osgi.os=linux)(osgi.ws=gtk))
      </filter>

Looks like that IU also contains the mapping from root.linux.gtk.x86_64.permissions.755 ->

        <instructions size='1'>
          <instruction key='install'>
            chmod(targetDir:${installFolder}, targetFile:libcairo-swt.so, permissions:755);
          </instruction>

A lot of this work is done in org.eclipse.equinox.p2.publisher.eclipse.FeaturesAction in the org.eclipse.equinox.p2.publisher.eclipse  plugin.
Comment 11 Thanh Ha CLA 2012-09-19 10:44:51 EDT
Created attachment 221250 [details]
root resources patch

Attached patch uses maven-resources-plugin to copy the necessary files from rt.equinox.binaries to allow org.eclipse.equinox.core.feature and org.eclipse.equinox.starterkit.product.feature to build.
Comment 12 Igor Fedorenko CLA 2012-09-19 10:52:26 EDT
Comment on attachment 221250 [details]
root resources patch

Although references outside of project basedir are not explicitly disallowed by maven-resources-plugin, they are certainly NOT an intended use. This is especially so for referencing resources from other git repositories. As a general rule, all project resources must come either from project scm or from remote artifact repositories.
Comment 13 Paul Webster CLA 2012-09-19 11:32:19 EDT
(In reply to comment #12)
> Although references outside of project basedir are not explicitly disallowed
> by maven-resources-plugin, they are certainly NOT an intended use. This is
> especially so for referencing resources from other git repositories. As a
> general rule, all project resources must come either from project scm or
> from remote artifact repositories.

Then we need maven code to publish these binaries (to the local maven repo?) so that maven can find them later (there are at least 4 consumers of these binaries treating them as root files).

How can this be set up?  Do we build the rt.equinox.binaries repo first and create these artifacts so maven can consume them when building these various features?  Is that the best way?

PW
Comment 14 Paul Webster CLA 2012-10-24 11:35:54 EDT
(In reply to comment #11)
> Created attachment 221250 [details]
> root resources patch

Thanh, does this still need to be applied?

PW
Comment 15 Thanh Ha CLA 2012-10-25 02:02:54 EDT
(In reply to comment #14)
> (In reply to comment #11)
> > Created attachment 221250 [details]
> > root resources patch
> 
> Thanh, does this still need to be applied?
> 
> PW

This was already applied per bug 390837
Comment 16 Paul Webster CLA 2012-10-25 11:07:59 EDT
oh, which IUs was that supposed to produce?

PW
Comment 17 Thanh Ha CLA 2012-11-02 13:34:05 EDT
(In reply to comment #16)
> oh, which IUs was that supposed to produce?
> 
> PW

It produces the IUs related to the root files for these 2 features.

org.eclipse.equinox.core.feature:

- org.eclipse.equinox.core.feature_root.win32.win32.x86-1.1.0.v20121002-1506-root.zip
- org.eclipse.equinox.core.feature_root.win32.win32.x86_64-1.1.0.v20121002-1506-root.zip


org.eclipse.equinox.starterkit.product.feature:

- org.eclipse.equinox.starterkit.product.feature_root.win32.win32.x86-1.1.0.v20121002-1506-root.zip
- org.eclipse.equinox.starterkit.product.feature_root.win32.win32.x86_64-1.1.0.v20121002-1506-root.zip
Comment 18 Gunnar Wagenknecht CLA 2013-03-25 08:43:50 EDT
I'm not sure if this is related, but I recently updated my target platform to M6 and PDE Build stopped including the native launcher executable in the resulting archive (all platforms).

The library fragments are still there, just the executable is missing.

It's a PDE Build based product build.
Comment 19 Thanh Ha CLA 2013-07-31 10:22:46 EDT
I believe these are produced now so I'm resolving this bug. Feel free to reopen if anyone disagrees.