Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 432379

Summary: Eclipse executable delivered two times in SDK
Product: [Eclipse Project] Platform Reporter: Pascal Rapicault <pascal.rapicault>
Component: RelengAssignee: David Williams <david_williams>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: daniel_megert, david_williams, tjwatson
Version: 4.4   
Target Milestone: 4.5 M6   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on: 459558    
Bug Blocks: 434596    

Description Pascal Rapicault CLA 2014-04-08 20:31:54 EDT
While investigating bug #427148, I noticed that the executable is delivered twice! Indeed if you look into the profile you will notice that an SDK install comes with with two IUs (and two Artifacts) delivering the executable!
More specifically I can see:
- org.eclipse.sdk.ide.executable.win32.win32.x86_64 (with a CU org.eclipse.sdk.ide.executable.win32.win32.x86_64.eclipse)
- org.eclipse.rcp.configuration_root.win32.win32.x86_64

Also it seems that the first IU is rebuilt everytime.
Comment 1 Pascal Rapicault CLA 2014-04-08 20:34:33 EDT
I can also confirm that our Ericsson built product (using tycho 0.19) only ship with one executable.
Comment 2 David Williams CLA 2014-04-09 08:55:31 EDT
Do you use, or have any thoughts on, "configuration units"? 

We do, and I've found "it's just not the same" in Tycho, as it was in PDE. So, not sure if our direction should be to focus on (continuing) to get rid of "configuration units" or ... focus on making it right? 

Also, where you describe what you "see" ... are you seeing that in the "installed product", would that be in artifacts.xml? Or, are seeing that "in the repo's content.xml? I wanted to check other "products" and other "platforms" and see if they were all the same in this regard.
Comment 3 Pascal Rapicault CLA 2014-04-09 10:13:23 EDT
You can get this information by looking at the most recent file in 
<install>/p2/org.eclipse.equinox.p2.engine/profileRegistry/<someName>.profile

What I do is search for eclipse.exe. This usually leads to matches in an iuProperties element, from which I can see the ID in the IU delivering it. If there is multiple matches, then we have multiple IUs delivering the exe. Then to confirm the content of the artifact, you can find the artifact in <install>\p2\org.eclipse.equinox.p2.core\cache\binary


As for trying to mimic what PDE use to do, I think that this is a waste of time. As long as we have all the right bits in place in the install folder, the shape of the metadata is not that relevant.
Comment 4 David Williams CLA 2014-04-09 12:03:14 EDT
(In reply to Pascal Rapicault from comment #3)
> You can get this information by looking at the most recent file in 
> <install>/p2/org.eclipse.equinox.p2.engine/profileRegistry/<someName>.profile
> 
> What I do is search for eclipse.exe. This usually leads to matches in an
> iuProperties element, from which I can see the ID in the IU delivering it.
> If there is multiple matches, then we have multiple IUs delivering the exe.
> Then to confirm the content of the artifact, you can find the artifact in
> <install>\p2\org.eclipse.equinox.p2.core\cache\binary
> 
> 
> As for trying to mimic what PDE use to do, I think that this is a waste of
> time. As long as we have all the right bits in place in the install folder,
> the shape of the metadata is not that relevant.

Thanks for the education. I'll definitely be taking some closer looks, as I'd noticed some of these anomalies before, but didn't realize the result was "installing executable twice". 

I'll also compare with earlier versions to see if has "always been true" (in our Tycho builds) or something accidentally introduced ... but, my guess is "always been true". 

We have this funky "configuration feature" (which, in PDE builds, was meant to make "reusable configuration units") ... but I'm not sure that's translated well to Tycho. I've tried removing it, and the darnedest thing is it "almost works" without it but (one example) then the "s390" distribution would not get an executable installed. 

And, I think, it's "primary need" right now is to produce the "delta pack" ... which, you'll recall, I've wanted to get rid of, partially for this reason. 

I'm just dumping some history here ... but may do some work in a branch, so that perhaps some p2 expert we know would help figure out "final missing pieces" ... hint hint. :) 

Again thanks for the tips.
Comment 5 David Williams CLA 2014-05-04 14:26:33 EDT
Here's what I've investigated, and learned about this issue so far. 

1. This was true in Kepler SR2 as well, so is not a regression. 

2. It is likely caused by some code where we "copy up" launchers from equinox executables directories, into "rcp.config". I do not know why we do that ... it's what I inherited, and via bug 420725 have cleaned some of it up. 

We currently do this for 'eclipse' for 4 platforms as determined by looking at pom.xml in rcp.config, and looking directly in "build area" under "rootfiles" 

./linux.gtk.s390x/eclipse
./linux.gtk.s390/eclipse
./win32.win32.x86/eclipsec.exe
./win32.win32.x86/eclipse.exe
./win32.win32.x86_64/eclipsec.exe
./win32.win32.x86_64/eclipse.exe
./macosx.cocoa.x86/Eclipse.app/Contents/MacOS/launcher
./macosx.cocoa.x86_64/Eclipse.app/Contents/MacOS/launcher

And to make things more complicated: 

A. we set permissions in build.properties files ... not sure we need to, any longer. 
B. It appears that launchers ("eclipse") end up in each of the produced "...configuration_roots...." produced in that project. 

org.eclipse.rcp.configuration_root.cocoa.macosx.x86-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.cocoa.macosx.x86_64-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.aix.ppc-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.aix.ppc64-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.hpux.ia64-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.ppc-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.ppc64-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.s390-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.s390x-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.x86-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.linux.x86_64-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.solaris.sparc-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.gtk.solaris.x86-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.win32.win32.x86-1.0.1.v20140430-0717-root.zip
org.eclipse.rcp.configuration_root.win32.win32.x86_64-1.0.1.v20140430-0717-root.zip

This might be because even though rcp.config is a feature, it contains a config.product file, and a config.p2.inf file, and part of that config.p2.inf file says 

org.eclipse.pde.build.append.launchers=true

Yet, the executables are also in the "executable feature", so my guess is those in "configuration" should not be there, at all. 

I'll try quick "test build" where I remove only the "copy up" of Windows executables ... and if that still works ... successively try and remove more and more.
Comment 6 David Williams CLA 2014-05-04 18:58:44 EDT
Just to make a note of observations as I go, blow by blow .... 

Running test build without the explicit "copy up" for windows, made no difference in the end result ... it was still "in there" twice, presumably because the rcp.config feature says, in the p2.inf file, to 
 org.eclipse.pde.build.append.launchers=true
?

And its is actually both 'eclipse.exe' and eclipsec.exe' that are duplicated. 

I noticed too that the eclpsec.exe had same check sum in both, 
 the "rcp.config binary" and the "sdk.ide.executable" binary ... 
but the eclipse.exe files did not have the same check sum (CRC), even though they are the same size. 
At first I thought maybe due to "signature", but in fact neither are signed. 
The difference appears to be that the "rcp.config" one is the "raw" one from equinox, with its "built in ICO" resource file, were as the one in sdk.ide.executable has the "newer" Tycho substituted ICO file in it. 

Not sure what to make of all this ... except it's clear I can leave out part of that unnecessary POM copy. Note, I was careful to tough "forceQualifierUpdate" since this was a "pom only" change: 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=48421ee99191d9e11c53536b95df6c64c093d435

My intuition is that we do not want any launchers/executables associated (or delivered) with "configuration" ... but, am reluctant to make that change right before our nightly I-build. (i.e. will do test build first).
Comment 7 David Williams CLA 2014-05-04 20:24:29 EDT
(In reply to David Williams from comment #6)

> My intuition is that we do not want any launchers/executables associated (or
> delivered) with "configuration" ... but, am reluctant to make that change
> right before our nightly I-build. (i.e. will do test build first).

There is one oddity in our "executables". For all the platform distributions, if you unzip, and look in p2.core/cache, they all have an "executable root" component there -- EXCEPT for s390 (and s390x). 

I know before I tried to simply remove the "configuration feature" from all our products, (and, this might have been in Kepler time frame?) and it was working fine ... until someone noticed that there was no "executable' in the s390 distribution. 

Any tips on how to debug that lack of "executable root"? 

[And, not that we have to right now ... but it's one of those "bugs" obviously exists, and may be hard to "encounter" since I doubt there's a lot of s390 users out there :) 

And in the mean time, everyone has to live with having "two executables" since the s390 platform doesn't have a spare :).
Comment 8 David Williams CLA 2014-05-20 13:33:27 EDT
We've fixed the S390(x) problem, btw. 

But I was originally going to point out another data point, in bug 434815 comment 10. 

Similar issue on "double shipping" some stuff ... don't know if it's right, or wrong, though.
Comment 9 David Williams CLA 2015-02-12 03:21:42 EST
The changes made for bug 459558 (removing rcp config feature from main products) did solve this problem. 

From tonight's N-build, 
under eclipse\p2\o.e.e.p2.core\cache\binary
from listing each blob there (via unzip -t) you can see eclipse.exe is there only once. 
= = = = = 
Archive:  org.eclipse.platform_root_4.5.0.N20150211-2000
    testing: .eclipseproduct          OK
No errors detected in compressed data of org.eclipse.platform_root_4.5.0.N20150211-2000.

Archive:  org.eclipse.rcp_root_4.5.0.N20150211-2000
    testing: readme/                  OK
    testing: readme/readme_eclipse.html   OK
No errors detected in compressed data of org.eclipse.rcp_root_4.5.0.N20150211-2000.

Archive:  org.eclipse.sdk.ide.executable.win32.win32.x86_64_4.5.0.N20150211-2000
    testing: eclipse.exe              OK
    testing: eclipsec.exe             OK
No errors detected in compressed data of org.eclipse.sdk.ide.executable.win32.win32.x86_64_4.5.0.N20150211-2000.
= =  = = = = 

From this week's I-build, not only are more "blobs" there, but two distinct ones contained eclipse.exe (and eclipsec.exe): 

= = = = = =
Archive:  org.eclipse.jdt_root_3.11.0.v20150210-1129
    testing: epl-v10.html             OK
    testing: notice.html              OK
No errors detected in compressed data of org.eclipse.jdt_root_3.11.0.v20150210-1129.

Archive:  org.eclipse.platform_root_4.5.0.v20150210-1224
    testing: .eclipseproduct          OK
No errors detected in compressed data of org.eclipse.platform_root_4.5.0.v20150210-1224.

Archive:  org.eclipse.rcp.configuration_root.win32.win32.x86_64_1.0.100.v20150204-1747
    testing: eclipse.exe              OK
    testing: eclipsec.exe             OK
No errors detected in compressed data of org.eclipse.rcp.configuration_root.win32.win32.x86_64_1.0.100.v20150204-1747.

Archive:  org.eclipse.rcp_root_4.5.0.v20150210-0800
    testing: readme/                  OK
    testing: readme/readme_eclipse.html   OK
No errors detected in compressed data of org.eclipse.rcp_root_4.5.0.v20150210-0800.

Archive:  org.eclipse.sdk.ide.executable.win32.win32.x86_64_4.5.0.I20150210-0800
    testing: eclipse.exe              OK
    testing: eclipsec.exe             OK
No errors detected in compressed data of org.eclipse.sdk.ide.executable.win32.win32.x86_64_4.5.0.I20150210-0800.