Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 312011 - Helios is missing several platforms
Summary: Helios is missing several platforms
Status: RESOLVED WONTFIX
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Cross-Project (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Thomas Hallgren CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 320408
  Show dependency tree
 
Reported: 2010-05-07 03:35 EDT by Thomas Hallgren CLA
Modified: 2011-05-23 02:38 EDT (History)
6 users (show)

See Also:


Attachments
Patch that will *revert* the change (2.48 KB, patch)
2010-05-27 13:38 EDT, Thomas Hallgren CLA
no flags Details | Diff
Patch for the amalgamation.releng.build.model plugins (998.81 KB, text/plain)
2010-06-08 00:55 EDT, Eike Stepper CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Hallgren CLA 2010-05-07 03:35:26 EDT
New platforms has been added to the platform and they are not included in Helios. The Helios aggregation is currently limited to these platforms:

win32.win32.x86
win32.win32.x86_64
macosx.cocoa.x86
macosx.cocoa.x86_64
linux.gtk.x86
linux.gtk.x86_64
linux.gtk.ppc

I think we should also add the following since they are provided by the platform:

linux.gtk.ppc64
linux.motif.x86
solaris.gtk.sparc
solaris.gtk.x86
hpux.motif.ia64_32
aix.motif.ppc
macosx.carbon.x86

Adding them will make the verification phase of the build twice as long (each platform is verified separately by the planner) but that part is very minor when the build is successful. When it's not, 99% of the errors are caught during the verification of the first platform.
Comment 1 Markus Knauer CLA 2010-05-07 05:36:08 EDT
I had a similar request last year, but for some reasons that I do not remember any more it was decided not to include them. I would like to see all the platforms included in the Helios aggregation build.
Comment 2 David Williams CLA 2010-05-09 23:52:44 EDT
I have no particular objection to this ... but also do not know who needs it, exactly. 

These other platforms are fairly rarely used, I suspect, and special purpose. 

My guess is that most "p2 repo" tasks that would need to be performed, could be done by using multiple repos, right? So, the advantage is it does a little more verification? But, who, besides the platform, actually has fragments for these variants? (And, I'm sure there are some ... I just think it should be spelled out a little). 

Another downside, I suspect, is that if for some reason we did not use the "trusted contribution" mechanism, and duplicated everything, that the central repository would grow by quite a bit more? Maybe that could be quantified? 

So, just questions for my education ... if others think there is a need for this, I have no reason to object.
Comment 3 David Williams CLA 2010-05-09 23:53:37 EDT
assigning to Thomas for resolution ... mostly to avoid large notification list.
Comment 4 Thomas Hallgren CLA 2010-05-10 11:34:18 EDT
(In reply to comment #2)
> I have no particular objection to this ... but also do not know who needs it,
> exactly. 
> 
> These other platforms are fairly rarely used, I suspect, and special purpose. 
> 
> My guess is that most "p2 repo" tasks that would need to be performed, could be
> done by using multiple repos, right? So, the advantage is it does a little more
> verification?

I think the advantage of not having to use more then one repository is a big one. Especially since it only applies to certain platforms. People have a hard time understanding why their builds produce valid output for some platform combinations but not for others. If Helios is in sync with the platform in this respect, then we have one source of confusion less.

> But, who, besides the platform, actually has fragments for these
> variants? (And, I'm sure there are some ... I just think it should be spelled
> out a little). 
>
Not sure if we have any. If we do, I'm sure they will be happy to know that all their fragments get included, not just some of them.

> Another downside, I suspect, is that if for some reason we did not use the
> "trusted contribution" mechanism, and duplicated everything, that the central
> repository would grow by quite a bit more? Maybe that could be quantified? 
> 
I don't think it will grow much. The platform specific fragments is a very small percentage. My guess is that it's even less then 1%.

> So, just questions for my education ... if others think there is a need for
> this, I have no reason to object.

Great. I'll look into this in a while then. One reason to defer it right now is that the Amalgamation model doesn't have the needed variants of the os/ws/arch. The b3 aggregator does.
Comment 5 David Williams CLA 2010-05-26 15:56:17 EDT
Thomas, I assume this is still a good idea? Can you spell out what should be changed in helios.build file?
Comment 6 Jeff McAffer CLA 2010-05-26 15:58:05 EDT
Add my vote for having all platforms.  There are a number of scenarios where people are doing cross-platform dev that make not having all the platforms challenging.  For example, if they are missing then the y have to use a completely different development process and workflow. The incremental cost of having them seems worthwhile here.
Comment 7 Thomas Hallgren CLA 2010-05-26 16:27:06 EDT
I can add this to the helios.build file. When I do, it will no longer run using the old buckyBuilder since the new values for os/ws/arch is missing from the Amalgamation Build model. If we for some reason would like to revert, then it's easy enough to do that by just reverting my change.

Is that OK with you David?
Comment 8 David Williams CLA 2010-05-26 18:10:57 EDT
yes, that's ok with me. Please attach patch, or version before and after and document those tags here so we'll know the changes made and can restore if needed. 

Much thanks.
Comment 9 Thomas Hallgren CLA 2010-05-27 13:38:46 EDT
Created attachment 170232 [details]
Patch that will *revert* the change

I added the changes to the helios.build and committed. The attached patch will revert those changes.

Before you run this, you need to update the b3.aggregator. This is because our backward compatibility layer didn't contain the needed additions.

I ran the aggregator against the current helios.build. Here's a snipped. Apparently there's some problem with cdt and solaris ...

[java] Starting planner verification [java] Verifying config win32,win32,x86...
[java] Verifying config win32,win32,x86_64...
[java] Verifying config linux,gtk,x86...
[java] Verifying config linux,motif,x86...
[java] Verifying config linux,gtk,x86_64...
[java] Verifying config linux,gtk,ppc...
[java] Verifying config linux,gtk,ppc64...
[java] Verifying config linux,gtk,s390...
[java] Verifying config linux,gtk,s390x...
[java] Verifying config solaris,gtk,sparc...
[java] Verifying config solaris,gtk,x86...
[java] Sending email to: Vivian Kong <vivkong@ca.ibm.com>,Doug Schaefer <dschaefer@rogers.com>,David Williams <david_williams@us.ibm.com> *** Using mock: filip.hrbek@cloudsmith.com *** [java] From: David Williams <david_williams@us.ibm.com> [java] Subject: [Helios] Failed for build 20100527192048 [java] Message content: The following errors occured when building Helios: [java] [java] Software being installed: all.contributed.content.feature.group 1.0.0 [java] [java] Missing requirement for filter properties ~= $0: org.eclipse.cdt.platform.feature.group 7.0.0.201005241228 requires 'org.eclipse.cdt.core.solaris [5.2.0.201005241228]' but it could not be found [java]
Comment 10 David Williams CLA 2010-05-27 15:55:59 EDT
fresh aggregator installed (still on Eclipse M7 base) ... rebuild in progress.
Comment 11 Vivian Kong CLA 2010-05-27 16:44:36 EDT
(In reply to comment #9)

The o.e.cdt.core.solaris fragment (org.eclipse.cdt.core.solaris_5.2.0.201005241228.jar) is there on our update site http://download.eclipse.org/tools/cdt/updates/helios/.  I'm not sure why it is not being picked up by the aggregator.

Here's how we specified our fragments in the cdt.build file:

<?xml version="1.0" encoding="ASCII"?>
<build:Contribution xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:build="http://www.eclipse.org/amalgam/2008/build" label="CDT">
  <contacts name="Vivian Kong" email="vivkong@ca.ibm.com"/>
  <contacts name="Doug Schaefer" email="dschaefer@rogers.com"/>
  <features id="org.eclipse.cdt.platform" version="7.0.0.201005241228" repo="//@repositories.0"/>
  <features id="org.eclipse.cdt" version="7.0.0.201005241228" repo="//@repositories.0">
    <category href="helios.build#//@categories.1"/>
  </features>
  <features id="org.eclipse.cdt.p2" version="1.0.0.201005241228" repo="//@repositories.0"/>
  <features id="org.eclipse.cdt.gnu.dsf" version="2.1.0.201005241228" repo="//@repositories.0">
    <category href="helios.build#//@categories.5"/>
  </features>
  <features id="org.eclipse.cdt.debug.ui.memory" version="2.1.0.201005241228" repo="//@repositories.0">
    <category href="helios.build#//@categories.5"/>
  </features>
  <features id="org.eclipse.cdt.launch.remote" version="6.0.0.201005241228" repo="//@repositories.0">
    <category href="helios.build#//@categories.5"/>
  </features>
  <repositories location="http://download.eclipse.org/tools/cdt/updates/helios/" label="CDT updates"/>
  <bundles id="org.eclipse.cdt.core.aix" version="5.1.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.linux" version="5.2.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.linux.ia64" version="5.1.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.linux.ppc" version="5.1.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.linux.x86" version="5.2.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.linux.x86_64" version="5.2.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.macosx" version="5.2.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.qnx" version="5.1.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.solaris" version="5.2.0.201005241228" repo="//@repositories.0"/>
  <bundles id="org.eclipse.cdt.core.win32" version="5.2.0.201005241228" repo="//@repositories.0"/>
</build:Contribution>
Comment 12 Thomas Hallgren CLA 2010-05-27 17:41:35 EDT
(In reply to comment #11)
> The o.e.cdt.core.solaris fragment
> (org.eclipse.cdt.core.solaris_5.2.0.201005241228.jar) is there on our update
> site http://download.eclipse.org/tools/cdt/updates/helios/.  I'm not sure why
> it is not being picked up by the aggregator.
> 
It is not found because its filter is more stringent then the enablement filter on the requirement.

The bundle fragment has this filter:
 (&(osgi.os=solaris)(osgi.arch=sparc))

But the requirment in the feature has only this:

 (&(osgi.os=solaris))

so when the b3.aggregator verifies solaris,gtk,x86 it finds the requirement, considers it valid, but fails to find a matching IU.
Comment 13 David Williams CLA 2010-05-28 02:33:37 EDT
> > 
> It is not found because its filter is more stringent then the enablement filter
> on the requirement.
> 
> The bundle fragment has this filter:
>  (&(osgi.os=solaris)(osgi.arch=sparc))
> 
> But the requirment in the feature has only this:
> 
>  (&(osgi.os=solaris))
> 
> so when the b3.aggregator verifies solaris,gtk,x86 it finds the requirement,
> considers it valid, but fails to find a matching IU.


So ... how could this happen? What's the fix? 

Thomas, I notice there's two related config entries: 

  <configs os="solaris" ws="gtk" arch="sparc" archiveFormat="tar"/>
  <configs os="solaris" ws="gtk" archiveFormat="tar"/>

What's the significance of having a line without "arch"? 
Does order matter? 

Thanks,
Comment 14 Thomas Hallgren CLA 2010-05-28 02:46:30 EDT
(In reply to comment #13)
> So ... how could this happen? What's the fix? 
> 
We have three possibilities:

The CDT team could change the filter in the feature to be the same as in the bundle fragment. I.e. point to solaris and sparc, not just solaris.

OR

The CDT team could provide the missing bundle fragment for solaris.gtk.x86.

OR

We skip solaris.gtk.x86 in the Helios build (this is a workaround that might lead to problems further, should anyone use that platform with the culprit feature).

> What's the significance of having a line without "arch"? 

arch="x86" is the default. It's not serialized. The same is true for ws="win32" and os="win32". For instance, the top entry that just says:

<configs/>

is the same as:

<configs os="win32" ws="win32" arch="x86"/>

> Does order matter? 

no.
Comment 15 David Williams CLA 2010-05-28 03:11:19 EDT
> For instance, the top entry that just says:
> 
> <configs/>
> 
> is the same as:
> 
> <configs os="win32" ws="win32" arch="x86"/>
> 

Oh, I'm so glad you explained that ... an empty element. very odd. I was thinking it was a typo. I should know better :) 

For right now, I've commented out the two solaris configs in order to allow builds to complete, temporarily, for right now ... and maybe a long weekend? Especially if CDT waits until their RC3 contribution to fix this ... but, if CDT "fixes early", just let me know and I'll add back those two lines. 

thanks,
Comment 16 Vivian Kong CLA 2010-05-28 15:42:32 EDT
(In reply to comment #14)
> We have three possibilities:
> 
> The CDT team could change the filter in the feature to be the same as in the
> bundle fragment. I.e. point to solaris and sparc, not just solaris.

Which feature is this?  And also where can I find this feature filter?  Thanks!
Comment 17 Thomas Hallgren CLA 2010-05-28 16:24:40 EDT
The feature name is 'org.eclipse.cdt.platform'. The filter is constructed from the <plugin> element in its feature.xml that references the fragment. It will have an attribute such as:

  os="solaris"

but lacks:

  arch="sparc"
Comment 18 Jeff McAffer CLA 2010-05-28 16:36:13 EDT
Just checking, should we expect that RC2 has all the various platforms or is this for RC3? I  just checked RC2 and there are still a mess of missing SWT fragments.
Comment 19 David Williams CLA 2010-05-28 16:51:24 EDT
(In reply to comment #18)
> Just checking, should we expect that RC2 has all the various platforms or is
> this for RC3? I  just checked RC2 and there are still a mess of missing SWT
> fragments.

Nope. Not yet. Slow down, and catch up! :)

I did just promote a "build" to 
http://download.eclipse.org/releases/staging
which is similar to RC2, and has these additional platforms (minus solaris). 
If you want to check that.
Comment 20 Vivian Kong CLA 2010-05-28 17:28:45 EDT
(In reply to comment #17)
Thanks!  I've just checked this fix in.
Comment 21 CDT Genie CLA 2010-05-28 18:23:04 EDT
*** cdt cvs genie on behalf of vkong ***
fix for  Bug 312011 -  Helios is missing several platforms

[*] feature.xml 1.26 http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cdt/all/org.eclipse.cdt.platform-feature/feature.xml?root=Tools_Project&r1=1.25&r2=1.26
Comment 22 Jeff McAffer CLA 2010-05-31 15:37:56 EDT
(In reply to comment #19)
> Nope. Not yet. Slow down, and catch up! :)
> 
> I did just promote a "build" to 
> http://download.eclipse.org/releases/staging
> which is similar to RC2, and has these additional platforms (minus solaris). 
> If you want to check that.

I do and I did.  looks good for the previously missing SWT and Launcher bits.  Thanks.
Comment 23 David Williams CLA 2010-06-01 17:36:31 EDT
Lets say "fixed". I noticed cdt recontributed, I added solaris configs back, a build succeded, and just promoted it to /releases/staging.
Comment 24 Eike Stepper CLA 2010-06-08 00:55:53 EDT
Created attachment 171366 [details]
Patch for the amalgamation.releng.build.model plugins

Not sure if this is the right place, but maybe someone is looking here. The patch adds the new enum literals to the ecore model so that the build model editor can read and write the helios.build file without problems.
Comment 25 Thomas Hallgren CLA 2010-07-20 13:37:39 EDT
I'm reopening this since we missed a couple of platforms that are included in the RCP feature but not otherwise provided. See bug 320408.
Comment 26 Thomas Hallgren CLA 2010-07-21 12:28:30 EDT
The combinations present in the org.eclipse.rcp feature but not in helios are:

macosx.carbon.ppc
macosx.cocoa.ppc
solaris.motif.sparc
qnx.photon.x86
Comment 27 David Williams CLA 2011-05-23 02:38:13 EDT
Just (re) noticed this ... and since helios, have no plans to do anything more, and assume those last few plaforms didn't get included. 

I know there's been similar discussions about Indigo, and believe as of RC1 we did, in Indigo, have all 15 platforms the Eclipse platform supports. 

I'm assuming the work around for helios, if someone needs it, is to also "point to" the specific platform repository that contains what they need.