Bugzilla – Full Text Bug Listing
| Summary: | Add HPUX.ia64 build to 4.3 | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Bogdan Gheorghe <gheorghe> | ||||||
| Component: | Releng | Assignee: | David Williams <david_williams> | ||||||
| Status: | VERIFIED FIXED | QA Contact: | |||||||
| Severity: | normal | ||||||||
| Priority: | P2 | CC: | david_williams, john.arthorne, pwebster, Silenio_Quarti, tjwatson | ||||||
| Version: | 4.3 | ||||||||
| Target Milestone: | 4.3 M3 | ||||||||
| Hardware: | PC | ||||||||
| OS: | Windows 7 | ||||||||
| Whiteboard: | |||||||||
| Bug Depends on: | 392521, 392522 | ||||||||
| Bug Blocks: | |||||||||
| Attachments: |
|
||||||||
|
Description
Bogdan Gheorghe
We have also built the launcher for hpux ia64, so we are now really ready for the build. What's needed from releng? I've just now looked at this (a little) and looks like its pretty much a global replace of ia64_32 with ia64_64. In a few cases there's case issues (with the replace) and some places seemed to have "a64_32" instead of ia64_32 (but can't find them now). I just wanted to get an idea of when you say "really ready" you mean for tonight's N-build? Or, did you want to see changes to eclipseBuilder first (e.g. in a patch). This would touch "eclipsebuilder, plus several features, build.properties, and pop in releng project (where master feature is defined). I just noticed a native for file system, too: org.eclipse.core.filesystem.hpux.ia64_32 Is that changing also? I don't think I could be ready for tonight's 8 PM build. Let's plan on Thursday's? I'd like to make a feature branch, sleep on it a while, let you sanity check for anything really odd looking and then we'll go. Sound like a plan? Looking again, I think the global replace should replace ia64_32 with (merely) ia64 (instead of ia64_64). (right?) I did not see a org.eclipse.core.filesystem.hpux.ia64 fragment in core.resources repository ... so, will need to know is that work that still needs to be done? Or, will we not have a platform specific fragment for hpux 64 bit file systems and it just needs to be removed from build? And, just to be explicit, again, we are talking about producing just ONE hpux distribution, right? 64 bit, and no longer 32 bit. (Its actually easier to change the one, that to add one). I have created feature branches for 2 repos called david_williams/hp64. If you compare with master, that might help you spot any changes that "look funny" (or wrong) but I did just literally change "hp64_32" to "hp64". eclipse.platform.releng eclipse.platform.releng.eclipsebuilder Tip: when you update the map files, you can "leave in" the 32 but versions temporarily, and they won't hurt anything if they are not referenced, so we have the ability to "revert" quickly if things seem way off and hard to understand. But, I predict it will go smoothly ... once we get the file system fragment question answered (need a new one? Or omit current one?). Other work: a. tiny change in test.xml (in maps/configuration) that doesn't impact anything currently, but figure it should match. b. larger change (I'm guessing) in the eclipse.platform.releng.aggregator repository. I know that work is focused at the moment on 3.8 and 4.2 but will soon need 4.3 too so it will need to be brought up to date. (I know less about it, and haven't looked, but would assume its changes to be similar to changing hp64_32 to hp64.). A core.filesystem fragment is not strictly required - I believe there are other platforms we have shipped in the past with no core.filesystem native. We do currently have one for HPUX ia64_32, so it would be a loss of functionality if we didn't end up with one for ia64. I don't think I or any core.resources committers have time to do this in the next week or two, but we might be able to get one compiled after M3. (In reply to comment #3) > I just noticed a native for file system, too: > > org.eclipse.core.filesystem.hpux.ia64_32 > > Is that changing also? No, this is an optional component. We don't need it to run (we might add this at a later time though). (In reply to comment #5) > Looking again, I think the global replace should replace ia64_32 with > (merely) ia64 (instead of ia64_64). (right?) > Correct! > And, just to be explicit, again, we are talking about producing just ONE > hpux distribution, right? 64 bit, and no longer 32 bit. (Its actually easier > to change the one, that to add one). That is also correct. IMing with Bogdan, we (he) found some places where we used "hpux" with "motif" that should have been "hpux" with "gtk". So while it has "been working" and is unrelated to 64 bit change, we changed that too. I also noticed lots of obsolete references to "hpux" on "PA_RISC" in our build scripts, which we don't build anymore ... so I open bug 392391 and we'll clean that up (remove them) once we get this bug fixed. (In reply to comment #8) > (In reply to comment #3) > > I just noticed a native for file system, too: > > > > org.eclipse.core.filesystem.hpux.ia64_32 > > > > Is that changing also? > > No, this is an optional component. We don't need it to run (we might add > this at a later time though). Ok, sounds like it won't hurt anything leaving in the reference to it, even though not used, so will leave it in there to make it easier to change to "64 bit" version if we get around to producing one. If we ever decide NOT to provide 64 bit one, though, I'd prefer to remove it ... it will always be there in the repo if we need it :) Thanks all. I think the current plan is we will (probably) give this a try on Friday, maybe a "test build" in afternoon or something. Merged my releng branches with master and pushed. Main commits are: http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/commit/?id=1990d55bb9e27d49935f8b54afa8a12a9a05fea9 http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?id=093289e5c4cdf6f3655ba4d9f95d82964a97094b I realized I'd forgotten to "change back" the file system plugin to refer to the old, temporary hp64_32, so did that with http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?id=ddb308660079dde5ee684f162efc4a90b460c3a0 Only the first two should be reverted ... if a revert is needed. In a test build, got an error that Processing inclusion from feature org.eclipse.platform: Bundle org.eclipse.core.filesystem.hpux.ia64_32_1.0.100.N20121019-1510 failed to resolve. Probably caused by changing "build configs" (so, there is not one to match?) ... so just commented it out, for now. http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?id=afda1b130e056625360ba9107fe419eeb97f1c6f Note to self. I just realized, thinking of these "build configs", that once we get the 4.3 build settled, many (most) of 20 files "touched" in eclipseBuilder will need to become stream specific and moved to the "configuration" section of the maps project. That allows us to continue to do 32 bit version for 3.8/4.2 maintenance, but 64 bit only for 4.3 Before next Wednesday! (when M builds run). I'll re-examine some of the files/changes involved ... not all of them are needed/required any longer, but I think just 2 or three. Got pretty far this time (basically finished a build). But ... SWT standalone jar creation failed. If it helps, you can read the results that were created on the build machine, at http://build.eclipse.org/eclipse/eclipse4N/siteDirTESTONLY/eclipse/downloads/drops4/N20121019-1534/ = = = = = packageSwtStandalone: BUILD FAILED /shared/eclipse/eclipse4N/build/supportDir/org.eclipse.releng.eclipsebuilder/buildAll.xml:124: The following error occurred while executing this line: /shared/eclipse/eclipse4N/build/supportDir/org.eclipse.releng.eclipsebuilder/buildAll.xml:1173: The following error occurred while executing this line: /shared/eclipse/eclipse4N/build/supportDir/org.eclipse.releng.eclipsebuilder/eclipse/helper.xml:148: The following error occurred while executing this line: /shared/eclipse/eclipse4N/build/supportDir/org.eclipse.releng.eclipsebuilder/eclipse/helper.xml:307: Basedir /shared/eclipse/eclipse4N/build/supportDir/src/plugins/org.eclipse.swt.gtk.hpux.ia64 does not exist Total time: 68 minutes 30 seconds An error has occurred. See the log file /opt/public/eclipse/eclipse4N/workspace-eclipse4/.metadata/.log. [end] [16:44:43] runSDKBuild setting eclipse 4.3.0-N-Builds ERROR. exit code: 13 Failed while building Eclipse-SDK Created attachment 222604 [details]
patch to not include ia63_32 fragment in e4.rcp
Status: We commented out the map entry for 32 bit version thinking it might make it more obvious what was trying to get it ... but, didn't help. Same error.
I did some more global searches and found
org.eclipse.swt.gtk.hpux.ia64_32 in
org.eclipse.e4.rcp (feature)
That needs to be changed to remove the _32, as in patch.
(I am not a committer in platform.eclipse.ui so someone else will have to apply it).
(In reply to comment #16) > Created attachment 222604 [details] > patch to not include ia63_32 fragment in e4.rcp Released to master PW
> Released to master
> PW
Thanks. I restarted N-build at 9:15 ... we'll know by noon if that's what was the (only) thing causing a problem.
Just do document it, I also found some "clickthough" files in maps project that were named with ...ia64_32.txt ... I renamed those to ...ia64.txt so they'd match their download counter part.
I unziped the eclipse sdk from today's N build, N20121020-0915, and the last one with 32 bit versions, N20121018-2330, and then did find ./ -regex ".*ia64.*" It appears the launchers are missing. My guess is its related to the 'motif' vs 'gtk' changes? (i.e. missed something somewhere?) = = = = ./eclipseN20121020-0915/plugins/org.eclipse.swt.gtk.hpux.ia64.source_3.101.0.N20121020-0915.jar ./eclipseN20121020-0915/plugins/org.eclipse.swt.gtk.hpux.ia64_3.101.0.N20121020-0915.jar ./eclipseN20121020-0915/p2/org.eclipse.equinox.p2.core/cache/binary/org.eclipse.rcp.configuration_root.gtk.hpux.ia64_1.0.0.N20121020-0915 = = = = ./eclipseN20121018-2330/plugins/org.eclipse.swt.gtk.hpux.ia64_32_3.101.0.N20121018-2330.jar ./eclipseN20121018-2330/plugins/org.eclipse.swt.gtk.hpux.ia64_32.source_3.101.0.N20121018-2330.jar ./eclipseN20121018-2330/p2/org.eclipse.equinox.p2.core/cache/binary/org.eclipse.rcp.configuration_root.gtk.hpux.ia64_32_1.0.0.N20121018-2330 ./eclipseN20121018-2330/plugins/org.eclipse.core.filesystem.hpux.ia64_32_1.0.100.N20121018-2330.jar [[expected differences]] - - - - ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330 ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330/eclipse_1503.so ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330/about.html ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330/launcher.gtk.hpux.ia64_32.properties ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330/META-INF ./eclipseN20121018-2330/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.N20121018-2330/META-INF/MANIFEST.MF = = = = Found a good hypothesis ... remember that a64_32 I mentioned ... found it in build.xml <equinoxlaunchers> target where it said arch="a64_32" ... so ... guess that'd never have been working quite right? But, I changed to arch="ia64" and we'll see if that is the (only) problem. Will kill off build at 5:30 (even though another is scheduled for 8:00 ... I'll likely remove/defer that 8:00 one still we see the results of the 5:30 one.
> Will kill off
==> Will kick off
FYI, the "a64_32" typo in equinox/buildConfigs/equinox-launchers/build.xml was in a a file that was part of the original commit (just unnoticed): http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/diff/equinox/buildConfigs/equinox-launchers/build.xml?id=1990d55bb9e27d49935f8b54afa8a12a9a05fea9 So, even though this file will become part of the "split stream" configuration in maps project (that is, the 32 bit version should continue to work as is) I think it is still best to fix that typo in maintenance streams. That to be tracked in bug 392514. (In reply to comment #20) > Found a good hypothesis ... remember that a64_32 I mentioned ... found it in > build.xml <equinoxlaunchers> target where it said > arch="a64_32" ... so ... guess that'd never have been working quite right? > > But, I changed to > arch="ia64" > and we'll see if that is the (only) problem. > > Will kick off build at 5:30 (even though another is scheduled for 8:00 ... > I'll likely remove/defer that 8:00 one still we see the results of the 5:30 > one. Nope. 5:30 build no different. hpux launchers till missing.
>
> Nope. 5:30 build no different. hpux launchers till missing.
If you like the blow-by-blow, on the build machine itself, it appears one the 32 bit versions being "produced".
= = = =
[20:19:30] david_williams@build:/opt/public/eclipse/eclipse4N/build/supportDir/src
$ find ./ -name org.eclipse.equinox.launcher*ia64*
./plugins/org.eclipse.equinox.launcher.win32.win32.ia64
./plugins/org.eclipse.equinox.launcher.motif.hpux.ia64_32
./plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32
Created attachment 222612 [details]
patch for equinox feature to include correct fragment
guess it'd help if equinox said what to include.
(BTW, I'd recommending removing the "old stuff" in the feature.xml no longer used, like motif.hpux ... but ... my patch only includes the minimum to fix the issue at hand).
(In reply to comment #25) > Created attachment 222612 [details] > patch for equinox feature to include correct fragment > > guess it'd help if equinox said what to include. > > (BTW, I'd recommending removing the "old stuff" in the feature.xml no longer > used, like motif.hpux ... but ... my patch only includes the minimum to fix > the issue at hand). There are other files in that project that may need changing. Such as there is a target.build.properties that has root.hpux.motif.PA_RISC=bin/motif/hpux/PA_RISC root.hpux.motif.ia64_32=bin/motif/hpux/ia64_32 root.hpux.gtk.ia64_32=bin/gtk/hpux/ia64_32 But, not sure these are used? I'm just saying, someone who know what this project does will likely need to take a look at the whole thing: org.eclipse.equinox.executable While I wait for org.eclipse.equinox.executable to be fixed, I'm going to temporarily revert the changes made to eclipse builder in comment 12. I've been through the list of 21 files, and I'd stay approximately a third of them are not required (e.g file is no longer required), 1/3rd of them are debatable or unknown, and 1/3rd of them will require "split stream" or special handling. So, I'd like to revert, open bugs for the removal of not required files, and other changes not directly related to "building the right thing", then make a commit with the changes specific to making hpux ia64 build. It will be a cleaner history that way? (And, easier to independently revert some of the other changes I make, in case I error). I'll attach longer analysis of each file, once done. [There could be better ways to do this, with git ... but, not that I know.] Of the 21 files originally changed in eclipseBuilder, the following list shows their latest status. This was primarily done thinking of the easiest/safest way to "split stream" eclipseBuilder. There is still a question about some (the "build.properties" type files) But, I've made enough of the changes I'm going to start another N-build shortly after midnight, 0015 on 10/21. I'll make sure these changes produce same "N-build" results before actually finishing the split streaming. = = = = Legend: V: handled by using stream-specific variable (bug 392521) S: hanlded with split-stream files (bug 392522) x: removed the need to change (deleted files, obsolete code, etc.) (bug listed for each case) ?: Not sure yet ... I think maybe we do not need but ... will learn by trying with/without. V 1. buildAll.xml Required to addess. But the "meat" of the whole build it is less desireable to split stream. Might be best to handle with simple variable? (Which is easy/feasible here, since only used as value to a variable: arch="ia64" (or arch="ia64_32)". (bug 392521) ? 2. eclipse/buildConfigs/master/build.properties config property does not appear used will try simply changing to default of configs = *, *, * ? 3. eclipse/buildConfigs/org.eclipse.equinox.executable/build.properties ditto (same as 2) ? 4. eclipse/buildConfigs/pde.build.config.launcher/build.properties Not sure. looks important, but I could not find where used? Perhaps left over and now performed via p2 oriented functions elsewhere? ? 5. eclipse/buildConfigs/rcp.config/build.properties ditto (same as 4) x 6. eclipse/buildConfigs/sdk.tests/build.properties Not reqruied. Comment only, will remove the whole comment (but leave default of configs=*,*,*) (bug 392518) x 7. eclipse/buildConfigs/sdk.tests/testScripts/runtests.sh Part of a check if "valid configuration to test" ... Honestly, I don't think this check is very valuable? IMHO, not worth spliting streams over, and easier to remove whole "if" clause. (bug 392516) x 8. eclipse/buildConfigs/sdk.tests/testScripts/runtestsmac.sh ditto (same as 7) (bug 392516) ? 9. eclipse/buildConfigs/sdk/build.properties similar to 2 and 3, but in addition to "configs" mentions "archivesFormat". Appears not to be used? S 10. eclipse/buildConfigs/sdk/customTargets.xml Required. Required to split stream, since part of assemble.org.eclipse.sdk... definition (part of target name, not mere value of a variable). x 11. eclipse/buildConfigs/sdk/srcBuild/build Nothing in srcBuild required. (bug 377764) V 12. eclipse/helper.xml This file (pretty sure) no longer used anywhere, being replaced by "helpernew". The one place its referenced is <property name="sdkHelper" location="${eclipse.build.configs}/eclipse/helper.xml" /> and I could find no (other) occurance of "sdkHelper". Whoops. It is used. In fact it is this one that provide buildStandAloneSWT ... assuming its used any longer. I'll sort out helper and helpernew later, but need to end up with just one, I'd think (will address it later in bug 392520, but both it and helpernew.xml will be "fixed" by some simple variable, rather than split streams. (bug 392521) V 13. eclipse/helpernew.xml Required. Part of packageSwtStandalone code. May handle with simple variable check rather than split streaming? (bug 392521) S 14. eclipse/publishingFiles/testManifest.xml Required. Data for webpages. Easiest to split stream? V 15. equinox/buildConfigs/equinox-launchers/build.xml Requried. Could be split, or could be handled with simple variable? (bug 392521) S 16. equinox/buildConfigs/equinox.prov/run.xml S Required. Should be split. (Could be done with variable, but in some cases would change "logic" since in some places the name itself is part of a variable name ... if we ever needed to do _both_ it would no longer be correct ... and would not follow same pattern as others. ? 17. equinox/buildConfigs/osgistarter.config.launcher/build.properties Not sure. Similar to 4. Appears old style ... but ... hard to say. S 18. equinox/buildConfigs/osgistarter.config.launcher/buildConfiguration.xml Required for osgistarter kit. Appears required to split, as written (the whole method/logic looks like it could be done better ... but ... easiest to split). S 19. equinox/publishingFiles/testManifest.xml Required. Web page data for equinox. x 20. scripts/process-publish-build.sh Not requried. This script not used by us any longer. Will remove it. Looks to be part of an initial "refactoring" attempt that wasn't finished (or, which I missed). (bug 392519) x 21. scripts/sdk/testsummaries/out.txt Not required (to change). This is simple test data for (or from) some test. I will as least remove "matching lines" so it won't match in future ... but it should be "gitignore'd" if it is "output from a local test". (bug 392519) I did have to make the changes in the build.properties files (not sure all were _required_ bug I changed them anyway. This restored the build to "normal", as of N20121021-0230, meaning swt fragments are there, but still waiting for the equinox fragments feature to be patched so that launcher fragments are included, see comment 25. (In reply to comment #29) > I did have to make the changes in the build.properties files (not sure all > were _required_ bug I changed them anyway. > > This restored the build to "normal", as of N20121021-0230, > meaning swt fragments are there, but still waiting for the equinox fragments > feature to be patched so that launcher fragments are included, see comment > 25. I went ahead and released this patch. Please let me know if anything else is needed to do a test build today. (In reply to comment #30) > I went ahead and released this patch. Please let me know if anything else > is needed to do a test build today. Thanks Tom, I scheduled and extra N build for 10 AM this morning. We should know by about noon how it goes. The latest N build, http://download.eclipse.org/eclipse/downloads/drops4/N20121022-1000/ does have the *ia64* launchers, so I think we can say the "build is working" ... now it will be up to others to say if it is building the right stuff! (i.e. does it actually work on an hpux ia64 machine). I'll open a few follow up bugs, but I think this one is done. I think the releng part of this is done, for now. I did (just now) commit/pushed the insignificant change to test.xml mentioned in comment 6 just so it is correct in master (even though we don't actually use that section of code at the moment, and looks wrong for maintenance streams anyway). I also opened two follow up bugs for work that will have to be done at some point, but think we are set for now ... if the code actually works. Follow up bugs: Bug 392587 - provide org.eclipse.core.filesystem.hpux.ia64 fragment Bug 392590 - adapt master of eclipse.platform.releng.aggregator for hpux ia64
> I also opened two follow up bugs for work that will have to be done at some
> point, but think we are set for now ... if the code actually works.
>
We tested out the 2200-1000 build and everything worked well. I've updated the hpux clickthrough with some new steps needed to start up Eclipse.
Thanks for your help David!
With today's M-build, I confirmed they did still have the ia64_32 bit fragments, at least by their names ... so will consider this 'verified' (naturally, others will have to confirm they "run right" but seems pretty likely). 3.8.2 M-build ./eclipse-SDK-M20121024-1400-hpux-gtk-ia64_32 .../eclipse/p2/org.eclipse.equinox.p2.core/cache/binary/org.eclipse.rcp.configuration_root.gtk.hpux.ia64_32_1.0.0.M20121024-1400 .../eclipse/plugins/org.eclipse.core.filesystem.hpux.ia64_32_1.0.100.v20120913-135826.jar .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.v20120913-144808 .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.v20120913-144808/launcher.gtk.hpux.ia64_32.properties .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64_32_3.8.1.v3835b.jar .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64_32.source_3.8.1.v3835b.jar 4.2.2 M-build ./eclipse-SDK-M20121024-1600-hpux-gtk-ia64_32 .../eclipse/p2/org.eclipse.equinox.p2.core/cache/binary/org.eclipse.rcp.configuration_root.gtk.hpux.ia64_32_1.0.0.M20121024-1600 .../eclipse/plugins/org.eclipse.core.filesystem.hpux.ia64_32_1.0.100.v20120913-135826.jar .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.v20120913-144808 .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_32_1.0.100.v20120913-144808/launcher.gtk.hpux.ia64_32.properties .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64_32_3.100.1.v4235b.jar .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64_32.source_3.100.1.v4235b.jar 4.3.0 I-build ./eclipse-SDK-I20121024-1200-hpux-gtk-ia64 .../eclipse/p2/org.eclipse.equinox.p2.core/cache/binary/org.eclipse.rcp.configuration_root.gtk.hpux.ia64_1.0.0.I20121024-1200 .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_1.0.100.v20121019-144147 .../eclipse/plugins/org.eclipse.equinox.launcher.gtk.hpux.ia64_1.0.100.v20121019-144147/launcher.gtk.hpux.ia64.properties .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64_3.101.0.v4311.jar .../eclipse/plugins/org.eclipse.swt.gtk.hpux.ia64.source_3.101.0.v4311.jar To cross-reference, and hopefully remember in future, another change was required for Simultaneous Release repository aggregation, since we specify all valid "configurations" available, and I forgot to change ia64_32 to ia64, which I think is now causing bug 393952. |