Community
Participate
Working Groups
In doing some practice CBI builds, I've noticed that in some cases, the CBI build appears to be pulling on "latest" from Orbit build, instead of a "predictable version" we specify, as we used to specify in map files. I've heard you can specify "PDE target-like" files to Tycho (though, not sure how to do it for our Orbit bundles, right off). For example, comparing a recent "practice build" to M5a, the following jump out: org.apache.commons.codec_1.3.0.v201101211617.jar | org.apache.commons.codec_1.4.0.v201209201156.jar org.apache.commons.codec.source_1.3.0.v201101211617.jar | org.apache.commons.codec.source_1.4.0.v201209201156.jar org.apache.felix.gogo.command_0.8.0.v201108120515.jar | org.apache.felix.gogo.command_0.10.0.v201209301215.jar org.apache.felix.gogo.command.source_0.8.0.v201108120515.jar | org.apache.felix.gogo.command.source_0.10.0.v201209301215.jar org.apache.felix.gogo.runtime_0.8.0.v201108120515.jar | org.apache.felix.gogo.runtime_0.10.0.v201209301036.jar org.apache.felix.gogo.runtime.source_0.8.0.v201108120515.jar | org.apache.felix.gogo.runtime.source_0.10.0.v201209301036.jar org.apache.felix.gogo.shell_0.8.0.v201110170705.jar | org.apache.felix.gogo.shell_0.10.0.v201211091412.jar org.apache.felix.gogo.shell.source_0.8.0.v201110170705.jar | org.apache.felix.gogo.shell.source_0.10.0.v201211091412.jar
I'm going to mark this as a blocker, since we need this function in our builds for them to be reproducible (predictable). Otherwise, every time we switch to a new Orbit repo URL, we could get different versions of third party bundles ... of course, we do want the latest qualifier, of a particular major, minor, service version ... but, we don't want to suddenly get a new minor or service version without intending to.
As I understand Tycho, it has this ability ... was just never implemented in "cbi" ... according to http://wiki.eclipse.org/Tycho/Target_Platform#Target_platform_configuration We need a new module of packaging type eclipse-target-definition in eclipse.platform.releng.tychoeclipsebuilder and uses it to "constrain" any of our pre-reqs, especially from Orbit, since its well known its repos can have multiple versions of same bundle. We might want to use for EMF repos or ECF repos? But, for those, we can be relatively sure there is only one version available, as long as we get bug 400522 fixed.
Thanh, Is this something you could work on? Have you experience with it? If the answer is "no" feel free to move back to "in box". Thanks,
(In reply to comment #3) > Thanh, > > Is this something you could work on? Have you experience with it? > > If the answer is "no" feel free to move back to "in box". > > Thanks, I don't have any experience using the Tycho PDE Target Files specifically. However usually when I need to restrict a version of a bundle I use another method which also uses the tycho-platform-configuration plugin. For example I limited the version of org.apache.lucene using this snippet: <build> <plugins> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>target-platform-configuration</artifactId> <configuration> <dependency-resolution> <extraRequirements> <requirement> <type>eclipse-plugin</type> <id>org.apache.lucene</id> <versionRange>[2.9,3.0)</versionRange> </requirement> </extraRequirements> </dependency-resolution> </configuration> </plugin> </plugins> </build> I'm not sure what the difference is between this method and the PDE Target Files method. But I noticed the wiki mentions "Note: Tycho has only limited support for target definition files:" I can try to come up with a patch using my method and we can see if it does what you need. I'll need to modify the poms of all the bundles that pull in those jars though. So I might need some help figuring out which bundles require them. For now I'll try grepping some MANIFEST.MF and feature.xml.
I think the target approach might be best, since a bundle may well specify a "wide range", such as [1.3,2.0) ... by design ... but for our build we want specifically org.apache.commons.codec_1.3.0.v201101211617 The idea is that "6 months after release" if we want we want to reproduce a build with exact same content (except say for one patch) then we want a way to pull exact same pre-reqs. While its true "Tycho has only limited support for target definition files" ... I think it ways it support p2 repositories, not directories, and it is p2 repositories that we want for Orbit, so I think that's ok. I could, likely, produce some initial .target files, but its unclear to me how to "fit those in" to the eclipse parent pom. I just thought maybe you'd seen existing examples, or something.
(In reply to comment #5) > I think the target approach might be best, since a bundle may well specify > a "wide range", such as [1.3,2.0) ... by design ... but for our build we > want specifically > > org.apache.commons.codec_1.3.0.v201101211617 > > The idea is that "6 months after release" if we want we want to reproduce a > build with exact same content (except say for one patch) then we want a way > to pull exact same pre-reqs. While its true "Tycho has only limited support > for target definition files" ... I think it ways it support p2 repositories, > not directories, and it is p2 repositories that we want for Orbit, so I > think that's ok. > > I could, likely, produce some initial .target files, but its unclear to me > how to "fit those in" to the eclipse parent pom. > > I just thought maybe you'd seen existing examples, or something. Ok that makes sense. Unfortunately I have no experience with the target files method. In that case I'll move the assignee back to inbox for now. Let me know if there's anything I can do to help.
Ian, Pascal, and Tom, Long story short, we will get "perfectly reproducible builds" going eventually. But by then, we will have published the bundles in comment 0, in some I-builds, with the "higher" version number than we were using in PDE build. Seems a good solution is for you (or, someone) to open CQs to "re-use from Orbit". Partially because many consumers/distributors prefer we "use the latest version" as it fits in to their distributions better. Plus, my thinking is that once we publish something with "higher" version, even for these transitional builds, it can cause grief if we later want to reverse the monotonicity. I could see that o.a.c.codec was used in p2, but did not see org.apache.felix.command and others, with what I happen to have loaded in workspace. From their name, I'd guess equinox? Let me know if you agree ... or disagree. If the later (e.g. you know there is a technical reason NOT to use the higher versions), we'll make other plans or just be sure to notify everyone well.
(In reply to comment #0) > org.apache.felix.gogo.command_0.8.0.v201108120515.jar | > org.apache.felix.gogo.command_0.10.0.v201209301215.jar > org.apache.felix.gogo.command.source_0.8.0.v201108120515.jar | > org.apache.felix.gogo.command.source_0.10.0.v201209301215.jar > org.apache.felix.gogo.runtime_0.8.0.v201108120515.jar | > org.apache.felix.gogo.runtime_0.10.0.v201209301036.jar > org.apache.felix.gogo.runtime.source_0.8.0.v201108120515.jar | > org.apache.felix.gogo.runtime.source_0.10.0.v201209301036.jar > org.apache.felix.gogo.shell_0.8.0.v201110170705.jar | > org.apache.felix.gogo.shell_0.10.0.v201211091412.jar > org.apache.felix.gogo.shell.source_0.8.0.v201110170705.jar | > org.apache.felix.gogo.shell.source_0.10.0.v201211091412.jar The gogo bundles are used by equinox console. We already have an approved CQ open for the latest 0.10.0 version: https://dev.eclipse.org/ipzilla/show_bug.cgi?id=6834 We are fine with moving up, and in reality should have done this in M5, but it slipped through the cracks.
I'll take care of the CQ for commons.codec
(In reply to comment #5) > I could, likely, produce some initial .target files, but its unclear to me > how to "fit those in" to the eclipse parent pom. > I just thought maybe you'd seen existing examples, or something. example: http://git.eclipse.org/c/tycho/org.eclipse.tycho-demo.git/tree/itp04-rcp (see the <target> configuration in the parent pom and the "helios" target module) a .target will lock down versions to the ones specified (and restrict repo content visible to the build to what is specified in .target) so this is the right approach for reproducible builds. tycho does have support for p2 repos in .target files so you can use the e.g. the PDE target editor to create/edit them (the editor has some quirks and it's slow though). Also, this can provides a better link from headless build to PDE UI build as .target files can be reused in eclipse ("set as target platform" in target editor). Given that the eclipse platform build is monolithic and has quite some p2 <repositories> configured in parent pom.xml, you will need to translate all of these into target <location> elements and corresponding list of (feature) IUs. this will probably be a huge .target which may not be fun to work with/resolve in the IDE but that's just an effect of the monolithic approach I guess.
(In reply to comment #10) > (In reply to comment #5) > > Given that the eclipse platform build is monolithic and has quite some p2 > <repositories> configured in parent pom.xml, you will need to translate all > of these into target <location> elements and corresponding list of (feature) > IUs. Do you mean this literally, that one "has to"? I was hoping to do it only for Orbit. The other repos we point to are fairly specific to "just one version", but, as we know, by design, Orbit is one big heap. Thanks for the pointers,
(In reply to comment #11) > Do you mean this literally, that one "has to"? I was hoping to do it only for > Orbit. The other repos we point to are fairly specific to "just one version", > but, as we know, by design, Orbit is one big heap. Yes, this is possible, too. You can add some p2 repositories to the target platform entirely (by listing them in the POM), and from some p2 repositories only some selected units (through a target file). If you are concerned about build reproducibility, it is crucial that you understand how the target platform is computed in Tycho. (The target platform's primary purpose is to allow reproducible builds.) I think it is all explained in sections 2 and 3 on this page [1], but let me quickly outline how it works again: - First, the different configuration options (repositories in the POM, target files, possibly POM dependencies) are interpreted _independently_, and the resulting units are added to the target platform. So this part of the configuration is additive. - Then, everything from the reactor is added (unconditionally), and everything installed into the local Maven repository (i.e. built with "mvn install" or "mvn deploy") - Then, the target platform filters are applied to the set of units. So the filters configuration is subtractive, and has a higher precedence than the additions. So for the problem at hand, you theoretically have two options: Control which bundles from the Orbit are added (through a target file), or remove all bundles from the Orbit that you don't want (through filters). Since the Orbit usage is based on a positive list of approved bundles, it is clear that the first approach is the right one. Here is how you do this: - Create a target file with the approved Orbit bundles, and configure it in the Platform parent POM. This target file should use includeMode="slicer" so that there is no "slack" in the target file. AFAIK, there are no features in the Orbit, but you can also add bundles in a target file (if you know the bundle IDs and use a text editor). You can see which bundles are added to Tycho's target platform by looking at the Content tab of the target editor. (As long as you stick to the features supported by Tycho, Tycho and the PDE resolve target files in the same way.) - Remove the Orbit p2 repository from the repositories configuration in the Platform parent POM. This is important, because otherwise you'll still have the entire Orbit in the target platform. It is a common misconception that specifying an explicit version in a target file guarantees that only that version is in the target platform. This may not be the case, because a target file does not prevent that you add other versions via the other additive configurations (POM repositories, or other target files). [1] http://wiki.eclipse.org/Tycho/Target_Platform
Coming back to the reproducible build: The build is reproducible if the target platform is reproducible. The easiest way to achieve this is to only reference immutable p2 repositories. If you need to reference a p2 repository which is changed (in permitted ways, i.e. only content may be added), you may only reference these in target files with includeMode="slicer". In all other cases, additions to the p2 repository may also show up in the target platform, and so the build would not be reproducible. p2 repositories which change in "illegal" ways (i.e. content is removed or changed) may not be referenced by a reproducible build.
(In reply to comment #4) > I don't have any experience using the Tycho PDE Target Files specifically. > However usually when I need to restrict a version of a bundle I use another > method which also uses the tycho-platform-configuration plugin. Although this approach works in most of the cases, this method can *not* be recommended if you have to be sure that only one exact version is used. The reason is the following: Everything I've talked about so far is the "target platform", i.e. the set of bundles that the dependencies (from MANIFEST.MF, ...) may be resolved against. If a bundle is not in the target platform (e.g. because you have taken care not to add it, or if it is kicked out by a filter), you can be sure that it will not be used to resolve a declared dependency at build time. The <extraRequirements> configuration however doesn't affect the target platform, but gives extra constraints to the depencency resolver (= a p2 planner). So if you tell the p2 planner to use some version of a bundle through an extraRequirement, and this bundle also satisfies the version range dependencies in the manifest, the dependency resolver will (by the planner's policy to prefer a smaller result) not resolve other versions. If however the extraRequirement version conflicts with the dependencies from the manifest, the dependency resolver be able to still satisfy both requirements and resolve two versions of a (non-singleton) bundle. So, <extraRequirements> should be seen as a hint to the dependency resolver, not a way to restrict the depencency resolution.
Guess I can't call this a blocker. The work around (for now) will be to check from build to build (when we change pre-reqs, such as Orbit URL) to see if any undesired changes occurred. But, in the long term (such as for LTS) we still need reproducible builds, so needs to be done.
Should do for "4.4" first, but where we really need it is in maintenance streams.
Created attachment 237573 [details] patch to "activate" target artifact I went ahead and checked in my current guess as to what "orbit-prereqs" target artifact should be (as a folder in aggregator). It should have to impact until it's built and made part of the model, via the attached patch. Before "activating" it with the patch, waiting to hear if there's some way to resolve bug 422103 (that is, I think activating it will cause bug 422103).
(In reply to David Williams from comment #17) > Created attachment 237573 [details] > patch to "activate" target artifact > > I went ahead and checked in my current guess as to what "orbit-prereqs" > target artifact should be (as a folder in aggregator). > > It should have to impact until it's built and made part of the model, via > the attached patch. > > Before "activating" it with the patch, waiting to hear if there's some way > to resolve bug 422103 (that is, I think activating it will cause bug 422103). Now that bug 422103 has been fixed, I've pushed these "activation" changes into master, and will now do a proper test build (on my local machine). If all goes well, will be "live" in tonight's nightly. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=c3cb852dea7f0433ee647d17e41139aaa0a1346a Note: if we decide to respin yesterday's I-build? This change should be reverted to reduce risk of complications.
I'll comment on the "design" of this target artifact, in case there is discussion/advice. My first thought was to have "separate" ones for Orbit, EMF, and ECF, with the idea that someday, if we ever have a less monolithic build, some of them might "move" lower in the stack ... such as only p2 "pre-reqs" ECF but, I've learned, some of the Orbit ones are very specific to Equinox, so they could move lower too (they are currently "commented" in the orbit-prereqs.target file). But, since PDE only allows you to "pull in" one target, my second thought is to keep them all together for now, so any developer can "get the latest" prereqs in their PDE runtime target easily, and just keep the various uses/sections well commented in target file. Not, using the "target editor" would likely lose the comments ... but ... I think pretty much have to edit the file with text editor, since we (normally) specify specific bundles (not features). If we (I) go this second route, will do some re-naming to remove the word "orbit" and just is "sdk-prereqs".
(In reply to David Williams from comment #18) > (In reply to David Williams from comment #17) > > Created attachment 237573 [details] > > patch to "activate" target artifact > > > > I went ahead and checked in my current guess as to what "orbit-prereqs" > > target artifact should be (as a folder in aggregator). > > > > It should have to impact until it's built and made part of the model, via > > the attached patch. > > > > Before "activating" it with the patch, waiting to hear if there's some way > > to resolve bug 422103 (that is, I think activating it will cause bug 422103). > > Now that bug 422103 has been fixed, I've pushed these "activation" changes > into master, and will now do a proper test build (on my local machine). If > all goes well, will be "live" in tonight's nightly. > > http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/ > commit/?id=c3cb852dea7f0433ee647d17e41139aaa0a1346a > > Note: if we decide to respin yesterday's I-build? This change should be > reverted to reduce risk of complications. Forgot one important change in my "patch": http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=ce474e0d73ccfbd6a962d7edce005d9ee2bd121c
Local build on "true" repo went ok, so will want to start back port soon. And before I do that, want to settle on "name". I have decided to rename module: eclipse.platform.releng.prereqs.sdk and the target file itself: eclipse-sdk-prereqs.target I'll run these changes though my test builds and once ok, will begin to look at back porting to "maintenance branch". I plan to back port this "orbit only" version first ... and once that's done "start again" with adding specific ECF and EMF pre-regs to target file. The reason is to get "orbit versions" backported to 4.2.2+ as soon as possible, and (currently) have no plans to back port ECF and EMF pre-reqs back to that stream, only to master and R4_3_maintentance. This assumes the EMF and ECF repos we use in 4.2.2+ remain non-changing and permanent (which we do expect). The reason for backporting orbit to 4.2.2+ is to update the version of jsch used there, in an easy, reliable way.
rename changes: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=b320e5dac8307edd5086373d9dca12262f2ddefa (apologies, contains a few "no op" touches to unrelated files that just got "caught up" from my test repo). Doing (local) test build now.
This is commit for sdk-prereqs for R4_3_maintenance branch ... just the artifact, activation to come in next. There appears to be about 8 or 10 differences from Luna stream, so I think close enough I'll commit directly to canonical repo, and do (local) test builds to confirm correct. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_3_maintenance&id=5ed861c83630503f60f55174a12a1e4a83298778 This commit is the one that "activates" the target artifact for R4_3_maintenance stream. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_3_maintenance&id=9e0ae0618aaea88d497f3f55df7db6458e228cfb
Those commits broke my p2 build. The point is that I check out only the rt.equinox.p2 and use https://repo.eclipse.org/content/repositories/eclipse-snapshots/org/eclipse/ as a source of eclipse-platform-parent, but starting from today I'm getting Internal error: java.lang.RuntimeException: Could not resolve target platform specification artifact org.eclipse:eclipse-sdk-prereqs:target:4.4.0-SNAPSHOT, because the eclipse-sdk-prereqs module is not available in repo.eclipse.org. Would it be possible to publish it there (to achieve consistency)?
(In reply to David Williams from comment #23) > This is commit for sdk-prereqs for R4_3_maintenance branch ... just the > artifact, activation to come in next. There appears to be about 8 or 10 > differences from Luna stream, so I think close enough I'll commit directly > to canonical repo, and do (local) test builds to confirm correct. > > http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/ > commit/?h=R4_3_maintenance&id=5ed861c83630503f60f55174a12a1e4a83298778 > > This commit is the one that "activates" the target artifact for > R4_3_maintenance stream. > > http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/ > commit/?h=R4_3_maintenance&id=9e0ae0618aaea88d497f3f55df7db6458e228cfb Your definitions all say 4.3.0-SNAPSHOT, but this commit lists <version>4.4.0-SNAPSHOT</version> as the version to get (this is different than kdaniel's problem) PW
(In reply to Paul Webster from comment #25) > (In reply to David Williams from comment #23) > Your definitions all say 4.3.0-SNAPSHOT, but this commit lists > <version>4.4.0-SNAPSHOT</version> as the version to get (this is different > than kdaniel's problem) > Yes, there were two subsequent commits in Kepler stream to fix things, one an error, one a surprise ... and I thought I could get some sleep before completing the bug entry pastes :) http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_3_maintenance&id=abfe80ec0711438660715efff9a908abd9320d62 http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_3_maintenance&id=9ff40d7ef6ffa9243f80939417c3fd029545e7a2 Does anyone ever paste commits as queries ... http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/log/?h=R4_3_maintenance&qt=grep&q=Bug+400518 Seems as a general rule it's very easy to find "corrections" or "all related commits".
(In reply to Krzysztof Daniel from comment #24) > Those commits broke my p2 build. The point is that I check out only the > rt.equinox.p2 and use > https://repo.eclipse.org/content/repositories/eclipse-snapshots/org/eclipse/ > as a source of eclipse-platform-parent, but starting from today I'm getting > Internal error: java.lang.RuntimeException: Could not resolve target > platform specification artifact > org.eclipse:eclipse-sdk-prereqs:target:4.4.0-SNAPSHOT, > because the eclipse-sdk-prereqs module is not available in repo.eclipse.org. > Would it be possible to publish it there (to achieve consistency)? I don't mind someone doing this ... presumably at the same time the parent pom is published ... https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/deploy-eclipse-platform-parent-pom-4.4/ Which I think is just once a day. So will ask ... are you sure this wasn't a "point in time" problem? There's so much that "automatically happens" I'm surprised this would not be one of them? Perhaps because its the only artifact directly required by parent pom? In either case, since I know nothing about this, I will ask Thanh for help. Thanh, HELP! :) Is it possible to add these "target artifacts" to the eclipse-parent deploy jobs? (4.4 and 4.3 are ready now ... 4.2 should be coming a bit later today).
(In reply to David Williams from comment #27) > (In reply to Krzysztof Daniel from comment #24) > Which I think is just once a day. So will ask ... are you sure this wasn't a > "point in time" problem? There's so much that "automatically happens" I'm > surprised this would not be one of them? Perhaps because its the only > artifact directly required by parent pom? > Given similar "build break" msg on platform-ui-dev list, I suspect this is an issue because the "CI jobs" in question really only build one thing, using the build-individual-bundles profile? So guess they need "everything else" to already exist, somewhere?
Example for a Gerrit build which is broken with his error message: https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/
(In reply to David Williams from comment #28) > > Given similar "build break" msg on platform-ui-dev list, I suspect this is > an issue because the "CI jobs" in question really only build one thing, > using the build-individual-bundles profile? So guess they need "everything > else" to already exist, somewhere? Yes, the Gerrit jobs build only one repo with build-individual-bundles, so they need any maven artifacts that aren't in that repo to be on repo.eclipse.org. It used to only be the eclipse-project-parent, but now it's the eclipse-sdk-prereqs as well. PW
(In reply to Lars Vogel from comment #29) > Example for a Gerrit build which is broken with his error message: > https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/ Thanks Krzysztof for publishing work around, at least for "local builds"? http://dev.eclipse.org/mhonarc/lists/cbi-dev/msg01113.html I'll quote it here in case it helps others (but mostly for my own notes, so I won't forget :) Still assuming (hoping) Thanh can work into the parent publishing job so it gets into repo.eclipse.org? = = = = = git clone git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git cd eclipse.platform.releng.aggregator/eclipse.platform.releng.prereqs.sdk/ mvn clean install = = = = = Thanks all for the help and education.
I've made this commit to pull in exact versions of Orbit bundles for R4_2_maintenance builds: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_2_maintenance&id=f0ab62a1c2bc27e55b7b45658cd7f6dfbb389414 This one should duplicate what's in "released version" of 4.2.2 ... once we confirm that works, we will modify to get most recent version of jcraft's jsch.
(In reply to David Williams from comment #31) > I'll quote it here in case it helps others (but mostly for my own notes, so > I won't forget :) Still assuming (hoping) Thanh can work into the parent > publishing job so it gets into repo.eclipse.org? I've moved our parent deploy jobs to our HIPP instance, https://hudson.eclipse.org/platform/ and disabled the ones on the main hudson instance. I'll try and build/deploy the sdk module there as well. https://hudson.eclipse.org/platform/job/deploy-eclipse-sdk-target-pom-4.4/ So far, not so good :-) PW
(In reply to Paul Webster from comment #33) > I'll try and build/deploy the sdk module there as well. > https://hudson.eclipse.org/platform/job/deploy-eclipse-sdk-target-pom-4.4/ OK, this module is not deployed in repo.eclipse.org. Lars, did that help your jobs? PW
> OK, this module is not deployed in repo.eclipse.org. Lars, did that help > your jobs? > > PW I assume you mean s/not/now? I test this not (now). ;-)
> I test this now Works fine for me: cd .m2/repository rm -rf * cd ~/home/git/eclipse.platform.ui/bundles/org.eclipse.ui.images.renderer mvn clean install -> Work fine If I manually trigger https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit it fails but with a different error message Couldn't find remote ref $GERRIT_REFSPEC. I'm not sure if Gerrit build jobs can be manually retriggered so I push a change to test the build.
You can re-trigger a specific Gerrit job but not the general one. PW
(In reply to Paul Webster from comment #37) > You can re-trigger a specific Gerrit job but not the general one. Gerrit builds are also working again. Thanks! Example https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/294/console
(In reply to David Williams from comment #32) > I've made this commit to pull in exact versions of Orbit bundles for > R4_2_maintenance builds: > > http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/ > commit/?h=R4_2_maintenance&id=f0ab62a1c2bc27e55b7b45658cd7f6dfbb389414 > > This one should duplicate what's in "released version" of 4.2.2 ... once we > confirm that works, we will modify to get most recent version of jcraft's > jsch. I've reverted this commit ... it had a couple of errors, and a dozen omissions, so will commit corrected version soon.
(In reply to David Williams from comment #39) > (In reply to David Williams from comment #32) > I've reverted this commit ... it had a couple of errors, and a dozen > omissions, so will commit corrected version soon. Well, not as soon as I thought ... I've encountered some issue that prevents progress, which I'll track in bug 422302.
(In reply to David Williams from comment #40) > (In reply to David Williams from comment #39) > > (In reply to David Williams from comment #32) > > > I've reverted this commit ... it had a couple of errors, and a dozen > > omissions, so will commit corrected version soon. > > Well, not as soon as I thought ... I've encountered some issue that prevents > progress, which I'll track in bug 422302. With this commit, I can get everything to resolve: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_2_maintenance&id=a70f767b51f15d9c5396391562af970d88846755 but depends on "fix" in bug 422302 (which is in equinox). Plus, in my "local test builds" SWT had a lot of compile errors, so not sure my build environment is exactly right. I think the submodules are not necessarily up to date ... if I get aggregator, from R4_2_maintenance branch, submodule init, and submodule update, I get the following "errors": $ git submodule update fatal: reference is not a tree: ec0e77282e430b1f559858901b1ede64f352f1e3 fatal: reference is not a tree: 9c081d0b66d04026d98dee8be2375a4b8ac3fdd5 fatal: reference is not a tree: 7e4eb0d63722fa49ecb444ff43eb870ede2b7561 fatal: reference is not a tree: 2430e2404e2d54588ce3c0e2d227da4ae6aac17a fatal: reference is not a tree: c3eb2803e06f9abc9f85c8c469f3646dabefc58e fatal: reference is not a tree: a0a37a8e7269bf63e1566ce447fb04b9fef9bddf fatal: reference is not a tree: 78a180fa30c360fd614a9ad63c4eccf70846f3c4 fatal: reference is not a tree: 5986b87392244b57388c536e64185d6465aa26de fatal: reference is not a tree: 5d18777374ebfc44dff2bbb7bb412056cc3543ce fatal: reference is not a tree: c13988d21bcef73cec6604a9d21d7c08ffd855df fatal: reference is not a tree: 008232a3a6b909d6172e052b252fa7f7f94c248b Unable to checkout 'ec0e77282e430b1f559858901b1ede64f352f1e3' in submodule path 'eclipse.jdt' Unable to checkout '9c081d0b66d04026d98dee8be2375a4b8ac3fdd5' in submodule path 'eclipse.jdt.debug' Unable to checkout '7e4eb0d63722fa49ecb444ff43eb870ede2b7561' in submodule path 'eclipse.jdt.ui' Unable to checkout '2430e2404e2d54588ce3c0e2d227da4ae6aac17a' in submodule path 'eclipse.pde' Unable to checkout 'c3eb2803e06f9abc9f85c8c469f3646dabefc58e' in submodule path 'eclipse.pde.build' Unable to checkout 'a0a37a8e7269bf63e1566ce447fb04b9fef9bddf' in submodule path 'eclipse.platform.debug' Unable to checkout '78a180fa30c360fd614a9ad63c4eccf70846f3c4' in submodule path 'eclipse.platform.resources' Unable to checkout '5986b87392244b57388c536e64185d6465aa26de' in submodule path 'eclipse.platform.swt' Unable to checkout '5d18777374ebfc44dff2bbb7bb412056cc3543ce' in submodule path 'eclipse.platform.swt.binaries' Unable to checkout 'c13988d21bcef73cec6604a9d21d7c08ffd855df' in submodule path 'eclipse.platform.team' Unable to checkout '008232a3a6b909d6172e052b252fa7f7f94c248b' in submodule path 'eclipse.platform.text' To get around that, I just "checked out" the branch of those submodules that streams/repositories.txt says they should have.
Created attachment 237641 [details] patch to add .source bundles for lucene 2.9.1 level I can see (from LTS job) it is complaining about not finding "source" bundles for lucene at 2.9.1 level. Not sure why it needs it, when my local job did not, so might indicated something (else) is not quite right. (As far as I can tell, we shipped lucene 3.5 level in 4.2.2 and while I included "code" at 2.9.1 level in my local builds (and first commit), I was assuming this was some "technicality" to satisfy some constraint that ultimately was not used/delivered). So, I've attached this patch to add the ".source" bundles it says it needs, it simply adds 3 lines to the .target file: 12 <unit id="org.apache.lucene" version="2.9.1.v201101211721"/> 13 <unit id="org.apache.lucene.analysis" version="2.9.1.v201101211721"/> 14 <unit id="org.apache.lucene.core" version="2.9.1.v201101211721"/> 15 +<unit id="org.apache.lucene.source" version="2.9.1.v201101211721"/> 16 +<unit id="org.apache.lucene.analysis.source" version="2.9.1.v201101211721"/> 17 +<unit id="org.apache.lucene.core.source" version="2.9.1.v201101211721"/> and I'll hold off committing until I hear or learn more. I have reconstructed my local environment, set right memory limits, etc., and swt does seem to be building now :) .... but will wait for a local test job to complete before coming to any conclusions.
(In reply to David Williams from comment #42) > Created attachment 237641 [details] > patch to add .source bundles for lucene 2.9.1 level > > and I'll hold off committing until I hear or learn more. I have > reconstructed my local environment, set right memory limits, etc., and swt > does seem to be building now :) .... but will wait for a local test job to > complete before coming to any conclusions. My re-vamped local build did finally "complete" and had the same error about needed source bundles for 2.9.2 level for the help feature ... so I'm relatively sure something is off :) But to keep things going, I committed that addition: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_2_maintenance&id=ef70c4537091fb267572d930b5f918c6d91ca75b And will try again.
(In reply to David Williams from comment #43) > (In reply to David Williams from comment #42) > > Created attachment 237641 [details] > > patch to add .source bundles for lucene 2.9.1 level > > > And will try again. My local build did complete normally. I did quickly check the "products" built, and see that lucene 2.9.1 was included in SDK, which led me to re-check and see that 2.9.1 WAS in Juno SR2 ... so, not what led me to believe 3.5 was required. Perhaps that can "come out" of current .target file. I'll do a few more experiments, but then plan to commit the changes needed for bug 418837 -- the whole reason I've not been sleeping lately :) And if that works we can fine tune the .target file from there if necessary.
I think we are set. I did a local build, and compared all non "org.eclipse" jars to the final 4.2.2 repo (on eclipse.org) and my local test build: $ diff eclipse422Plusrepo-sorted.txt eclipse422repo-sorted.txt 5,6c5,6 < com.jcraft.jsch_0.1.50.v201310081430.jar < com.jcraft.jsch.source_0.1.50.v201310081430.jar --- > com.jcraft.jsch_0.1.46.v201205102330.jar > com.jcraft.jsch.source_0.1.46.v201205102330.jar 48d47 < org.aspectj.weaver_1.6.7.20091231194938.jar 54c53 < org.junit4_4.8.1.v20120523-1205.jar --- > org.junit4_4.8.1.v20120523-1257.jar The com.jcraft.jsch differences are what were trying to achieve, in this 422 branch. The other differences are anomalies. In production builds, we remove org.aspectj.weaver from the repo, and the "org.junit4" bundle actually comes from eclipse Platform itself ... back in those old days it was a "transition bundle" ... not sure why it's qualifier is slight different ... but ... 53 minutes doesn't seem like it'd mean anything (actually many of the "org.eclipse" bundles are "slightly off" in qualifiers ... perhaps some subtle change in "server time" or the way qualifier is computed? But I think this concludes "the Orbit work". I'm going to close this as fixed, and open a clone to do similar targets for other non-platform repositories. They are not as urgent since the URLs we use for them point to one specific repository, that should always have only one version of a bundle in it. (But, I think still, a "best practice, as a way of guarding against others unexpectedly changing things, plus, easier for developers to load changed pre-reqs into their PDE target to see if any unexpected effects.
