Community
Participate
Working Groups
This bug is to document specifics work required to transition to the builder initially described in bug 270058 and further documented in http://wiki.eclipse.org/Buckminster_Galileo_Builder The old builder will be left in place, at least until we fully transition to new one. The primary "working area" for these new builds will be in /shared/galileo/buckyBuild I've created a new project in same location as o.e.galileo.build /cvsroot/callisto named org.eclipse.galileo.tools that contains some initial bash scripts to run the build on build.eclipse.org. One of the first items needed it to run those scripts from the Hudson builder, so that they can be easy to start, etc. From what I understand, this would be best done by "translating" the shell scripts to ant tasks. A second task is to create/change the "build results page" to fit with the new build and locations. Note, for now the o.e.galileo.build project will be left "as is", but eventually a few of the files in there will eventually no longer be needed (such as the "generate.xml" file, the map files, etc.).
Just so you can see progress ... https://build.eclipse.org/hudson/view/BuckyBuild/ I've added callisto-dev, Gabe, and Thomas to have the ability to configure/run/delete jobs, etc. General public should only have "read" access. The index.php page (even though wrong in many ways right now :) can be seen at http://build.eclipse.org/galileo/buckyBuild/buildresults/ The categories and features can be seen via update manager by using http://build.eclipse.org/galileo/buckyBuild/buildresults/mirror [but, remember, can not actually use that for updates, since mirrors for that site are spec'd for that site]. I figure next I'll work on a "promoteToStaging" job, then maybe on that index page. Oh, and need to check fail-mail and make sure it is pointing to right log locations, etc. Thomas, once your next version of Buckminster-Builder is ready, you should just be able to push the "update builder" button and have in "installed" in the version of the platform we are using (still using the I20090407-1430 version right now). To update the platform is still a shell script command ... and figure I'd leave it that way, since rarely needs to be done, and since you have to provide it input anyway (give it the URL you want installed). Off hand, I'd say we can turn on the "production" flag on Friday? Maybe experiment with triggering builds if CVS changes?
(In reply to comment #1) > Thomas, once your next version of Buckminster-Builder is ready, you should just > be able to push the "update builder" button and have in "installed" in the > version of the platform we are using (still using the I20090407-1430 version > right now). > I tried this, but no go. It complains that the top feature is in conflict with itself, i.e. the director doesn't seem to recognize that it's an update. I even tried to add an -uninstallIU first. When I do that, it claims that it uninstalls the *new* feature. I.e. the one that just failed to install on my previous attempts. The install that follows still complains about the conflict. See log: https://build.eclipse.org/hudson/view/BuckyBuild/job/updateBuckyBuilder/6/console Not sure how to move forward with this. I would recommend that we start fresh with a new Eclipse installation.
I moved prereqs/eclipse to prereqs/eclipse.old and installed a new one from /eclipse/downloads/drops/I20090421-0930/eclipse-SDK-I20090421-0930-linux-gtk-ppc.tar.gz. After that, the update succeeded.
The current build doesn't seem to honor the exit code from the application. I think anything but zero should cause the build to fail.
(In reply to comment #4) > The current build doesn't seem to honor the exit code from the application. I > think anything but zero should cause the build to fail. > Agreed. I committed fix to CVS ... but appeared you were editing the build.xml file on build.eclipse.org ... so I won't update until hear from you. FYI, forgot to mention, to refresh the org.eclipse.galileo.tools directory with latest in CVS, you can execute getRelengTools.sh from that same directory, org.eclipse.galileo.tools. As with platform, not sure we need to automate that, but if you think we do we can.
(In reply to comment #5) > (In reply to comment #4) > FYI, forgot to mention, to refresh the org.eclipse.galileo.tools directory with > latest in CVS, you can execute getRelengTools.sh from that same directory, > org.eclipse.galileo.tools. As with platform, not sure we need to automate that, > but if you think we do we can. > I changed my mind about the "bootstrap" job, after remembering there seems to be good job-specific security controls ... so you can bootstrap the HEAD version of these control files from the Hudson build page ... well, me and Thomas can. If others need it, we'll change security settings then.
I've turned on "auto detect" in the Hudson BuckyBuilder. Changes in HEAD of org.eclipse.galileo.build will be detected every 15 minutes, at 0, 15, 30, 45 minutes past the hour. If a build fails, it will be retried at 10 minute intervals, up to 5 times. Remember, some builds will fail because a change is needed on the projects update site .. and will not necessarily result in another change in the o.e.g.build project. We can tweak these intervals once we get more experienced with daily workflow.
Another task which I've just realized is very important, is to update the wiki documents to point to the correct places for what projects need to do to contribute to a build. The actual contribution part is exactly the same, but the rest of the document at http://wiki.eclipse.org/Galileo/Build should be marked as "deprecated". And some 'cross links' provided. I'll work on that this afternoon. Once up to date, I would like to turn on -production flag so mail is sent out on failures, and I'll announce "open for business" on cross-project ... unless anyone thinks otherwise?
(In reply to comment #8) > Once up to date, I would like to turn on -production flag so mail is sent out > on failures, and I'll announce "open for business" on cross-project ... unless > anyone thinks otherwise? > I'm just uploaded a new version with some major improvements. I think it would be worth while to run this version an iteration or two, just to verify that everything is OK. The news are: 1. If trusted contributions are used, metadata is always copied to get all categories right. Only the artifact repository is a mirror (in a folder now renamed to aggregate). 2. A new option -packedStrategy <strategy> where strategy is one of copy, unpack, unpackAsSibling, verify, skip (documentation on wiki forthcoming). 3. Bug 273530 probably fixed.
(In reply to comment #9) > (In reply to comment #8) > I'm just uploaded a new version with some major improvements. I think it would > be worth while to run this version an iteration or two, just to verify that > everything is OK. > That sounds good. And then I'll update the "promoteToStaging" script (to use 'aggregate' instead of 'mirror', and then should probably wait a while to let that propagate some before pointing people to it. Plus, I've heard EclipseLink project is about ready to contribute, so maybe wait to see if that happens this afternoon ... so maybe will wait till Monday morning for broad announcement, so it doesn't "get lost" over the weekend.
Documentation updated, Hudson updated, and a new build is in progress. This build runs with -packedStrategy verify which means that the packed artifacts will be copied verbatim and then unpacked. The result of the unpack is however discarded so the resulting site should be much smaller then before. I took the liberty of adding some missing repo attributes to features in the ep.build and equinox.build. The builder will complain about that too now. B.t.w. is there any particular reason we don't have linux.gtk.x86_64 as one of the configurations?
Thomas, I think something is amiss ... if I point an Eclipse Update to the build machine site, at http://build.eclipse.org/galileo/buckyBuild/buildresults/ I get a "no repository found" message. Should that "content.jar" be named "compositeContent.jar"? (In reply to comment #11) > Documentation updated, Hudson updated, and a new build is in progress. This > build runs with -packedStrategy verify which means that the packed artifacts > will be copied verbatim and then unpacked. The result of the unpack is however > discarded so the resulting site should be much smaller then before. > Cool ... that coves all the bases. :) > B.t.w. is there any particular reason we don't have linux.gtk.x86_64 as one of > the configurations? > No. I suspect got left out accidentally in some editing (by me). The intent was to include all that are required by the EPP project. But, I think with your builder ... we are just verifying ... so should we do them "all"? What happens if a project does not support all configurations? (that is, they have platform specific code, but not for all platforms.
(In reply to comment #12) > Thomas, I think something is amiss ... if I point an Eclipse Update to the > build machine site, at > http://build.eclipse.org/galileo/buckyBuild/buildresults/ > I get a "no repository found" message. > Should that "content.jar" be named "compositeContent.jar"? > No, but you should use: http://build.eclipse.org/galileo/buckyBuild/buildresults/final/
Looks good in my update manager, aside from the fact that all platform features are missing. I know why and I'm on it...
(In reply to comment #13) > (In reply to comment #12) > > Thomas, I think something is amiss ... if I point an Eclipse Update to the > > build machine site, at > > http://build.eclipse.org/galileo/buckyBuild/buildresults/ > > I get a "no repository found" message. > > Should that "content.jar" be named "compositeContent.jar"? > > > No, but you should use: > > http://build.eclipse.org/galileo/buckyBuild/buildresults/final/ > yep, realized that shortly after ... too many versions, too many bookmarks :)
(In reply to comment #14) > Looks good in my update manager, aside from the fact that all platform features > are missing. I know why and I'm on it... > Not sure if related, or even a problem, but a change. I could see "eclipselink" in one of the categories, (EclipseRT). It is listed in that category, but is not actually listed as a "contribution" yet. (Because, I think, there's a but in their update site, so would fail). That used to work by simply not trying to mirror the ommitted contribution, and also not listing in the categories. It is ok with me if it does, I guess, but it was a handy way to omit something temporarily. I'm mostly documenting this so we will all know how it's supposed to work. Or, would you consider current behavior a bug?
(In reply to comment #16) > That used to work by simply not trying to mirror the ommitted contribution, and > also not listing in the categories. > and it still does. The webtools contribution however, has this: <features id="org.eclipse.jpt.eclipselink.feature" version="2.2.0.v200902060000-438Z9oB55V6C7K5556" repo="//@repositories.0"> <category href="galileo.build#//@categories.9"/> </features> Omitting a contribution will make it vanish completely from the model. It will not participate at all (well, with the exception of capabilities since that's build completely aside from everything else).
I uploaded and installed a new version that takes care of the problem with lacking features from the trusted contributions. The result looks OK now and right now I don't have any more issues to deal with. So I'm all for flipping the -production switch now.
I'm going to mark this as complete. If there's other or new issues, please open specific bugs. For what it's worth, I did try to run the old Galileo Builder and it seemed to mostly still work, until it got to the capabilities part of the build (I think). See http://build.eclipse.org/galileo/build/galileo-I20090426-2023.log.txt I've opened bug 273921 to remind me to remove the "galileo build" jobs from Hudson in a week or two. At that time, I suggest we also version the o.e.g.build project with a distinctive tag and then "clean it up" to remove the builder-specific parts (e.g. build.xml, and generate.xml).
