Community
Participate
Working Groups
We need this for eclipse sdk too, but I put equinox in title, since I am not a committer there, so need to address that, as well as making both "systematic" if not completely automatic, eventually. Fortunately, for equinox, to "promote a build" is a very simple thing. Only one drop directory. only one stream and no index page to update and no update site to udpate. So, is basically "one step" using rsync (by an equinox committer, with a shell script). Eclipse SDK on the other hand is more complicated, involving 4 or 5 steps, since there are two streams, two drop directories, several "index pages" to update, and updating/correcting softw Plus, equinox does not have its own software site repository ... it uses Eclipse's repository (since needs to be there anyway ... and, for historical reasons its always been that way), so the Eclipse SDK has that extra step. So, the current plan (after a brief discussion with John A. today) is ... for both equinox and eclipse ... the builder will produce a "promote script" which any proper committer with a shell script could execute to promote what was just built if/when deemed appropriate. John will, for now, promote equinox drops. While I will still promote the eclipse SDK portions, so no committership changes needed, no new shell accounts, etc.). With time, and confidence, a committer could have a cron job which just peeked, say every 5 minutes at a known location and if there was such a "promote script" available there, then execute it ... but, that'll be phase two. This allows an "incremental" approach to making it automatic. Plus, the cron job could be turned off, at times, such as when nearing a release, and the build on the build machine given more scrutiny before actually moving it to downloads. [I only mention it, in case anyone sees any problems, security or otherwise, with this scheme.] (And, honestly, the equinox case is so simple, it likely doesn't really require a "promote script" ... but ... again, this allows an incremental approach). For the recored, all projects have this problem; no "build id" can copy things to "downloads" since it is desired to have restricted, traceable accounting of who does things there on that "public" site, and all projects have come up with 3 or 4 different patterns to "get around it" so it can be mostly/nearly automatic, some with varying degrees of security/safety. Again, for the record, when Kim was doing the builds on IBM hardware, she was a committer on both eclipse and equinox (releng site) and by its very nature required her own committer id to "push" things to downloads (not to mention some even more complicated "reverse mirroring" stuff, which we are now happy to get rid of). So, long wordy explanation of what will end up being very simple :) Just wanted a place to recored it all.
John, You can take a look in /shared/eclipse/equinox/queue to see the "promote script" produced by the 4.2 nightly build for, N20120418-2000. (I wouldn't run it blindly, but from a quick glance, it looked right to me.) Note, I have put in logic to _not_ create equinox promote scripts for 3.x builds, since as mentioned elsewhere, equinox, in 3.x builds should be identical to 4.x builds, so we will promote only the 4.x versions, to the equinox location. Also, I'm assuming for now there is no need for a separate "dev list notification" that a new equinox build is available and has been promoted to "downloads" ... I don't remember those happening before ... but, we could add that eventually if/when you think its important. (BTW, I'm not real happy with the location/name of the "promotion queue and script name" so I (we?) might think of something better in the next week or so, but after that point would want to leave it pretty constant). Last note, I think we eventually do not want to "maintain times" (-t) on the rsync command, but left it there for right now ... the Eclipse mirroring system can sometimes be "fooled" if you put something there that "looks like" its been there for several hours, it will report some mirrors have the content, when in fact they do not).
(In reply to comment #1) > > /shared/eclipse/equinox/queue > > to see the "promote script" produced by the 4.2 nightly build for, > N20120418-2000. it looks ok to me, but I don't have permission to run it as johna. I see it is in a group called "users" but johna does not belong to that group. I've never heard of that group so not sure if I should be. I did "chmod 755" on the script as e4Build and then I was able to execute it.
ok, thanks. 'users' is the group that all "buildids" belong to. The default group should have been eclipse.platform.releng well, I was hoping. I think it might not have been because I let e4Build create the directory. I think I'll try creating it first, as 'david_williams' then hopefully, group access will be correct.
John, I've settled on location and names of promotion scripts: For equinox, /shared/eclipse/equinox/promotion/queue new name form: promote-4.2.0-N-N20120421-2021.sh compared to old (I put "time created" as comment in the script, instead of name): promote-4.2.0-N-N20120419-2000-201204192117.sh promote-4.2.0-N-N20120420-2153-201204202301.sh the "key" to matching the names is "promote*.sh" For example, you'll notices some files named something like TEST-promote-4.2.0-N-N20120421-0324.sh Those are just from test builds and can just be deleted. (eventually, I can stop writing them .... just wanted to for now, to test that I was writing them correctly). For the SDK, I'm going to use /shared/eclipse/sdk/promotion/queue The reason why I came up with this is so I can have the cronjob itself (and related support files and logs relatively isolated in /shared/eclipse/sdk/promotion My little "cron job" doesn't quite work like I want it to yet, I'll resolve this bug once I'm happy with it.
Ok, I'm as happy as I'm going to get. The hard part was getting it to log errors (and send cron job failure notices) when something went wrong ... but not to send one every time, if all goes ok) John, I made this example for you, take it or leave it (or modify it) as you like. (Its like what I use, except the the exact location where it looks for promotion files). wget --no-verbose -O equinoxPromotionCronJob.sh http://git.eclipse.org/c/platform/eclipse.platform.releng.eclipsebuilder.git/plain/scripts/promotionRelated/equinoxPromotionCronJob.sh?h=master
I'm going to reopen this until I actually have the cron job running (if I don't have an open bug there's a danger I'll forget).
I tried running the script manually but nothing was output.. I'll try again tomorrow when there is a new build.
(In reply to comment #7) > I tried running the script manually but nothing was output.. I'll try again > tomorrow when there is a new build. Yes, I probably deleted some of those nightly builds. Guess I/we could put a check in the script to give an error message if doesn't exist ... but, normally, when ran from a crontab, would be rare it'd be removed before being promoted (but, then, all the more reason to have an error message!)
I have installed the crontab and it seems to be happily promoting away. Thanks David!
