Community
Participate
Working Groups
Currently in the build, each repository will be automatically tagged if it had changes since the last build. The tag format is vYYYYMMDD-HHMM in UTC time. However this means each repository has a different tag. We should consider also adding a tag matching the build id to each repository, regardless of whether it changed, something like the IYYYYMMDD-HHMM tag that goes onto the map files. This would enable us to do things like: - Use a script to tag releases, but applying the release tag to the same commit as the I-build tag. - Checkout by tag across multiple repositories. For example this would enable loading the exact contents of a build into your workspace at once (across multiple repos). Maybe could use a hierarchical tag name here to avoid confusion... "build/IYYYYMMDD-HHMM".
Paul, any thoughts? We could write tools to achieve the same effect but having a common build tag on all repositories might make life easier for people. Dani suggested this on a call this morning and it sounds reasonable.
We could tag the repos after they are all fetched for the build. It would be interesting to see how long this takes. The operation that was implemented this week to fetch the deltas for the repos and update the map files takes around 10 minutes.
Creating and pushing tags is not an expensive operation. It is only applied to a single commit so the repository size has no impact. Paul also mentioned he might be able to simplify the tagging scripts. That would probably make it faster.
We could do it after the "pull $1 $2" step. Maybe a "tagAndPush $1 $buildType$timestamp" and a simple tag() function, so we tag the build input that we're working with. PW
I've pushed updated scripts to a topic branch based on some discussions Kim and I had yesterday. http://git.eclipse.org/c/e4/org.eclipse.e4.releng.git/tree/org.eclipse.e4.builder/scripts/git-release.sh?h=pwebster/autoTagging http://git.eclipse.org/c/e4/org.eclipse.e4.releng.git/tree/org.eclipse.e4.builder/scripts/git-submission.sh?h=pwebster/autoTagging http://git.eclipse.org/c/e4/org.eclipse.e4.releng.git/tree/org.eclipse.e4.builder/scripts/git-map.sh?h=pwebster/autoTagging git-release.sh pulls the releng project on the correct branch, and then pulls in all of the repos from {relengProject}/tagging/repositories.txt Then it does a build submission report (currently for all the repos) from $oldBuildTag (which Kim's build saves, and I'm updating mine to as well) to the branch specified in repositories.txt. Then it does a git-map.sh passing in the current $buildTag so that all of the repos can be tagged. git-map.sh was updated to not includes a build submission report generation step. It generates all commands into run.txt, so it no longer applies some tags as part of its running step. It tags each repo with the build tag, and generates the push commands at the correct time. PW
I've merged my changes into org.eclipse.e4.releng.git master. PW
re: team.map. I'm not sure why it would wipe the map, but If we can't find a tag, as in: plugin@org.eclipse.jsch.ui=GIT,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.team.git,path=bundles/org.eclipse.jsch.ui then CURRENT_TAG will be the content of the entire line I'm not sure what the rest of git-map.sh#update_map() will do in that scenario, but I guess git:// could replace the contents of the line with empty :-) PW
This is not yet live, right? Also, the tag should correspond to the build name and not be some random time stamp, e.g. the tag created/used by the builder for I20111205-1518 should be 'v20111205-1518'.
No, I didn't implement this yet. I believe Paul did in the 4.2 stream.
(In reply to comment #9) > No, I didn't implement this yet. I believe Paul did in the 4.2 stream. Right, in 4.2 and in e4 0.12 I've modified the scripts to tag the build input with the build timestamp (you should be able to see it in eclipse.platform.ui in master) in addition to the standard auto-tagging of the qualifiers. PW
The current tagging script in the master branch is quite confusing. E.g. for I20120110-0800, it added 3 different tags to the eclipse.jdt.ui repository, because the last commits didn't touch all projects in the repo. That's overkill. Furthermore, the tag name still uses UTC time. The build process should only tag a repository once, with a tag that corresponds to the build ID. In this case, it should have been v20120110-0800. This tag should be used for all changed projects.
(In reply to comment #11) > The current tagging script in the master branch is quite confusing. > > E.g. for I20120110-0800, it added 3 different tags to the eclipse.jdt.ui > repository, because the last commits didn't touch all projects in the repo. > That's overkill. Furthermore, the tag name still uses UTC time. Oh, that is working as we designed it. The UTC timestamp of the last commit to touch a bundle if it has changed, and for now recorded in the map file. Not only is the reproducible from any git repo, it would also be predictable for other potential build systems (the LongTermSupport prototype that is maven based, for example). It's the commits that touch a project that can tell you if it has changed or not, and that's 100% unrelated to when a build had run (at some random time). > The build process should only tag a repository once, with a tag that > corresponds to the build ID. In this case, it should have been v20120110-0800. > This tag should be used for all changed projects. The idea behind this was to be able to look at the last commit to change a project, and recreate its qualifier reliably. The plan is to also tag the build input (the final commit) with I20120110-0800 for example, so you can match build inputs to builds. AFAICT it would be feasible to just tag once and update the map files by modifying git-map.sh/update_map. I'd recommend against it though. PW
(In reply to comment #12) This is a nice technical explanation but the current approach is not user friendly. We never use UTC when talking about a build or a build ID. So, if we keep the current approach then we must at least change the tag format, so that it obviously differs from the known tags/build IDs. When we discussed and decided for the auto-tagging we also said that we want the same tag (that maps 1:1 to the build name/version) on all repositories (even if there was no change - just don't change the map file) so that it is very easy to reconstruct the source of a build across several repositories inside the workspace. Either we achieve this by changing the auto-tagging script to use that common tag (and put it on all repos) or we keep the current approach with clarified tag format (see first section) and add another step that adds such a tag.
(In reply to comment #13) > > This is a nice technical explanation but the current approach is not user > friendly. We never use UTC when talking about a build or a build ID. So, if we > keep the current approach then we must at least change the tag format, so that > it obviously differs from the known tags/build IDs. I think we just haven't migrated that change into the eclipse builder auto-tagging script. Have a look at master in eclipse.platform.ui or eclipse.platform.runtime (that is currently using the 4.2 short build auto-tagging script). What I expect: The build input (origin/integration) is tagged with the build ID, I20120110-2200 or M20111215-0900. The bundles are tagged in such a way that their qualifier is v<UTC timestamp>, derived from the last change to touch that bundle (so the same input will give the same result). So if we tag the build input with the standard build ID, that is what you expect? PW
> What I expect: The build input (origin/integration) is tagged with the build > ID, I20120110-2200 or M20111215-0900. The bundles are tagged in such a way > that their qualifier is v<UTC timestamp>, derived from the last change to touch > that bundle (so the same input will give the same result). > > So if we tag the build input with the standard build ID, that is what you > expect? Yes, if we add the I*/M* tag to each build then I'm happy.
(In reply to comment #15) > > So if we tag the build input with the standard build ID, that is what you > > expect? > > Yes, if we add the I*/M* tag to each build then I'm happy. Kim, see http://git.eclipse.org/c/e4/org.eclipse.e4.releng.git/tree/org.eclipse.e4.builder/scripts/git-map.sh#n113 where the script tags each build input repo with the IbuildTag PW
This has been the case for quite some years now.
