Bugzilla – Full Text Bug Listing
| Summary: | [git] Converted repos and maintenance branches not quite right | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Paul Webster <pwebster> | ||||||
| Component: | Releng | Assignee: | Paul Webster <pwebster> | ||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||
| Severity: | normal | ||||||||
| Priority: | P3 | CC: | aniefer, bokowski, cgold, daniel_megert, david_williams, dj.houghton, gheorghe, jamesblackburn+eclipse, john.arthorne, kim.moir, markus.kell.r, Silenio_Quarti, steffen.pingel | ||||||
| Version: | 3.7 | ||||||||
| Target Milestone: | --- | ||||||||
| Hardware: | PC | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Bug Depends on: | |||||||||
| Bug Blocks: | 345479 | ||||||||
| Attachments: |
|
||||||||
|
Description
Paul Webster
Are we doing the conversion off of a copy of our CVS repository? If yes, one experiment that we could do is branching the remaining projects off for 3_6_maintenance as well, and tagging projects with R3_6 that weren't tagged (all of this on the CVS side, to the copied repository), and then trying to convert one more time. (In reply to comment #1) > Are we doing the conversion off of a copy of our CVS repository? If yes, one > experiment that we could do is branching the remaining projects off for > 3_6_maintenance as well, and tagging projects with R3_6 that weren't tagged > (all of this on the CVS side, to the copied repository), and then trying to > convert one more time. That might be one option, but that will involve a fair bit of manual handwaving. We would need to find each tag that we care about, find its matching date/time, check out everything from CVS based on timestamp, tag everything not already tagged in each case (like the e4 bundles that graduated with 4.0), and branch them all if they weren't already at the branch point. PW And keep in mind that this doesn't just apply to 3.6 but to all the previous maintenance branches we want to convert....like 10 years worth of them :-) I'm going to write a note to the cvs2git mailing list about this once I have a few minutes after dealing with the Indigo release stuff. I'm looking through the cvs2git tool's issue tracker to see if any of the open issues match this problem, some seem similar http://cvs2svn.tigris.org/issues/buglist.cgi?Submit+query=Submit+query&component=cvs2svn&issue_status=NEW&issue_status=STARTED&issue_status=REOPENED&email1=&emailtype1=exact&emailassigned_to1=1&email2=&emailtype2=exact&emailreporter2=1&issueidtype=include&issue_id=&changedin=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&short_desc=&short_desc_type=fulltext&long_desc=&long_desc_type=fulltext&issue_file_loc=&issue_file_loc_type=fulltext&status_whiteboard=&status_whiteboard_type=fulltext&keywords=&keywords_type=anytokens&field0-0-0=noop&type0-0-0=noop&value0-0-0=&cmdtype=doit&order=Reuse+same+sort+as+last+time We're looking at cvs2svn to see if it can be told not to generate those "manufactured commits" . But the git commands to fix my R3_6_maintenance example are: The fix on master was commit 26d03f83969d9f7c60cf8f985f48dc1400473de4 Fix the R3_6 tag: git tag -f R3_6 26d03f83969d9f7c60cf8f985f48dc1400473de4 Fix the R3_6_maintenance branch: git rebase --onto 26d03f83969d9f7c60cf8f985f48dc1400473de4 \ a305cecb492d6d12513e5c0ba338bc9f51f0dc0a The rebase replays all of the R3_6_maintenance commits on top of the real commit on master. from http://stackoverflow.com/questions/3810348/setting-git-parent-pointer-to-a-different-parent PW As for changing the tool, here's a patch we can try to see if that gets rid of the delete commits (adapted from http://permalink.gmane.org/gmane.comp.version-control.subversion.cvs2svn.devel/2711 Issue 3) ---- Index: cvs2svn_lib/dvcs_common.py =================================================================== --- cvs2svn_lib/dvcs_common.py (revision 5360) +++ cvs2svn_lib/dvcs_common.py (working copy) @@ -278,10 +278,9 @@ source_node = self._mirror.get_old_lod_directory(source_lod, svn_revnum) except KeyError: raise InternalError('Source %r does not exist' % (source_lod,)) - return ( - set([cvs_symbol.cvs_file for cvs_symbol in cvs_symbols]) - == set(self._get_all_files(source_node)) - ) + set1 = set([cvs_symbol.cvs_file for cvs_symbol in cvs_symbols]) + set2 = set(self._get_all_files(source_node)) + return (set1 < set2) def _get_all_files(self, node): """Generate all of the CVSFiles under NODE.""" ---- I haven't tried it yet, but for the simple tag and branch cases it should not create the delete commits. Downside: We can't run the verification step without modifying that as well. PW A work in progress for moving tags on "manufactured by cvs2svn" commits back onto their parent.
#!/bin/bash
#
TAG=R3_6
T=$( git log -1 --format="format:%H:%an" $TAG )
if echo "$T" | grep cvs2svn >/dev/null; then
CMT=$( echo $T | cut -f1 -d: )
echo Found cvs2svn $CMT
if [ -z "$(git log -1 --merges --format="format:%h" $TAG)" ]; then
PT=$( git log -1 --format="format:%an" "${CMT}^1" )
if ! echo "$PT" | grep cvs2svn >/dev/null; then
echo git tag -f "$TAG" "${CMT}^1"
fi
fi
fi
Based on further analysis of what the cvs2git script does, I think that we need to: 1) run the cvs2git script and then run the verification. 2) pick the set of branches we care about, find their correct commits, and move the tag using "git tag -f". In Platform UI, I'd think, R3_1 R3_1_1 R3_1_2 R3_2 R3_2_1 R3_2_2 R3_3 R3_3_1 R3_3_1_1 R3_3_2 R3_4 R3_4_1 R3_4_2 R3_5 R3_5_1 R3_5_2 R3_6 R3_6_1 R3_6_2 Maybe some of the Root_* branch points. 3) pick the branches we care about, find their starting commits, and re-parent them using "git rebase --onto <newParent> <oldParent> perf_31x perf_32x perf_33x perf_34x R3_1_maintenance R3_2_maintenance R3_3_maintenance R3_4_1_maintenance_patches R3_4_maintenance R3_4_maintenance_patches R3_4_maintenance_patches_2 R3_5_maintenance R3_6_maintenance R3_6_maintenance_patches R4_HEAD That'll make our branches accurate, and we'll just ignore the rest of the manufactured commits. PW Created attachment 198496 [details] Fix tags report script This script scans through the tags and generates three types of lines: A tag that's probably applied correctly: Tag clean: Bug182059_before_HEAD_rebase commit: 4e9b17ddda7d26513ae89a265af4d131b3c98e8c A tag that can be fixed with a simple command looks like: git tag -f I200060718-0800 92449d402cf8102c457dd97041200694bfeefc07^1 A manufactured commit that includes source from another location looks like: Cherrypick tag: v20110614-1530 commit: 434167c0a45e445c4febc9f5c21246a2caeeccf1 If it's important to fix this tag, a developer would have to track down the "cherrypick" parents and try and resolve the commit manually: git log -1 --format="format:remove:%H:%an:%b" 434167c0a45e445c4febc9f5c21246a2caeeccf1 | grep Cherrypick Cherrypick from master 2010-06-03 14:58:28 UTC Boris Bokowski <bbokowski@eclipse.org> Cherrypick from R4_HEAD 2011-06-14 17:45:22 UTC Paul Webster <pwebster@eclipse.org> Created attachment 198532 [details]
Skip branch commits with parent filter script
This will skip the delete commits at the beginning of most branches. But it doesn't do what we want: The commit goes away, but the delete changes are just pulled up into the child commit.
PW
(In reply to comment #6) > As for changing the tool, here's a patch we can try to see if that gets rid of > the delete commits (adapted from This was a bad idea, it trashed the repo. PW One way to make the secondary tags like R3_6_1 and R3_6_2 is to generate a symbol-info.txt file and then modify it to be a symbol-hints.txt file: -0 R3_6_2 tag . .trunk. -0 R3_6_1 tag . .trunk. +0 R3_6_2 tag . R3_6_maintenance +0 R3_6_1 tag . R3_6_maintenance 0 R3_6 tag . .trunk. Then those tags are generated as manufactured commits off of something appropriate in the branch, as opposed to franken-built off of master. PW From what you've said cvs2git is doing the right thing. The stuff that wasn't tagged really shouldn't be part of the git tag. Even if you remove the 'delete', what version of the missing files do you choose? At the time of the branch, clearly the version on HEAD at that time is appropriate. However as soon as HEAD moves on, won't history diverge? I.e. When doing the maintenance branch build, what version of the non-tagged stuff do you use? (In reply to comment #13) > Even if you remove the 'delete', what version of the missing files do you > choose? At the time of the branch, clearly the version on HEAD at that time is > appropriate. However as soon as HEAD moves on, won't history diverge? I.e. When > doing the maintenance branch build, what version of the non-tagged stuff do you > use? What our git repo *should* look like: A--R3_6--B--C--master \ ----D--E--R3_6_1--F--G--R3_6_2--H--I--R3_6_maintenance Because we tagged platform UI R3_6 but then only branched 12 of 42 projects, the D commit I don't like deletes 30 projects (but they're supposed to be visible as the R3_6 commit while on R3_6_maintenance). The R3_6 branch point is correct. I'm testing a CVS pre-conditioning. If I make sure that all the projects are tagged R3_6 and all of the projects are branched, cvs2git produces a much better history for the tags and branches I care about (without D as a delete commit). The other thing I need to try is to convert the e4 graduated repo by itself, and then stitch it into the platform ui repo (probably from a different branch). PW I've pushed my steps and my change scripts to a public repo: git://git.eclipse.org/gitroot/e4/org.eclipse.migration.git I wrote down what I did this time in scripts/cvs2git_prep.txt. Most of my generated files went into eclipse.platform.ui/pass2. I've pushed the temporary Platform UI repo to git://git.eclipse.org/gitroot/platform/eclipse.platform.ui.git ... it looks much better, except I should re-run my id scanner to pick up Chris G :-) PW OK, pre-conditioning the repo was the best option to get decent looking branches for 3.1-3.6. My scripts and steps are published in git://git.eclipse.org/gitroot/e4/org.eclipse.migration.git PW Paul, are you doing the pre-conditioning in the CVS master repository? If so do I need to do do anything to precondition the three UI projects ( two if you don't include org.eclipse.ui.forms.examples which does not go into the build and does not have branches ). (In reply to comment #17) > Paul, are you doing the pre-conditioning in the CVS master repository? If so do > I need to do do anything to precondition the three UI projects ( two if you > don't include org.eclipse.ui.forms.examples which does not go into the build > and does not have branches ). Check out git://git.eclipse.org/gitroot/platform/eclipse.platform.ui.git ... I'm pretty sure I've got all 3 projects in there already. I made them match when I did our projects, like org.eclipse.jface.snippets. PW |