Bugzilla – Full Text Bug Listing
| Summary: | migrate platform team to git | ||
|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Kim Moir <kim.moir> |
| Component: | Team | Assignee: | Platform Team Inbox <platform-team-inbox> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | P3 | CC: | daniel_megert, john.arthorne, malgorzata.tomczyk, pwebster, remy.suen, Szymon.Brandys, tomasz.zarna |
| Version: | 3.8 | ||
| Target Milestone: | 3.8 M3 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Bug Depends on: | 359030, 359067, 359462, 359557 | ||
| Bug Blocks: | 345479, 359032, 359033 | ||
|
Description
Kim Moir
Your test repo is ready userid@git.eclipse.org:/gitroot/platform/eclipse.platform.team.git/ Please let me know if you see any issues and let me know when you'd like to schedule the actual migration. Right now, we are waiting until after 3.7.1 is ready for further migrations. We should group projects into bundles, tests, examples like for other repos. I wonder if there is a chance to move org.eclipse.compare, org.eclipse.comapre.core and org.eclipse.compare.win32 to top-level in Git repo? By waiting until after 3.7.1 is reaqy you mean Aug 7, right? Okay, I'll group them into bundles, tests, examples for the real migration. Why do you want compare in a separate repo? We are trying to group the bundles into the same repos based on the unix group and they have the same group as the other bundles in the repo. The repo is only 26MB too so don't think size is a reason to split the repo :-) (In reply to comment #3) > Why do you want compare in a separate repo? Not a separate repo. I meant to just move these three compare plug-ins to the bundles folder in the team repo. Right now they all three would be in bundles/org.eclipse.compare/plugins. So to clarify, you would like the projects arranged as follows for the actual migration bundles: org.eclipse.compare org.eclipse.core.net org.eclipse.jsch.core org.eclipse.jsch.ui org.eclipse.team.core org.eclipse.team.cvs.core org.eclipse.team.cvs.ssh org.eclipse.team.cvs.ssh2 org.eclipse.team.cvs.ui org.eclipse.team.ui examples: org.eclipse.compare.examples org.eclipse.compare.examples.xml org.eclipse.team.examples.filesystem tests org.eclipse.compare.tests org.eclipse.core.tests.net org.eclipse.jsch.tests org.eclipse.team.tests.core org.eclipse.team.tests.cvs.core features org.eclipse.cvs-feature Is this correct? I talked to Kim about the compare plug-in issue. In our cvs repo there is org.eclipse.compare. And below is its structure: org.eclipse.compare <-- not a plug-in |_ plugins |_ org.eclips.compare <-- plug-in |_ org.eclipse.compare.core <-- plug-in |_ org.eclipse.compare.win32 <-- plug-in So my point in comment 2 was to move these three plug-ins two levels up. I think it is clear now. I asked Kim to run another test migration with projects grouped into bundles, tests, examples and features and 3 compare plug-ins in the bundles folder. Szymon, I'm trying that now have noted that org.eclipse.compare <-- not a plug-in (was really once a plugin and has history) |_ plugins |_ org.eclips.compare <-- plug-in |_ org.eclipse.compare.core <-- plug-in |_ org.eclipse.compare.win32 <-- plug-in I guess you don't care about this history before the bundle was split? Okay your test repo is ready in the new format. I'd suggest removing your local repo and then adding the repo again. (In reply to comment #7) > Szymon, I'm trying that now have noted that > > org.eclipse.compare <-- not a plug-in (was really once a plugin and has > history) > |_ plugins > |_ org.eclips.compare <-- plug-in > |_ org.eclipse.compare.core <-- plug-in > |_ org.eclipse.compare.win32 <-- plug-in > > I guess you don't care about this history before the bundle was split? I would expect that the history for org.eclipse.compare <-- not a plug-in (was really once a plugin and has history) was moved to plugins/org.eclipse.compare <-- plug-in. (In reply to comment #8) > Okay your test repo is ready in the new format. I'd suggest removing your local > repo and then adding the repo again. Thanks Kim. We'll check the new repo today/tomorrow. Szymon, The way that the conversion works is that I copy the portion of the CVS repository that needs to be copied to a temporary location and run the conversion tools on it. That being said, the fact that you want org.eclipse.compare in plugins/ both the old and new content is problematic. If I copy the new version of org.eclipse.compare to plugins/ and then copy the old content to org.eclipse.compare, some files will be overwritten and the conversion won't have the content you would expect. If we want to preserve history, perhaps we should just keep the compare plugin (old version) in it's current location and move things around when it's in the git repo. I've heard it's easy to move things around once we have the content in git. I've looked thought the repository today and it looks good: the projects are in the correct folders, the history is complete, the tests pass. I have only two comments: 1. .cvsignore files were migrated to git, I think they may safely be removed as they are not needed anymore 2. Team git repo has 164 branches and there are only 117 branches found for the same projects in cvs. An example of an extra branch is "AttemptAtBug17940". I couldn't find it anywhere in CVS. (In reply to comment #10) > If we want to preserve history, perhaps we should just keep the compare plugin > (old version) in it's current location and move things around when it's in the > git repo. I've heard it's easy to move things around once we have the content > in git. Ok, I agree. Let's keep the compare plugin (old version) in it's current location. We will move things around once we migrate to git. Could we run the migration again and prepare a test Git repo with compare plug-ins in old locations? As we discussed on IM we just run final migration we keep the structure as it is in the cvs repo now we add bundled, tests, examples etc. folders .cvsignore files are migrated too, this isn't needed Szymon, when do you want me to do the git migration? I've updated the build schedule with git migrations for this week and next. http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html Your migrated repo is ready here userid@git.eclipse.org:/gitroot/platform/eclipse.platform.team.git/ (In reply to comment #12) > Ok, I agree. Let's keep the compare plugin (old version) in it's current > location. We will move things around once we migrate to git. It's bug 359032 for bundles under org.eclipse.compare and bug 359033 for fragments under org.eclipse.core.net. Red alert: the repo is missing org.eclipse.ui.net which is expected to be under bundles/org.eclipse.ui.net! I thought it has been migrated to eclipse.platform.ui by mistake, but it's not there either. In the map file, I will temporarily point to the old CVS location for the missing bundle. My apologies, I missed it. I have to do some reading on how to stitch it into the repo and preserve the history. So today, I tried to make a new repo (in a test location) using git-stitch-repo. http://stackoverflow.com/questions/277029/combining-multiple-git-repositories https://metacpan.org/module/git-stitch-repo It's in userid@git.eclipse.org:/gitroot/platform/eclipse.platform.team.test.git/ Unfortunately, it looks like the tags and branches have A or B appended to them, depending on where they came from. Does anyone have a better strategy on how to merge a bundle into a repo while preserving history? John, do you have any suggestions? I stitched many svn2git repos into one for Linux Tools. I didn't find it to be very easy and it required lots of manual cleanup at the end. It worked in the end but I'm not sure I'd recommend it if your goal isn't to have a history that makes it appear the projects were all one since their inception. I'm not very familiar with this particular situation but you could perhaps look into grafting the histories together. I've never done this but I know it's possible and I think Paul has done something like this in this past. (In reply to comment #20) > I'm not very familiar with this particular situation but you could perhaps look > into grafting the histories together. I've never done this but I know it's > possible and I think Paul has done something like this in this past. The stitching looked to be fairly difficult. The best thing you could do is to set up your entire CVS copy like you want it to be. Massage the history before you do the conversion. In my case that wasn't possible either, so I created the 2 repos separately, and then fetched the entire history of my e4 repo into our eclipse.platform.ui repo (non-overlapping directories). Then I did a merge from the e4/master into origin/master. Of course that means I can only see both repos going forward, if I go back to R0_9 I have to pick either platform ui or e4, or create a new branch with a merge node of some kind. PW Thanks Paul and Andrew. Paul, I'll have to try that tomorrow. I have been reading bug 349177, I think this is where you did this for e4. (In reply to comment #22) > Thanks Paul and Andrew. Paul, I'll have to try that tomorrow. I have been > reading bug 349177, I think this is where you did this for e4. See http://git.eclipse.org/c/e4/org.eclipse.migration.git/tree/scripts/migration_script.sh#n876 That's where I added the e4 runtime repo to the eclipse.platform.runtime repo. PW Are we just talking about org.eclipse.ui.net being missing? My $0.02, if this is proving to be very difficult, and/or will make the git history convoluted, I'm not sure it is worth it. The other option would be to dump the correct state of the plugin for each of the main branches as a new commit at the end of the history. This is a very small plugin, and the most complex code is a preference page. It doesn't seem the end of the world to lose the complete history as long as our master and maintenance branches have the correct state. This kind of thing happened all the time in the past when files were renamed or moved across packages, etc. Szymon/Tomasz what do you think? Fine with me, we need to remember that the tag in the map file (i.e. I20110511-0800) will be no longer valid for said bundle. Another option could be to restart the migration, I don't mind committing/tagging our stuff again. I will just need to know which commits to redo. It doesn't look like there have been many changes since the migration.
kmoir@build:/gitroot/platform/eclipse.platform.team.git> git log
commit b8b5cf1989da59c541670cb08eab66dc62780d32
Author: Dani Megert <dmegert>
Date: Wed Sep 28 14:29:57 2011 +0200
Fixed .gitignore
commit 8afabc748e2b34711690564a9b390a43c3f5a295
Author: Tomasz Zarna <Tomasz.Zarna@pl.ibm.com>
Date: Mon Sep 26 18:25:55 2011 +0200
bug 358392: Compare editor does not show author if opened from
Synchronize view (NPE)
If I rerun the migration, will you be okay to release these commits again?
(In reply to comment #26) > If I rerun the migration, will you be okay to release these commits again? Yes. Please rerun. Let us know when you are planning to do it, so we can notify team/compare devs. And I recommend fixing the gitignore during the new migration: bin/ ==> /*/*/bin/ target ==> Just remove this line entirely - we don't use Maven This way it will be correct in all branches (In reply to comment #28) > And I recommend fixing the gitignore during the new migration: > > bin/ ==> /*/*/bin/ > target ==> Just remove this line entirely - we don't use Maven > > This way it will be correct in all branches And fix the following tag, so that we don't get Bug 359067: V20030310-precopyrightupdate and v20030310-precopyrightupdate cause error on Windows back. Will I be able to only fetch from upstream or do I have to re-clone the repository? If we migrate again you would need to re-clone. All the commits will have new hashcodes, so it will appear like your local clone has nothing in common with the new remote. (In reply to comment #29) > Will I be able to only fetch from upstream or do I have to re-clone the > repository? I think you could fetch with "--force" to override the check John mentioned, but I would re-clone, to be 100% the clone is in the right state. The repo is almost ready. I just tried to delete the old repo and couldn't because to file permission issues. I've opened a bug with the webmaster. eclipse.platform.team.test.git should probably also be deleted.
> And fix the following tag, so that we don't get
>
> Bug 359067: V20030310-precopyrightupdate and v20030310-precopyrightupdate cause
> error on Windows
>
> back.
This did not happen. Tomek is fixing this right now.
>The repo is almost ready.
Looks like the ACL is not set on the repo yet - we can't push.
Not sure if this is just me, but after re-cloning the repo and importing its projects into Eclipse "git status" informs about 3 new folders: $ git status -s ?? bundles/org.eclipse.compare/plugins/org.eclipse.compare.core/bin/ ?? bundles/org.eclipse.compare/plugins/org.eclipse.compare.win32/bin/ ?? bundles/org.eclipse.compare/plugins/org.eclipse.compare/bin/ The gitignore was changed to this /*/*/bin/ *~ *.rej *.bak javacore.* heapdump.* core.* Snap.* because that's what it was in the previous repo. If this is a problem, please let know and I'll rerun the filter-branch operation. Also, I'll ask the webmaster to fix the ACLs on the repo, sorry about that. Well, until bug 359032 is fixed "/*/*/bin/" won't match to bin folders mentioned in comment 35. Can't we just use "bin" to ignore these folders at any level? Tomasz, I can run the git-filter branch operation on the repo again to add back bin to gitingore. (Takes several hours) However, this means the hashes will change and you'll have to reclone the repo. Is this okay? I don't think we need to run the op again. I'm fine with fixing the .gitignore with a commit once the migration is finally done. I think the ACL is fixed so you should be able to commit again and the missing ui.net bundle should be there. Please update the map files to point to the correct location for ui.net. (In reply to comment #40) > Please update the map files to point to the correct location for ui.net. Done. I've renamed the duplicated tag, committed the change from comment 26 and tagged it and finally updated .gitignore by temporarily adding "/*/*/*/*/bin/" (to be removed with bug 359032). All pushed, so ACL works for me. Okay, thanks, I'm going to close this bug, hopefully it will stay that way :-) (In reply to comment #42) > Okay, thanks, I'm going to close this bug, hopefully it will stay that way :-) No - sorry Kim. The repo is not ready. I started to use it (again), checked out projects just to learn that further reorgs to that repo are planned (bug 359032) - yes partially my fault not reading all comments in this bug. This means my workspaces (yes I have more than one) will be doomed again. > Done. I've renamed the duplicated tag, committed the change from comment 26 and > tagged it and finally updated .gitignore by temporarily adding "/*/*/*/*/bin/" > (to be removed with bug 359032). All pushed, so ACL works for me. No, not done. The duplicate tags are still there, see http://git.eclipse.org/c/platform/eclipse.platform.team.git/refs/ Also, we should not add wrong top-level Git ignore rules, even if temporarily. Please move the "/*/*/*/*/bin/" to the folder it belongs. . I've fixed the tag and the .gitignore problem so that we can close this bug and make Kim happy :-). We can discuss the remaining issue in bug 359032. |