Community
Participate
Working Groups
Hi, I'm happy to announce that project Xtext wants to migrate from CVS to GIT. We haven't agreed on project structure and schedule yet, but a questions came up: The wiki (http://wiki.eclipse.org/Git/Migrating_to_Git) tells that we can choose whether we want import our history to Git or not. I did an experimental import of our repository to git myself and noticed that our history (the .git directory) already has a size of roughly 170MB. Therefore, my question is, whether it would be possible to create two repositories for us: - One repository for us to work with: It should contain what currently is CVS HEAD, but no history. So there wont be the need to download 170MB with each "git clone". - One read-only repository containing the complete history of our CVS repository. We will need it infrequently, but if we need it, for example when a bug report refers to an older version, it's important to have it available. It would be good to have a web front-end for this repository. The advantage over CVS and it's ViewCVS would be that we can browse it based on commits (changesets). With regards to us planning the schedule, how much time do you (the webmaster) need for planning/preparation?
I just played around with git://dev.eclipse.org/org.eclipse.tmf/org.eclipse.tmf.git and noticed that there are files/changesets missing. I fixed some of this in http://git.softeys.de/org.eclipse.tmf.git/commitdiff/c6488279afa305f2e42c014c4d5ba48da7dfee7a but there are more errors. I guess only a diff with the version from CVS can tell what's actually missing. http://git.softeys.de/org.eclipse.tmf.git is a private server of mine. I experienced similar problems when importing from CVS to git myself. The command I used is: git cvsimport -p -x -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling -v org.eclipse.tmf/org.eclipse.xtext Is this a know problem? If there is no known solution I guess the best way to handle this is to do a file-based comparison of the imported git working copy and the CVS and than merge all missing changes from CVS to git.
Let's get specific. We'd like to migrate as soon as possible. However, we won't be able to do a "hard" switch at the moment. Therefore, we would like to have read/writable git-repositories for the projects below and keep out CVS writable. We don't need the CVS history in git for now, importing HEAD will do just fine. This will give us time to - get our integration builds running based on git - ensure the tooling (EGit) works as we need it or find alternatives for certain tasks. - ensure all committers become familiar with git. During this time CVS will remain our production repository. When we've figured out the things listed above, we'd like to dump all contents from the git repositories and import again from CVS, including its history since the Helios release. It will be helpful to have the Helios version in git because many support requests during the next year will relate to it. I don't think it is reasonable to import more of the history than necessary since it bloats the repository and importing from CVS has proven to be error-prone. So, I'd like to ask you (the webmaster) to create these repositories for us: - org.eclipse.xtext.git - org.eclipse.xtext.releng.git - org.eclipse.emf.mwe.git - org.eclipse.emf.mwe.releng.git - org.eclipse.xpand.git - org.eclipse.xpand.releng.git And please import CVS HEAD from the following repositories: http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext/?root=Modeling_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.tmf/org.eclipse.xtext.releng/?root=Modeling_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.mwe/?root=Modeling_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.mwe.releng/?root=Modeling_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.xpand/?root=Modeling_Project http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.xpand.releng/?root=Modeling_Project What possibilities will we have to administrate the git repositories on our own? For example, will we be able to truncate change sets from the repository via "git reset --hard"?
Sven, since you are the project lead of Xtext, MWE and Xpand, could you please give a "+1" on this? (http://wiki.eclipse.org/Git/Migrating_to_Git#Open_a_Bug asks for it)
+1
(In reply to comment #2) > When we've figured out the things listed above, we'd like to dump all contents > from the git repositories and import again from CVS, including its history > since the Helios release Also, if you write to Git and to CVS, how will you manage the merges? Regarless, can't we (the webmasters) just do this once? I mean, pick a hard date, migrate, then go? > What possibilities will we have to administrate the git repositories on our > own? For example, will we be able to truncate change sets from the repository > via "git reset --hard"? If you, or a committer on your team, has shell access, you'll be able to do what you want -- as long as do not rewrite history or mess with the update hook.
Hi Denis, Good so see you're back from vacation. (In reply to comment #5) > (In reply to comment #2) > > When we've figured out the things listed above, we'd like to dump all contents > > from the git repositories and import again from CVS, including its history > > since the Helios release > > Also, if you write to Git and to CVS, how will you manage the merges? We don't want to merge. We want to have a writeable git repository for some time as a playground to make sure our build is running and we feel comfortable with the tooling. Then, after some time, we intend to throw away this repository and start with a fresh import from CVS. So, in the time where cvs and git co-exist, we'll have to commit our change sets to both repositories. However, this will only be true for a few change sets, since cvs l will be our primary repository during this time. > Regarless, can't we (the webmasters) just do this once? I mean, pick a hard > date, migrate, then go? I don't think we can go without having this playground for a while. But if you say it's too much trouble maybe we can host it somewhere else. > > > What possibilities will we have to administrate the git repositories on our > > own? For example, will we be able to truncate change sets from the repository > > via "git reset --hard"? > > If you, or a committer on your team, has shell access, you'll be able to do > what you want -- as long as do not rewrite history or mess with the update > hook. Ok. I'm not interested in weakening the update hook ;) but what's wrong about rewriting the history as long as it doesn't add (possibly ip-unclean) code?
(In reply to comment #2) > So, I'd like to ask you (the webmaster) to create these repositories for us: > - org.eclipse.xtext.git > - org.eclipse.xtext.releng.git > - org.eclipse.emf.mwe.git > - org.eclipse.emf.mwe.releng.git > - org.eclipse.xpand.git > - org.eclipse.xpand.releng.git Those repos have been set up and imported. You can find all the clone URIs from cgit: http://git.eclipse.org/c/ Let me know how it looks. For my own reference, here's how I did xpand. Others were similar: 8350 mkdir m2t 8351 cd m2t 8352 ls -l 8353 export GIT_DIR=org.eclipse.xpand.git 8354 /usr/local/libexec/git-core/git-cvsimport -v -i -p -x -d :local:/home/data/cvs/modeling org.eclipse.m2t/org.eclipse.xpand -C ./ 8355 export GIT_DIR=org.eclipse.xpand.releng.git 8356 /usr/local/libexec/git-core/git-cvsimport -v -i -p -x -d :local:/home/data/cvs/modeling org.eclipse.m2t/org.eclipse.xpand.releng -C ./ 8357 ls -l 8358 cd .. 8359 ls -l 8360 getent group | grep m2t 8361 chown sefftinge.modeling.m2t m2t 8362 chmod 2775 m2t 8363 cd m2t 8364 chown sefftinge.modeling.m2t.xpand org.eclipse.xpand.* -R 8365 ls -l 8366 find . -type d -exec chmod g+ws {} \;
hi Denis, thanks for the import! However, I tried to push to the repository and it failed. I'm a committer of project Xtext, so, I'd expect it to work. Is there maybe something wrong with permissions in the file system? --- $ git push Password: Counting objects: 9, done. Delta compression using up to 2 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 418 bytes, done. Total 5 (delta 2), reused 0 (delta 0) error: insufficient permission for adding an object to repository database ./objects fatal: failed to write object error: unpack failed: unpack-objects abnormal exit To ssh://meysholdt@git.eclipse.org/gitroot/tmf/org.eclipse.xtext.git ! [remote rejected] master -> master (n/a (unpacker error)) error: failed to push some refs to 'ssh://meysholdt@git.eclipse.org/gitroot/tmf/org.eclipse.xtext.git' --- my config is: $ git config --get user.name meysholdt $ git config --get user.email moritz.eysholdt@itemis.de
The good news is that the tests from org.eclipse.xtext.tests and org.eclipse.xtext.ui.tests pass. This means Xtext's imported source code is intact :)
(In reply to comment #8) > error: insufficient permission for adding an object to repository database > ./objects I had set file permissions to modeling.tmf, instead of modeling.tmf.xtext. Try in a few minutes, it should work.
Now it works. Thank you so much! The next tasks are now up to us committers: - keep CVS as the production repository - get the build running based on the git repositories. - evaluate the git tooling. - decide for a time when to re-import the git repositories and turn CVS read-only.
Hi Denis, we're now ready to do the final migration. On Wednesday, the 25th of August, could you please: - Replace the contents of the following repositories with a fresh import of their CVS-counterparts: http://git.eclipse.org/c/tmf/org.eclipse.xtext.git/ http://git.eclipse.org/c/emf/org.eclipse.emf.mwe.git/ http://git.eclipse.org/c/m2t/org.eclipse.xpand.git/ - Delete the following repositories. We thought that they would be necessary for the build, but now we know that we can live without them: http://git.eclipse.org/c/tmf/org.eclipse.xtext.releng.git/ http://git.eclipse.org/c/m2t/org.eclipse.xpand.releng.git/ http://git.eclipse.org/c/emf/org.eclipse.emf.mwe.releng.git/ After those steps, the git repositories will be our primary repositories to work with. Please leave the CVS-Repositories writable for now. We'd like to have them available in case something unexpected happens. Denis, can you please confirm that the schedule as planned works for you? regards, Moritz
> On Wednesday, the 25th of August, could you please: Yes, that works for me. I've marked it into my calendar for Aug 25/10:00am Eastern.
Great, thanks! For my colleagues: 10 a.m. EDT translates to 4 p.m. CEST (Kiel Standard Time)
(In reply to comment #0) > [...] I did an experimental > import of our repository to git myself and noticed that our history (the .git > directory) already has a size of roughly 170MB[...] You could also truncate the history while cloning if you're low on bandwidth and/or disk space (from http://www.kernel.org/pub/software/scm/git/docs/git-clone.html ): --depth <depth> Create a shallow clone with a history truncated to the specified number of revisions. A shallow repository has a number of limitations (you cannot clone or fetch from it, nor push from nor into it), but is adequate if you are only interested in the recent history of a large project with a long history, and would want to send in fixes as patches.
Thanks Daniel, but we prefer to keep the whole history. @Denis: Did you already start the import?
> @Denis: Did you already start the import? The import just completed without errors. Please go have a look and let me know how it looks.
(In reply to comment #17) > > @Denis: Did you already start the import? > > The import just completed without errors. Please go have a look and let me > know how it looks. I tried to push a tag and got the following feedback: Counting objects: 1, done. Writing objects: 100% (1/1), 314 bytes, done. Total 1 (delta 0), reused 0 (delta 0) error: insufficient permission for adding an object to repository database ./objects fatal: failed to write object error: unpack failed: unpack-objects abnormal exit To ssh://sefftinge@git.eclipse.org/gitroot/tmf/org.eclipse.xtext.git ! [remote rejected] v1.0.1 -> v1.0.1 (n/a (unpacker
(In reply to comment #16) > Thanks Daniel, but we prefer to keep the whole history. And I expected nothing less :) I just wanted to point out that contributors will still have the choice to truncate the history when cloning.
I forgot to set it as a shared repository. I've done this for all three, and I've fixed the permissions. You should be all set.
Thanks, works like a charm. If I didn't miss anything I think this bugzilla can be closed.
